Commit Graph

21 Commits

Author SHA1 Message Date
jordan
ddcfe52b5c feat: implement shared notify host model for platform email delivery
Some checks failed
ci/woodpecker/push/woodpecker Pipeline failed
Replace per-project notify host provisioning (7-9 API calls + DNS + async
Resend verification) with a shared platform host for all *.threesix.ai projects.

Under the new model:
- CreateProjectNotify: 3 calls only (account + send key + host grant)
- No per-project Resend domain, DNS records, or async verification
- All *.threesix.ai projects share `threesix.ai` as the platform host
- Custom domains still get a dedicated host via ReprovisionNotifyHost

Changes:
- domain/notify.go: slim NotifyCredentials (no Host/From/ResendDomainID);
  add NotifyHostCredentials for reprovision return path
- port/notify_provisioner.go: update interface signatures and docs
- adapter/notify/provisioner.go: rewrite CreateProjectNotify (3 steps);
  rewrite DeleteProjectNotify (account-only vs full cleanup)
- adapter/notify/provisioner_reprovision.go: return *NotifyHostCredentials
- adapter/notify/provisioner_test.go: update tests for new model
- service/project_infra_crud.go: store only NOTIFY_API_KEY on provision
- domain/credential.go: add CredKeyNotifySharedHost/CredKeyNotifySharedFrom
- cmd/rdev-api/config.go: add NotifySharedHost/NotifySharedFrom to InfraConfig
- service/component.go: add notifySharedHost/notifySharedFrom + WithNotifyDefaults
- service/component_deploy.go: inject shared host defaults when no custom host stored
- handlers/notify.go: handle shared-host projects in Reprovision guard;
  add WithSharedNotifyHost builder
- cmd/rdev-api/main.go: wire SharedHost to provisioner, component service,
  and notify handler

Bootstrap: NOTIFY_SHARED_HOST=threesix.ai and NOTIFY_SHARED_FROM=noreply@threesix.ai
stored in credential store (host id=1 already provisioned with Resend provider).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 17:04:11 -07:00
jordan
fa0d030def feat: improve notify domain verification reliability and add status endpoints
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
- Add verifyWithRetry to provisioner: 60s initial DNS propagation delay,
  5 retries with 30s backoff before marking verification as failed
- Add GetNotifyDomainStatus: polls Resend API for domain verification status,
  returns "not_configured" when Resend not set up
- Add VerifyProjectNotify: synchronous re-verification for handler use
- Add getDomainStatus to resendAPI interface + resendClient implementation
- Add NotifyDomainStatus domain struct (host, resend_domain_id, status)
- Guard NOTIFY_RESEND_DOMAIN_ID storage against empty string writes
- New handler: GET /projects/{id}/notify/status (returns verification state)
- New handler: POST /projects/{id}/notify/verify (triggers re-verification)
- Add verify-notify-domain cookbook step to persona-community,
  slackpath-1, and slackpath-4 trees (polls status for up to 6 min)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-23 16:25:55 -07:00
jordan
4f01015132 feat: implement project access enforcement and management API
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
- Fix no-op RequireProjectAccess middleware to enforce project_ids
- Apply project access middleware to all project-scoped routes
- Filter GET /projects by allowed project IDs for restricted keys
- Add GET /me endpoint with key identity, scopes, and project access info
- Add PATCH /keys/{id} for partial key updates (name, scopes, project_ids, allowed_ips, expires_in)
- Add GET/POST/DELETE /projects/{id}/access for project-centric access management
- Auto-grant creating key access when using POST /project/create-and-build
- Accept grant_to_key_ids in create-and-build to grant multiple keys on project creation
- Move newProvisionerWithDeps test helper from production code to test file

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-21 15:38:37 -07:00
jordan
0f25bd8dbe feat: hook in notify service for per-project email delivery
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
- Add NotifyProvisioner (port + adapter) using real notify admin API
- Create notify account + send key + host grant per project
- Inject NOTIFY_API_KEY/HOST/FROM into component deployments
- Store NOTIFY_URL, NOTIFY_ADMIN_KEY, RESEND_API_KEY in credential store
- Add setup-notify.sh for one-time host/provider/domain setup
- Add NOTIFY_ADMIN_KEY constant to domain/credential.go
- Wire provisioner in main.go with connection test guard
- Add .claude/guides/services/notify.md and CLAUDE.md entry

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-21 00:30:32 -07:00
jordan
a8c8a0a14d feat: add GCS-based persistent media storage, AI generation pipeline, and composable skeleton packages
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
Adds complete media storage pipeline with GCS presigned uploads, AI image/video/text generation
via queue-based workers, realtime SSE event streaming, and comprehensive skeleton packages
(storage, mediagen, textgen, generation, realtime, persona, routing, ai-client). Includes
security fixes for media delete authorization, nil pointer guards in handlers, video persistence
via download-then-upload, consistent signed URLs, and Image→ImageIcon rename to avoid DOM collision.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 21:29:09 -07:00
jordan
542bc722ab fix(architect): handle missing projects in repo, add cookbook hooks/validation
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
The architect API returned "failed to start conversation" because
projectRepo.Get() failed — the in-memory K8s repo watches the rdev
namespace but projects deploy to the projects namespace. Made project
lookup non-fatal with fallback to default pod. Added error logging to
all architect handler methods (were silently swallowing errors).

Also adds setup-hooks, commit-after-qa, and pre-merge-validate steps
to the foundary cookbook tree for git hooks and code quality gates.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 02:25:40 -07:00
jordan
9226454b85 feat: label-based undeploy, GC reconciliation, checkout/sessions, pool status
Some checks failed
ci/woodpecker/push/woodpecker Pipeline failed
- Add UndeployAll() using label selectors to clean up monorepo components
  on project deletion (replaces name-based Undeploy in DeleteProject and
  the direct undeploy handler)
- Add ResourceGC background worker that periodically finds K8s resources
  whose project label has no matching DB record, deletes after 1h safety
  window
- Widen deployer client type from *kubernetes.Clientset to
  kubernetes.Interface for testability
- UndeployAll accumulates errors via errors.Join instead of failing fast
- Add checkout/checkin sidecar dev flow: temporary git tokens, branch
  checkout, review on checkin with cleanup workers
- Add interactive sessions: pod binding, command execution, SSE streaming,
  ephemeral preview URLs with session cleanup workers
- Add GET /workers/pool endpoint for aggregate capacity and queue depth
- Add sessions:read and sessions:execute auth scopes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 19:11:28 -07:00
jordan
adcea2fc1f fix(templates): upgrade Go to 1.25 and fix Woodpecker syntax
Some checks failed
ci/woodpecker/push/woodpecker Pipeline failed
## Template Version Alignment
- Go: 1.23 → 1.25 across all templates (go.work, go.mod, Dockerfiles, CI)
- Alpine: latest → 3.19 (explicit version pinning)
- Woodpecker: failure:retry → failure:ignore (invalid syntax fix)

## SDLC Tree Fixes (slackpath-5-full-lifecycle)
Fixed merge failures by correcting lifecycle flow:

1. **Branch Creation**: Added missing create-branch step (planned → ready)
   - Bug: Merge command requires feature.Branch field to be set
   - Fix: POST /projects/{id}/sdlc/features/{slug}/branch

2. **Artifact Status**: Changed approval to pass for execution artifacts
   - Bug: Review/audit/QA need status="passed" not "approved"
   - Fix: /artifacts/{type}/approve → /artifacts/{type}/pass
   - Added: pass-qa step after wait-qa

3. **Phase Transition Order**: Reordered merge phase transition
   - Bug: Merge command checks if phase == "merge" first
   - Fix: transition-to-merge BEFORE merge-feature (not after)

## GCS Provisioner Fix
- Replaced deprecated option.WithCredentialsFile with env var approach
- Now uses GOOGLE_APPLICATION_CREDENTIALS for ADC (Application Default Credentials)
- Avoids security risk from deprecated credential options
- Fixed test: Added ComponentTypeGCS to ValidComponentTypes test

## Critical Rules Added
- Version alignment: All template versions must stay in sync
- When updating versions, grep entire templates/ tree

## Files Changed
- 27 template files: Go version + Woodpecker syntax
- 1 tree file: SDLC lifecycle flow corrections
- 1 CLAUDE.md: Version alignment rule
- 1 GCS provisioner: Deprecated API fix
- 1 test file: Added missing component type

Root cause: Skeleton templates lagged behind Go 1.25 release and had
invalid Woodpecker syntax. SDLC tree skipped required branch creation
and used wrong artifact approval endpoints.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-08 23:57:38 -07:00
jordan
4486042155 fix(registry): delete container images on project teardown
Some checks failed
ci/woodpecker/push/woodpecker Pipeline failed
Root cause of DIGEST_INVALID errors was registry disk exhaustion.
Project teardown wasn't cleaning up container images, causing the
registry PVC to fill up over time.

Changes:
- Add RegistryProvider port interface for registry operations
- Extend zot.Client with DeleteProjectRepositories method
- Wire registry provider into ProjectInfraService
- Delete images during DeleteProject cleanup (step 4)

The zot client uses the OCI distribution API:
- Lists all repos, filters by project prefix
- Gets manifest digests via HEAD request
- Deletes manifests by digest to trigger GC

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-08 02:56:18 -07:00
jordan
863dfd3214 fix: skip root deployment for empty template (defaults to skeleton)
Some checks failed
ci/woodpecker/push/woodpecker Pipeline failed
When req.Template is empty, it defaults to 'skeleton' but the check
in createInitialDeployment only matched 'skeleton' explicitly, not
empty string. This caused a broken deployment to be created for
monorepo projects with a non-existent image.

Root cause: slackpath-5 creates project with empty template, which
defaults to skeleton, but createInitialDeployment was still creating
a root deployment that references registry.threesix.ai/{project}:latest
which never gets built (skeleton has no root Dockerfile).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-07 19:32:19 -07:00
jordan
af91bad0ff feat: add Slate documentation templates to skeleton
Some checks failed
ci/woodpecker/push/woodpecker Pipeline failed
Adds complete Slate documentation infrastructure to generated projects:
- docs/ directory with Gemfile, config.rb, and source templates
- Dockerfile for building docs site
- Dockerfile.nginx for serving static docs
- generate-docs.sh script for CI integration
- Claude command for AI-assisted docs generation
- OpenAPI → Slate markdown conversion via widdershins

Also includes:
- --export-openapi flag for service binaries
- DNS provisioning for docs.{domain} subdomain
- Updated project_infra for docs DNS records

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-07 16:06:36 -07:00
jordan
853ec4cf81 fix: go.work race condition with batch components and idempotent provisioning
Three coordinated fixes for CI pipeline race conditions:

1. Woodpecker step dependencies: Added depends_on: [deps] to all 6 component
   templates (service, worker, cli, app-astro, app-react, app-nextjs) so build
   steps wait for go work sync to complete.

2. Idempotent resource provisioning: Modified provisionResources() to check
   for existing database/cache before creating, preventing "already exists"
   errors on component re-adds.

3. Batch component endpoint: POST /projects/{id}/components/batch enables
   atomic multi-component additions in a single git commit. Validates all
   components upfront, provisions infra sequentially, commits code components
   atomically.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 12:31:40 -07:00
jordan
d69da6d627 feat: add structured logging infrastructure and SDLC extensions
Major changes:
- Add internal/logging package with field constants, context propagation,
  sensitive data auto-redaction, and per-component log levels
- Add worker timeout constants (TimeoutQuickOp, TimeoutHealthCheck, etc.)
- Extend SDLC with callback handlers, generate endpoints, and executor
- Add new cookbook trees for aeries and slackpath progression
- Add skeleton templates for queue, realtime, and microservices
- Add worker component template with async job processing
- Refactor services and handlers to use new logging infrastructure
- Split component.go into component_infra.go and component_listing.go

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 22:56:04 -07:00
jordan
1790afd0ee feat: add path-based ingress management for component lifecycle
Adds AddIngressPath and RemoveIngressPath to the Deployer interface
for managing per-component ingress rules in monorepo projects.

- Implement conflict retry logic for concurrent ingress updates
- Add K8s client interface for testability
- Add comprehensive unit tests for ingress path operations
- Add component deployment and teardown methods to ComponentService
- Update service templates with OpenAPI spec improvements
- Add evolving-app cookbook tree for reference
- Split resources.go into resources_ingress.go for path-based routing
- Split component.go into component_deploy.go for deployment helpers

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 01:31:50 -07:00
jordan
572b221e20 feat: add automatic cleanup for cookbook test projects
- Add AUTO_TEARDOWN env var and --auto-teardown flag to cookbook scripts
- Scripts automatically delete created projects on exit (including Ctrl+C)
- Add DELETE /projects/cleanup API endpoint for bulk cleanup
- Supports shell-style glob patterns (e.g., "tree-test-*")
- Includes dry_run mode and older_than_hours filter for safety
- Requires admin scope for actual deletion
- Update cookbook scripts: landing-test, composable-test, template-validation,
  feature-test, tree-runner

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 17:54:15 -07:00
jordan
c280a92012 feat: add operations audit system and template improvements
Operations Audit (new feature):
- Add Operation domain model with status tracking (pending, running, completed, failed, cancelled)
- Add OperationRepository with PostgreSQL implementation
- Add OperationService for CRUD and lifecycle management
- Add operations handlers (list, get, cancel endpoints)
- Add migration 015_operations.sql for operations table
- Add operation cleanup worker for stale operation handling
- Add ErrOperationNotFound to domain errors

Template Improvements:
- Add CLAUDE.md configuration files to astro-landing, default, and go-api templates
- Fix PORT template variable usage in nginx configs for app templates
- Add replace directives for local pkg module in Go templates
- Simplify Go service/worker Dockerfiles for workspace builds
- Fix TypeScript error in logger template

Other:
- Refactor landing-test.sh cookbook script
- Update CLAUDE.md version reference

Note: Some files exceed 500-line limit (pre-existing debt + new feature)
- component.go: 550 lines (unchanged, pre-existing)
- main.go: 522 lines (added operations wiring)
- operation_repo.go: 569 lines (new, needs splitting)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 19:08:57 -07:00
jordan
c2b0447d80 feat: add per-component deploy steps and component templates endpoint
Add deploy-{name} CI steps to all component templates (app-astro,
app-react, service, worker) so each component deploys independently
via kubectl set image on merge to main. Replace the skeleton's
generic deploy step with a verify step that confirms deployments.

Add GET /templates/components endpoint for listing available component
templates with optional type filter. Simplify component API by merging
type+template into a single type field (e.g., "app-react" instead of
type="app" template="app-react").

Include ESLint configs and pnpm-workspace.yaml in app templates.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-31 22:31:41 -07:00
jordan
8282d60c69 feat: implement composable monorepo template system with component architecture
Adds the composable monorepo template system that generates project skeletons
with pluggable components (service, worker, app-react, app-astro, cli).

Key changes:
- Monorepo skeleton templates with shared pkg/, scripts/, and git hooks
- Component templates (service, worker, app-react, app-astro, cli) with
  Dockerfiles, CI steps, and component.yaml manifests
- Component domain model with validation and dependency resolution
- Component handler endpoints for CRUD and composition
- Template provider extended with BuildComposableProject and component assembly
- Deployer extended with composable project deployment support
- Handler timeout constants (TimeoutFastLookup through TimeoutLongRunning)
- envutil package for centralized env var reads with defaults
- api.DecodeJSON helper for standardized request body decoding
- Standardized response helpers (WriteBadRequest, WriteNotFound, etc.)
- Replaced fullstack-app cookbook with composable-app cookbook
- Hardened handler timeouts, logging, and error responses across all handlers

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-31 19:11:42 -07:00
jordan
c59d348040 chore: prepare for composable monorepo template implementation
This commit captures the current state before implementing the composable
monorepo template system. Key changes included:

Infrastructure:
- Add CockroachDB provisioner adapter for database provisioning
- Add Redis provisioner adapter for cache provisioning
- Add build events system with PostgreSQL storage
- Add WebSocket endpoint for real-time build progress

Code agent improvements:
- Fix Claude Code adapter to use default allowed tools instead of dangerously-skip-permissions
- Add context-aware stream closing for cancellation support
- Improve parser tests for edge cases

Build system:
- Add build event constants and metrics
- Remove deprecated git_operations.go (replaced by pod_git_operations.go)
- Add rollback logic for multi-step provisioning operations

Documentation:
- Add composable-monorepo feature documentation
- Add DNS/Cloudflare service documentation
- Update deployment and troubleshooting guides

Cookbooks:
- Add fullstack-app cookbook
- Refactor landing-test with shared library

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-31 11:39:28 -07:00
jordan
34e72687e6 feat: Complete automation gaps for repeatable project deployments
- Initial K8s deployment auto-creation during project creation
- DNS record upsert support (create or update existing records)
- Ingress host management for domain aliases (AddIngressHost/RemoveIngressHost)
- Woodpecker deployer RBAC manifest for CI deploy steps
- Single-commit template seeding via Gitea bulk file API

Closes automation gaps exposed during www.threesix.ai launch:
- Projects now auto-create K8s Deployment/Service/Ingress on creation
- Domain aliases automatically update both DNS and K8s ingress
- CI deploy steps work without manual RBAC setup
- Template seeding triggers only one CI pipeline (not per-file)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 15:18:31 -07:00
jordan
c86516c53a feat: Add multi-domain support with auto-generated slugs for landing page cookbook
Landing page cookbook implementation (Weeks 1-4):

Domain Infrastructure:
- Add project_domains table with migration (013_project_domains.sql)
- Add ProjectDomain model with domain types (primary_auto, primary_custom, alias)
- Add SlugGenerator and ProjectDomainRepository interfaces
- Implement postgres adapters for domain and slug management

Service Layer:
- Add domain CRUD methods to ProjectInfraService
- Generate 8-char random slugs for auto-domains
- Support custom subdomains during project creation
- Add site_live health check to project status
- Trigger CI build after template seeding

Handler Updates:
- Add DomainService interface and adapter pattern
- Rewrite domain handlers to use database-backed service
- Add proper error handling for duplicate/missing domains

CI Integration:
- Add TriggerBuild to CIProvider interface
- Implement TriggerBuild in Woodpecker adapter
- Manually trigger initial build after template seed

Cookbook & Scripts:
- Add landing-test.sh script for E2E testing
- Add release.sh for version releases
- Add logs.sh for quick log access

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 12:55:59 -07:00