fix: clarify database types across docs and fix video storage persistence

Two distinct fixes:

1. Database terminology: Make it crystal clear that generated projects use
   CockroachDB in production and PostgreSQL for local dev, while the rdev
   platform itself uses PostgreSQL. Updated 15 files across skeleton agents,
   component templates, cookbook trees, and platform docs.

2. Video storage: VideoHandler was ignoring vid.Data bytes (already downloaded
   by the Gemini adapter with auth) and re-downloading from the provider URL
   with a plain GET — which fails because Gemini URLs require API key auth.
   Now uses vid.Data first, falls back to downloadURL only for public URLs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
jordan 2026-02-19 23:13:21 -07:00
parent a8c8a0a14d
commit 592b2d5ec0
15 changed files with 62 additions and 23 deletions

View File

@ -14,6 +14,16 @@ Run Claude Code instances in isolated Kubernetes pods with REST API control. Ena
When discussing code: "add to **platform**" = edit rdev; "add to **skeleton**" = edit project templates. When discussing code: "add to **platform**" = edit rdev; "add to **skeleton**" = edit project templates.
### Database Rule
| Context | Database | Details |
|---------|----------|---------|
| **rdev platform** | PostgreSQL | API keys, audit logs, work queue, credentials (`internal/adapter/postgres/`) |
| **Generated projects (production)** | CockroachDB | Provisioned per-project by rdev (`internal/adapter/cockroach/`) |
| **Generated projects (local dev)** | PostgreSQL | Via docker-compose, wire-compatible with CockroachDB |
Both use `lib/pq` driver. The `type: postgres` component API provisions **CockroachDB** in production — the name is a legacy artifact. Skeleton SQL must be compatible with both PostgreSQL and CockroachDB.
## Find Your Guide ## Find Your Guide
| If you need to... | Read this | | If you need to... | Read this |

View File

@ -22,7 +22,7 @@ steps:
- domain: .data.domain - domain: .data.domain
add-db: add-db:
description: Add Postgres description: Add CockroachDB
depends_on: [create-project] depends_on: [create-project]
action: api action: api
method: POST method: POST

View File

@ -19,7 +19,7 @@ steps:
name: "{{ .vars.project_name }}" name: "{{ .vars.project_name }}"
description: "Foundary Studio: Task management with Kanban board" description: "Foundary Studio: Task management with Kanban board"
template: "skeleton" template: "skeleton"
prompt: "Set up the monorepo workspace. Ensure the root README describes a task management studio with Kanban board, REST API, and Postgres persistence." prompt: "Set up the monorepo workspace. Ensure the root README describes a task management studio with Kanban board, REST API, and CockroachDB persistence."
auto_commit: true auto_commit: true
auto_push: true auto_push: true
outputs: outputs:
@ -59,7 +59,7 @@ steps:
poll_interval: 5 poll_interval: 5
add-components: add-components:
description: "Add React frontend, API service, and Postgres database" description: "Add React frontend, API service, and CockroachDB database"
depends_on: [wait-setup-hooks] depends_on: [wait-setup-hooks]
action: api action: api
method: POST method: POST
@ -101,7 +101,7 @@ steps:
method: POST method: POST
endpoint: "/projects/{{ .outputs.create-project.project_id }}/architect/start" endpoint: "/projects/{{ .outputs.create-project.project_id }}/architect/start"
body: body:
prompt: "I want to build a task management studio. The product needs: 1) Core data models for Task, Project, Label, and Assignment entities with full CRUD stored in Postgres via studio-db, exposed as REST endpoints on studio-api. 2) A React frontend in studio-ui with a Kanban board (drag-and-drop columns: To Do, In Progress, Done), task creation/edit modals, and filtering by label and assignee. Propose the architecture and identify the two MVP features we should build." prompt: "I want to build a task management studio. The product needs: 1) Core data models for Task, Project, Label, and Assignment entities with full CRUD stored in CockroachDB via studio-db, exposed as REST endpoints on studio-api. 2) A React frontend in studio-ui with a Kanban board (drag-and-drop columns: To Do, In Progress, Done), task creation/edit modals, and filtering by label and assignee. Propose the architecture and identify the two MVP features we should build."
outputs: outputs:
- conversation_id: .data.id - conversation_id: .data.id
@ -143,7 +143,7 @@ steps:
- Assignment: id, task_id, label_id (many-to-many join) - Assignment: id, task_id, label_id (many-to-many join)
2. Database: Create SQL migrations for all tables with foreign keys and indexes. 2. Database: Create SQL migrations for all tables with foreign keys and indexes.
Use the studio-db Postgres connection (DATABASE_URL env var). Use the studio-db CockroachDB connection (DATABASE_URL env var).
3. Repository layer: Implement CRUD operations for each entity using sqlx. 3. Repository layer: Implement CRUD operations for each entity using sqlx.

View File

@ -21,7 +21,7 @@ steps:
- domain: .data.domain - domain: .data.domain
add-db: add-db:
description: Add PostgreSQL for user storage description: Add CockroachDB for user storage
depends_on: [create-project] depends_on: [create-project]
action: api action: api
method: POST method: POST

View File

@ -292,7 +292,7 @@ type MockStorage struct {
- Deletes service account and keys - Deletes service account and keys
**Orphan Prevention:** **Orphan Prevention:**
- Project deletion hook cleans up all infra (postgres, redis, gcs) - Project deletion hook cleans up all infra (CockroachDB, Redis, GCS)
- If cleanup fails, logs warning but continues (manual cleanup required) - If cleanup fails, logs warning but continues (manual cleanup required)
### Cost Management ### Cost Management
@ -398,5 +398,5 @@ type MockStorage struct {
- **GCS Client Docs:** https://cloud.google.com/go/docs/reference/cloud.google.com/go/storage/latest - **GCS Client Docs:** https://cloud.google.com/go/docs/reference/cloud.google.com/go/storage/latest
- **IAM Best Practices:** https://cloud.google.com/iam/docs/best-practices - **IAM Best Practices:** https://cloud.google.com/iam/docs/best-practices
- **Signed URLs:** https://cloud.google.com/storage/docs/access-control/signed-urls - **Signed URLs:** https://cloud.google.com/storage/docs/access-control/signed-urls
- **rdev Postgres Provisioner:** `internal/adapter/postgres/provisioner.go` - **rdev CockroachDB Provisioner:** `internal/adapter/cockroach/provisioner.go`
- **rdev Redis Provisioner:** `internal/adapter/redis/provisioner.go` - **rdev Redis Provisioner:** `internal/adapter/redis/provisioner.go`

View File

@ -18,4 +18,6 @@ AUTH_ENABLED=false
JWT_SECRET=dev-secret-change-in-production JWT_SECRET=dev-secret-change-in-production
# Database (if needed) # Database (if needed)
# Local dev: PostgreSQL via docker-compose. Production: CockroachDB (platform-provisioned).
# The postgres:// scheme works for both — CockroachDB is wire-compatible.
DATABASE_URL=postgres://dev:dev@localhost:5432/{{PROJECT_NAME}}?sslmode=disable DATABASE_URL=postgres://dev:dev@localhost:5432/{{PROJECT_NAME}}?sslmode=disable

View File

@ -10,6 +10,8 @@ LOG_LEVEL=debug
LOG_FORMAT=text LOG_FORMAT=text
# Database (required for job queue) # Database (required for job queue)
# Local dev: PostgreSQL via docker-compose. Production: CockroachDB (platform-provisioned).
# The postgres:// scheme works for both — CockroachDB is wire-compatible.
DATABASE_URL=postgres://dev:dev@localhost:5432/{{PROJECT_NAME}}?sslmode=disable DATABASE_URL=postgres://dev:dev@localhost:5432/{{PROJECT_NAME}}?sslmode=disable
# Worker # Worker

View File

@ -1,6 +1,6 @@
--- ---
name: database-architect name: database-architect
description: Database schema design and query optimization for {{PROJECT_NAME}} - PostgreSQL, migrations, indexing description: Database schema design and query optimization for {{PROJECT_NAME}} - CockroachDB (production), PostgreSQL (local dev), migrations, indexing
color: yellow color: yellow
--- ---
@ -10,11 +10,16 @@ You design database schemas and optimize queries for {{PROJECT_NAME}}. Every ser
## Stack ## Stack
- **Primary:** PostgreSQL - **Production:** CockroachDB (distributed SQL, provisioned by the platform)
- **Driver:** sqlx (no GORM) - **Local dev:** PostgreSQL via docker-compose (wire-compatible with CockroachDB)
- **Driver:** sqlx with lib/pq (no GORM) — works with both PostgreSQL and CockroachDB
- **Migrations:** Per-service in `services/{name}/migrations/` - **Migrations:** Per-service in `services/{name}/migrations/`
- **Naming:** snake_case for tables and columns - **Naming:** snake_case for tables and columns
> **Important:** Write SQL that is compatible with both PostgreSQL and CockroachDB.
> Avoid PostgreSQL-specific features not supported by CockroachDB (e.g., advisory locks, listen/notify, full-text search with tsvector).
> Use `UUID` primary keys (CockroachDB handles these efficiently with no hotspotting).
## Schema Conventions ## Schema Conventions
### Tables ### Tables

View File

@ -1,6 +1,6 @@
--- ---
name: queue-specialist name: queue-specialist
description: Async job processing patterns for {{PROJECT_NAME}} - PostgreSQL queues, producer/consumer, retry logic, idempotency description: Async job processing patterns for {{PROJECT_NAME}} - SQL queues (CockroachDB/PostgreSQL), producer/consumer, retry logic, idempotency
color: purple color: purple
--- ---

View File

@ -11,7 +11,7 @@ You design and implement background workers for {{PROJECT_NAME}}. Workers are re
## Worker Types ## Worker Types
### Queue Consumer ### Queue Consumer
Processes jobs from a queue (PostgreSQL SKIP LOCKED, Redis, etc.): Processes jobs from a queue (CockroachDB/PostgreSQL SKIP LOCKED, Redis, etc.):
```go ```go
func (w *Worker) Run(ctx context.Context) error { func (w *Worker) Run(ctx context.Context) error {
for { for {

View File

@ -29,7 +29,7 @@ You are a librarian who transforms ephemeral conversation knowledge into permane
| `architecture/` | System design facts | "How the work queue flows" | | `architecture/` | System design facts | "How the work queue flows" |
| `debugging/` | How to diagnose issues | "How to debug pod execution" | | `debugging/` | How to diagnose issues | "How to debug pod execution" |
| `conventions/` | Naming, style, standards | "Error type naming convention" | | `conventions/` | Naming, style, standards | "Error type naming convention" |
| `integrations/` | External system knowledge | "How we talk to PostgreSQL" | | `integrations/` | External system knowledge | "How we talk to CockroachDB" |
## Storage Structure ## Storage Structure

View File

@ -38,6 +38,7 @@
- **OpenAPI first:** Document endpoints in `spec.go` using `openapi.*` helpers. Mount with `application.EnableDocs(spec)`. - **OpenAPI first:** Document endpoints in `spec.go` using `openapi.*` helpers. Mount with `application.EnableDocs(spec)`.
- **CSS variables:** All UI components use CSS custom properties (`var(--background)`, `var(--accent)`, etc.). Never hardcode colors. - **CSS variables:** All UI components use CSS custom properties (`var(--background)`, `var(--accent)`, etc.). Never hardcode colors.
- **Monorepo imports:** Go packages from `{{GO_MODULE}}/pkg/*`, TypeScript from `@{{PROJECT_NAME}}/*`. - **Monorepo imports:** Go packages from `{{GO_MODULE}}/pkg/*`, TypeScript from `@{{PROJECT_NAME}}/*`.
- **Database:** Production uses **CockroachDB** (provisioned by the platform). Local dev uses **PostgreSQL** via docker-compose. Both are wire-compatible via `lib/pq`. Write SQL compatible with both — avoid PostgreSQL-only features (advisory locks, listen/notify, tsvector).
- **NO WEBSOCKETS. EVER.** All real-time communication uses HTTP2 + SSE. User → server is HTTP2 POST. Server → user is SSE. This includes chat, notifications, progress, everything. - **NO WEBSOCKETS. EVER.** All real-time communication uses HTTP2 + SSE. User → server is HTTP2 POST. Server → user is SSE. This includes chat, notifications, progress, everything.
- **Event flow:** `POST → Service (enqueue) → Queue → Worker (generate) → Redis pub/sub → Service SSE subscriber → User`. Service is thin, worker does AI work. - **Event flow:** `POST → Service (enqueue) → Queue → Worker (generate) → Redis pub/sub → Service SSE subscriber → User`. Service is thin, worker does AI work.
- **Channel naming:** `user:<id>` = events for a specific user. `channel:<id>` = events for a topic/room/resource. Document all channels in `./docs/channels.md`. - **Channel naming:** `user:<id>` = events for a specific user. `channel:<id>` = events for a topic/room/resource. Document all channels in `./docs/channels.md`.

View File

@ -1,5 +1,8 @@
version: '3.8' version: '3.8'
# Local development uses PostgreSQL for convenience.
# Production uses CockroachDB (provisioned by the platform).
# Both are wire-compatible — code using lib/pq works with either.
services: services:
postgres: postgres:
image: postgres:16 image: postgres:16

View File

@ -1,4 +1,8 @@
// Package database provides a standardized PostgreSQL/CockroachDB connection pool. // Package database provides a standardized database connection pool.
//
// Production uses CockroachDB (provisioned by the platform).
// Local development uses PostgreSQL via docker-compose.
// Both are wire-compatible and use the lib/pq driver ("postgres").
// //
// This package wraps sqlx to provide: // This package wraps sqlx to provide:
// - Connection pool management with sensible defaults // - Connection pool management with sensible defaults
@ -31,7 +35,7 @@ import (
"time" "time"
"github.com/jmoiron/sqlx" "github.com/jmoiron/sqlx"
_ "github.com/lib/pq" // PostgreSQL/CockroachDB driver _ "github.com/lib/pq" // PostgreSQL-compatible driver (works with both PostgreSQL and CockroachDB)
) )
// Pool wraps a sqlx.DB with additional lifecycle management. // Pool wraps a sqlx.DB with additional lifecycle management.
@ -64,7 +68,7 @@ type Options struct {
} }
// Connect establishes a connection pool to the database. // Connect establishes a connection pool to the database.
// The URL should be a PostgreSQL connection string: // The URL should be a PostgreSQL-compatible connection string (works with CockroachDB):
// //
// postgres://user:pass@host:port/dbname?sslmode=disable // postgres://user:pass@host:port/dbname?sslmode=disable
func Connect(ctx context.Context, url string, opts Options) (*Pool, error) { func Connect(ctx context.Context, url string, opts Options) (*Pool, error) {

View File

@ -179,13 +179,25 @@ func VideoHandler(mg *mediagen.Manager, store storage.Store, pub realtime.EventP
for i, vid := range resp.Videos { for i, vid := range resp.Videos {
videoURL := vid.URL videoURL := vid.URL
// Persist to storage: download from provider URL, then upload to GCS. // Persist to storage if available.
if store != nil && vid.URL != "" { // Prefer vid.Data (already downloaded by provider adapter) over re-downloading from URL.
// Provider URLs (e.g., Gemini API) often require authentication and fail with plain GET.
if store != nil {
storagePath := fmt.Sprintf("media/%s/videos/%s_%d.mp4", userID, job.ID, i) storagePath := fmt.Sprintf("media/%s/videos/%s_%d.mp4", userID, job.ID, i)
videoData, downloadErr := downloadURL(ctx, vid.URL)
if downloadErr != nil { var videoData []byte
logger.Warn("failed to download video from provider", "error", downloadErr, "job_id", job.ID) if len(vid.Data) > 0 {
} else { videoData = vid.Data
} else if vid.URL != "" {
downloaded, downloadErr := downloadURL(ctx, vid.URL)
if downloadErr != nil {
logger.Warn("failed to download video from provider", "error", downloadErr, "job_id", job.ID)
} else {
videoData = downloaded
}
}
if len(videoData) > 0 {
persistedURL, uploadErr := store.Upload(ctx, storagePath, videoData, "video/mp4") persistedURL, uploadErr := store.Upload(ctx, storagePath, videoData, "video/mp4")
if uploadErr != nil { if uploadErr != nil {
logger.Warn("failed to persist video to storage", "error", uploadErr, "job_id", job.ID) logger.Warn("failed to persist video to storage", "error", uploadErr, "job_id", job.ID)