rdev/.claude/guides/services/composable-monorepo.md

Composable Monorepo Templates

When to use: Implementing the monorepo skeleton, component templates, or the component addition API.

Overview

Composable Monorepo Templates evolve rdev from single-template projects to full monorepo scaffolding:

1. Skeleton: Every POST /projects creates the monorepo base (shared pkg/, scripts, CI)
2. Component Templates: POST /projects/{id}/components adds services/workers/apps/cli
3. Deployment: Target whole monorepo or individual components

Terminology: "skeleton" = generated project code; "platform" = rdev itself. See ai-lookup/terminology.md.

Plan Document: tmp/template-monorepo-plan.md

Implementation Phases

Phase 1: Skeleton Template

Create templates/skeleton/ with base monorepo structure.

Files to create:

internal/adapter/templates/templates/skeleton/
├── CLAUDE.md.tmpl
├── README.md.tmpl
├── Procfile.tmpl
├── docker-compose.yml.tmpl
├── go.work.tmpl
├── .golangci.yml
├── .gitignore
├── .woodpecker.yml.tmpl        # ⚠️ Template-provided CI (never AI-generated)
├── .claude/
│   ├── settings.local.json
│   ├── guides/
│   └── skills/
├── scripts/
│   ├── install.sh
│   ├── quality.sh
│   ├── dev.sh
│   └── discover.sh
└── pkg/
    ├── go.mod.tmpl
    ├── app/
    ├── middleware/
    ├── httpcontext/
    ├── httpresponse/
    ├── httpvalidation/
    ├── logging/
    ├── config/
    └── httpclient/

Critical: CI Must Be Templated

AI-generated .woodpecker.yml produces invalid YAML (broken anchor syntax). All CI comes from templates:

• Skeleton provides base .woodpecker.yml.tmpl
• Components provide .woodpecker.step.yml.tmpl
• AddComponent service inserts component steps into main pipeline

Template Variables:

{{.ProjectName}}     // "acme"
{{.GoModule}}        // "github.com/orchard9/acme"
{{.Description}}     // "Project description"

Provider Changes:

// internal/port/template_provider.go
type TemplateProvider interface {
    // Existing
    GetTemplate(name string) (*Template, error)
    ListTemplates() []Template

    // New
    GetSkeleton() (*Template, error)
    GetComponentTemplate(componentType, templateName string) (*Template, error)
    ListComponentTemplates(componentType string) []Template
}

Phase 1.5: Shared Packages (pkg/)

Extract and combine packages from Aeries + Colix:

Package	Source	Key Files
app/	Aeries pkg/chassis/	app.go
middleware/	Colix pkg/middleware/	cors.go, recovery.go, request_id.go, logger.go
httpcontext/	Colix pkg/httpcontext/	keys.go
httpresponse/	Both	response.go, envelope.go
httpvalidation/	Colix pkg/httpvalidation/	validator.go, errors.go
logging/	Both	logger.go
config/	Aeries pkg/config/	config.go
httpclient/	Both	client.go

Phase 2: Component Templates

Migrate existing templates to component format:

internal/adapter/templates/templates/components/
├── service/           # Renamed from go-api
│   ├── cmd/server/main.go.tmpl
│   ├── internal/
│   ├── Makefile.tmpl
│   ├── Dockerfile.tmpl
│   ├── component.yaml.tmpl
│   └── .woodpecker.step.yml.tmpl  # CI step for this component type
├── worker/            # NEW
├── app-astro/         # Renamed from astro-landing
├── app-react/         # NEW
└── cli/               # NEW

Each component includes .woodpecker.step.yml.tmpl that defines its Kaniko build step. When components are added, the service renders this template and inserts it into the main pipeline.

Component Variables:

{{.ProjectName}}        // "acme"
{{.ComponentName}}      // "auth-api"
{{.ComponentNameCamel}} // "AuthApi"
{{.GoModule}}           // "github.com/orchard9/acme"
{{.Port}}               // 8080

Phase 3: Add Component API

Create endpoint for adding components to existing projects.

Handler:

// internal/handlers/components.go
func (h *Handler) AddComponent(w http.ResponseWriter, r *http.Request) {
    projectID := chi.URLParam(r, "projectID")

    var req AddComponentRequest
    if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
        api.Error(w, http.StatusBadRequest, "invalid request")
        return
    }

    component, err := h.svc.AddComponent(r.Context(), projectID, req.Type, req.Name, req.Template)
    if err != nil {
        api.Error(w, http.StatusInternalServerError, err.Error())
        return
    }

    api.OK(w, component)
}

Request/Response:

type AddComponentRequest struct {
    Type     string `json:"type"`     // service, worker, app, cli
    Name     string `json:"name"`     // auth-api, dashboard, etc.
    Template string `json:"template"` // optional: specific variant
}

Service Logic:

func (s *Service) AddComponent(ctx context.Context, projectID, compType, name, template string) (*domain.Component, error) {
    // 1. Render component template
    // 2. Commit files to project repo
    // 3. Update Procfile with new entry
    // 4. Update go.work if Go component
    // 5. Update CLAUDE.md routing
    // 6. Update .woodpecker.yml with component's build step
    // 7. Return component metadata
}

Phase 4: Component-Aware Deployment

Extend deploy to support component targets:

type DeployRequest struct {
    Component string `json:"component,omitempty"` // e.g., "services/auth-api"
}

// POST /projects/{id}/deploy                    → full monorepo
// POST /projects/{id}/deploy/services/auth-api  → single component

Phase 5: Discovery Scripts

Scripts that walk the monorepo and operate on all components:

# scripts/discover.sh
#!/bin/bash
for type in services workers apps cli; do
  for dir in "$type"/*/; do
    [ -d "$dir" ] && echo "$type/$(basename $dir)"
  done
done

# scripts/install.sh
#!/bin/bash
for service in services/*/; do
  [ -f "$service/go.mod" ] && (cd "$service" && go mod download)
done
for app in apps/*/; do
  [ -f "$app/package.json" ] && (cd "$app" && npm install)
done

# scripts/quality.sh
#!/bin/bash
golangci-lint run ./services/... ./workers/... ./cli/...
for app in apps/*/; do
  [ -f "$app/package.json" ] && (cd "$app" && npm run lint)
done

# scripts/dev.sh
#!/bin/bash
docker-compose up -d
overmind start

Phase 6: Quality Hooks

Pre-commit hooks for monorepo quality:

# .githooks/pre-commit
#!/bin/bash
# File length check (500 lines)
for file in $(git diff --cached --name-only --diff-filter=ACM | grep -E '\.(go|ts|tsx)$'); do
  lines=$(wc -l < "$file")
  if [ "$lines" -gt 500 ]; then
    echo "Error: $file exceeds 500 lines ($lines)"
    exit 1
  fi
done

# Go formatting
gofmt -l -w $(git diff --cached --name-only --diff-filter=ACM | grep '\.go$')

# Linting
golangci-lint run ./services/... ./workers/... ./cli/...

Domain Model

// internal/domain/component.go
type ComponentType string

const (
    // Code components (scaffold template files)
    ComponentTypeService  ComponentType = "service"
    ComponentTypeWorker   ComponentType = "worker"
    ComponentTypeAppAstro ComponentType = "app-astro"
    ComponentTypeAppReact ComponentType = "app-react"
    ComponentTypeCLI      ComponentType = "cli"

    // Infrastructure components (provision resources)
    ComponentTypePostgres ComponentType = "postgres"
    ComponentTypeRedis    ComponentType = "redis"
)

type Component struct {
    Type         ComponentType
    Name         string
    Template     string
    Path         string            // "services/auth-api" or "infra/postgres"
    Port         int               // from component.yaml or auto-assigned
    Dependencies []string          // postgres, redis, etc.
    BuildOrder   int
}

Infrastructure Components

Infrastructure components (postgres, redis) don't scaffold files — they provision actual resources:

Adding Database (CockroachDB)

POST /projects/acme/components
{
  "type": "postgres",
  "name": "main-db"
}

This:

1. Creates a CockroachDB database: project_acme
2. Creates a database user with full permissions
3. Stores DATABASE_URL and DATABASE_URL_STAGING in credential store

Adding Cache (Redis)

POST /projects/acme/components
{
  "type": "redis",
  "name": "job-queue"
}

This:

1. Creates a Redis ACL user: proj-acme
2. Scopes access to keys matching project:acme:*
3. Stores REDIS_URL, REDIS_URL_STAGING, and REDIS_PREFIX in credential store

Credential Injection

Critical: When code components are deployed, credentials are automatically injected.

The createInitialComponentDeployment() function:

1. Calls fetchProjectCredentials(projectID) to retrieve stored credentials
2. Populates DeploySpec.Secrets with DATABASE_URL, REDIS_URL, etc.
3. Deployer creates K8s Secret and mounts it as environment variables

Available credentials (if provisioned):

• DATABASE_URL - CockroachDB connection string
• DATABASE_URL_STAGING - Staging database (same as prod currently)
• REDIS_URL - Redis connection string with auth
• REDIS_URL_STAGING - Staging Redis (same as prod currently)
• REDIS_PREFIX - Key prefix for isolation (e.g., project:acme:)

Service Discovery: Sibling services are also injected as env vars:

• AUTH_SVC_URL=http://acme-auth-svc:8001
• CHAT_SVC_URL=http://acme-chat-svc:8002

File Pointers:

• Credential injection: internal/service/component_deploy.go:35-48, 160-212
• Infrastructure provisioning: internal/service/component_infra.go
• Deployer secret creation: internal/adapter/deployer/resources.go:81-135

## API Reference

**Important Route Note:** Project CRUD uses `/project` (singular), but component/build/pipeline operations use `/projects/{id}/...` (plural).

### Create Project (Skeleton)

```bash
POST /project
{
  "name": "acme",
  "description": "Acme Corp platform"
}

# Response: Creates monorepo skeleton in repo

Add Component

POST /projects/acme/components
{
  "type": "service",
  "name": "auth-api",
  "template": "service"  # optional
}

# Response:
{
  "type": "service",
  "name": "auth-api",
  "path": "services/auth-api",
  "port": 8001
}

Deploy

# Component deploy (single component at a time)
POST /projects/acme/deploy
{
  "component": "services/auth-api"
}

# Note: Full monorepo deploy is handled by CI pipeline, not a single API call

List Components

GET /projects/acme/components

# Response:
{
  "components": [
    {"type": "service", "name": "auth-api", "path": "services/auth-api", "port": 8001},
    {"type": "app-react", "name": "dashboard", "path": "apps/dashboard", "port": 3001}
  ]
}

Delete Project

DELETE /project/acme

List Component Templates

GET /templates/components

# Response shows available component types: service, worker, app-astro, app-react, cli

Testing

func TestAddComponent(t *testing.T) {
    // Setup project first
    project := createTestProject(t, "test-monorepo")

    // Add service component
    req := AddComponentRequest{
        Type:     "service",
        Name:     "auth-api",
        Template: "go-api",
    }

    component, err := svc.AddComponent(ctx, project.ID, req.Type, req.Name, req.Template)
    require.NoError(t, err)

    assert.Equal(t, "service", string(component.Type))
    assert.Equal(t, "auth-api", component.Name)
    assert.Equal(t, "services/auth-api", component.Path)

    // Verify files created in repo
    files, err := gitea.ListFiles(project.Owner, project.Name, "services/auth-api")
    require.NoError(t, err)
    assert.Contains(t, files, "cmd/server/main.go")
    assert.Contains(t, files, "Makefile")
}

Troubleshooting

Component endpoint returns 404

The component service is only initialized when GITEA_URL, GITEA_TOKEN, and template provider are all configured. Check startup logs for "component service initialized":

kubectl logs -n rdev deployment/rdev-api | grep "component service"

If missing, verify the rdev-api deployment has the required env vars.

Component addition fails with "directory exists"

The component path already exists. Either delete it or choose a different name.

Component addition fails with "connection refused"

Hairpin NAT issue. rdev-api is trying to reach Gitea via external URL which doesn't work from within the cluster. Use internal service hostnames:

# In deployments/k8s/base/rdev-api.yaml
- name: GITEA_URL
  value: "http://gitea.threesix.svc.cluster.local"

See ops/networking.md for details.

CI pipeline linter errors

Woodpecker may show linter warnings about oneOf/anyOf validation. These are usually benign. Check for actual schema violations like invalid when.event values.

Critical: The .woodpecker.yml marker must be exactly # COMPONENT_STEPS_BELOW on its own line. If the marker has trailing text (like - Do not remove), it will be appended to the last line of inserted component steps, corrupting the YAML.

File Pointer: internal/adapter/templates/templates/skeleton/.woodpecker.yml.tmpl:11

Procfile not updating

Check that the template includes the Procfile.tmpl and the service has write access to the repo.

go.work not syncing

Run ./scripts/discover.sh to verify component detection, then manually run go work sync.

Related

• Project Templates - Current single-template system
• Build Orchestration - Component builds
• ai-lookup: Composable Monorepo - Quick facts