# Technical Design: Async Jobs

**Feature:** async-jobs
**Status:** approved
**Author:** Claude
**Created:** 2026-02-05

## Architecture Overview

```
┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│   API Service   │────▶│      Redis      │◀────│ Background      │
│  (services/api) │     │                 │     │ Worker          │
└─────────────────┘     └─────────────────┘     └─────────────────┘
       │                        │                      │
       │ POST /jobs             │ jobs:queue           │ BLPOP
       │ GET /jobs/{id}         │ jobs:data:{id}       │ Update status
       └────────────────────────┴──────────────────────┘
```

## Component Design

### 1. Redis Job Queue Package (`pkg/redisqueue`)

A new shared package providing Redis-based job queue operations:

```go
// pkg/redisqueue/queue.go
type RedisQueue struct {
    client *redis.Client
    logger *logging.Logger
}

func NewRedisQueue(client *redis.Client, logger *logging.Logger) *RedisQueue

// Producer operations (for API)
func (q *RedisQueue) Enqueue(ctx context.Context, job *Job) error
func (q *RedisQueue) GetJob(ctx context.Context, jobID string) (*Job, error)

// Consumer operations (for Worker)
func (q *RedisQueue) Dequeue(ctx context.Context, timeout time.Duration) (*Job, error)
func (q *RedisQueue) UpdateStatus(ctx context.Context, jobID string, status JobStatus, err string) error
```

### 2. API Service Modifications

**New Files:**
- `services/api/internal/api/handlers/job.go` - Job HTTP handlers
- `services/api/internal/port/job.go` - JobQueue port interface
- `services/api/internal/service/job.go` - Job business logic

**Modified Files:**
- `services/api/cmd/server/main.go` - Add Redis client, job service initialization
- `services/api/internal/api/routes.go` - Register job routes
- `services/api/internal/config/config.go` - Add Redis URL config

### 3. Worker Modifications

**Modified Files:**
- `workers/background-processor/cmd/worker/main.go` - Add Redis client, job handler
- `workers/background-processor/internal/config/config.go` - Add Redis URL, work simulation config
- `workers/background-processor/internal/handlers/jobs.go` - Async job handler

## Redis Data Structure

### Queue List: `jobs:queue`
- Type: List
- Operations: RPUSH (enqueue), BLPOP (dequeue)
- Contains: Job IDs only (lightweight)

### Job Data: `jobs:data:{id}`
- Type: Hash (stored as JSON string for simplicity)
- Fields: id, type, payload, status, created_at, started_at, completed_at, error
- TTL: 24 hours after completion (configurable)

## Sequence Diagrams

### Create Job Flow
```
Client → API → JobService.Create()
                    │
                    ├── Generate UUID
                    ├── Create Job struct
                    ├── SET jobs:data:{id} (JSON)
                    ├── RPUSH jobs:queue (id only)
                    └── Return job with pending status
```

### Get Job Flow
```
Client → API → JobService.Get(id)
                    │
                    ├── GET jobs:data:{id}
                    └── Return job or 404
```

### Worker Processing Flow
```
Worker → RedisQueue.Dequeue()
              │
              ├── BLPOP jobs:queue
              ├── GET jobs:data:{id}
              ├── Update status to "running"
              ├── Simulate work (sleep)
              └── Update status to "completed"
```

## Configuration

### API Service
```bash
REDIS_URL=redis://localhost:6379
```

### Worker
```bash
REDIS_URL=redis://localhost:6379
JOB_SIMULATION_DURATION=2s  # Duration to simulate work
```

## Error Handling

| Scenario | Behavior |
|----------|----------|
| Redis connection failure | Return 503 Service Unavailable |
| Job not found | Return 404 Not Found |
| Invalid job payload | Return 400 Bad Request |
| Worker crash during processing | Job remains in "running" (future: add timeout/recovery) |

## Testing Strategy

1. **Unit Tests**: Mock Redis client, test service logic
2. **Integration Tests**: Real Redis (via testcontainers or local), test full flow

## Files to Create/Modify

### New Files
1. `pkg/redisqueue/queue.go` - Redis queue implementation
2. `pkg/redisqueue/job.go` - Job struct and status constants
3. `services/api/internal/api/handlers/job.go` - Job handlers
4. `services/api/internal/api/handlers/job_test.go` - Handler tests
5. `services/api/internal/port/job.go` - JobQueue interface
6. `services/api/internal/service/job.go` - Job service
7. `workers/background-processor/internal/handlers/jobs.go` - Job processor

### Modified Files
1. `services/api/cmd/server/main.go` - Add Redis setup
2. `services/api/internal/api/routes.go` - Add job routes
3. `services/api/internal/config/config.go` - Add Redis config
4. `workers/background-processor/cmd/worker/main.go` - Add Redis queue processing
5. `workers/background-processor/internal/config/config.go` - Add Redis config