# Ingestor Service

> **Crate:** `stemedb-ingest`
> **Status:** Implemented (Phase 1)

## Purpose

The Ingestor is the background worker that bridges the Write-Ahead Log (WAL) to the KV storage engine. It continuously tails the WAL and persists records to the HybridStore (fjall + redb) using content-addressed keys.

## Architecture

```
[WAL Journal] ---> [IngestWorker] ---> [KVStore (HybridStore)]
                         |
                         v
                   [Subject Index]
```

## Key Components

### RecordType

Discriminator for WAL payloads (8-byte aligned header):
- `Assertion = 0` - Knowledge claims
- `Vote = 1` - Consensus votes
- `Epoch = 2` - Paradigm definitions

### Storage Layout

| Key Pattern | Value | Description |
|-------------|-------|-------------|
| `H:{blake3_hash}` | Serialized Assertion | Content-addressed assertion store |
| `V:{assertion_hash}:{vote_hash}` | Serialized Vote | Votes on assertions |
| `E:{epoch_id_hex}` | Serialized Epoch | Epoch definitions |
| `S:{subject}` | BLAKE3 hash bytes | Subject adjacency index |

## Usage

```rust
use stemedb_ingest::{Ingestor, serialize_assertion};
use stemedb_wal::Journal;
use stemedb_storage::HybridStore;

// Create components
let journal = Arc::new(Mutex::new(Journal::open("./wal")?));
let store = Arc::new(HybridStore::open("./db")?);

// Create and start ingestor
let mut ingestor = Ingestor::new(journal.clone(), store);
ingestor.start(); // Spawns background task

// Write to WAL (records will be ingested automatically)
let assertion = Assertion { ... };
let payload = serialize_assertion(&assertion)?;
journal.lock().await.append(payload)?;
```

## Serialization

Records are serialized with an 8-byte header to maintain rkyv alignment:

```
[type: u8][padding: 7 bytes][rkyv payload...]
```

Helper functions:
- `serialize_assertion(&Assertion) -> Result<Vec<u8>>`
- `serialize_vote(&Vote) -> Result<Vec<u8>>`
- `serialize_epoch(&Epoch) -> Result<Vec<u8>>`

## Testing

The ingestor has integration tests covering:
- Single assertion ingestion
- Vote ingestion
- Epoch ingestion
- Multiple record processing
- Subject index creation

## Related

- [Storage Service](./storage.md) - KVStore trait and HybridStore (fjall + redb)
- [Content Addressing](../patterns/content-addressing.md) - BLAKE3 hashing