# Forage

A foraging engine. The Chrome extension browses, tidalDB learns, the feed sharpens.

---

This readme is an implementation-oriented summary. `plan.md` is the canonical build spec when details conflict.

---

## What It Does

Forage is two things:

**`forage-engine`** — a library crate wrapping tidalDB with a foraging-specific signal schema, MAB exploration layer, and clean API. This is the reusable thing. Other applications embed it.

**`forage-server`** — a thin Axum HTTP server + feed page that demonstrates the engine. Click something → the next 7 items shift. Skip something → it does not come back. The preference model updates in under 100ms. The feed page handles all signal posting itself via browser `fetch()` — no external tooling required.

The Chrome extension's role is light: navigate to the page, snapshot the feed state before and after a session of interactions, report what changed. Three tool calls. The interesting loop runs without it.

---

## Quickstart

```bash
# From repo root
cargo run -p forage-server --manifest-path applications/forage/server/Cargo.toml

# Feed is live at:
open http://localhost:4242
```

The server seeds 100 items across 8 categories on startup. Persistent by default at `~/.forage/data`.

For throwaway runs:

```bash
cargo run -p forage-server --manifest-path applications/forage/server/Cargo.toml -- --ephemeral
```

To override the default data directory:

```bash
cargo run -p forage-server --manifest-path applications/forage/server/Cargo.toml -- --data-dir ~/.forage/data
```

---

## API

### Post a signal

```bash
curl -X POST http://localhost:4242/signal \
  -H "Content-Type: application/json" \
  -d '{ "user_id": 1, "item_id": 42, "signal_type": "view" }'
```

Signal types: `view` · `dwell` · `save` · `skip` · `share`

For dwell, include duration:
```bash
-d '{ "user_id": 1, "item_id": 42, "signal_type": "dwell", "duration_ms": 240000 }'
```

### Get feed

```bash
curl "http://localhost:4242/feed?user=1&limit=7"
```

Each item in the response includes a `label` field:
- `"match"` — near a confirmed interest centroid
- `"exploring"` — from an underexplored category (the MAB exploration slot)
- `"trending"` — high velocity, regardless of personalization
- `"resurfaced"` — previously low-engagement content being re-checked

### List all items

```bash
curl http://localhost:4242/items
```

---

## The Exploration Budget

Every feed response includes approximately 1 exploration item (14% of 7 items, rounded up). This item comes from a category where the user has fewer than 5 signals. It is labeled `"exploring"`.

When you engage with an exploration item, that category graduates toward the interest model. When you skip it, the MAB deprioritizes that exploration arm. Over time, exploration items land more often — the system learns your exploration tolerance.

---

## Pre-built Users

Three users are seeded on startup:

| User ID | State | Description |
|---------|-------|-------------|
| `1` | Cold start | No signals. Pure exploration feed. |
| `2` | Light signals | ~10 signals across 2-3 categories. Partial preference model. |
| `3` | Converged | ~50 signals, 2 dominant interest categories. Strong personalization. |

Switch between users via the user dropdown in the feed page, or `?user=N` in the API.

---

## Running a Demo Session With Claude

In **P0**, Claude's role is **observer only** — the feed page posts its own signals. Ask Claude to:

> "Navigate to localhost:4242 and snapshot the feed. I'm going to interact with it for a few minutes. After I'm done, read the page again and tell me how the feed composition shifted — which categories rose, which fell, what exploration items appeared."

Claude will:
1. `navigate` to `localhost:4242`
2. `read_page` — snapshot the initial feed (one call)
3. Stay idle while you interact with the feed normally in the browser
4. `read_page` again when you say you're done — snapshot the final feed (one call)
5. Report: category distribution before vs. after, which labels changed, what the MAB surfaced

Three MCP tool calls total. No token-burning click-by-click automation. The loop runs in the page.

This is the P0 acceptance test.

---

## Categories

The seed corpus covers 8 categories:

| Category | Items | Sample |
|----------|-------|--------|
| `tech` | 15 | Distributed systems, CRDTs, WAL internals |
| `music` | 10 | Production, mixing, composition process |
| `jazz` | 15 | Coltrane changes, rhythm lineage, free jazz |
| `cooking` | 12 | Fermentation, sourdough chemistry, miso |
| `fitness` | 10 | Loaded carries, mobility, walking |
| `travel` | 10 | City guides, route essays, local craft cultures |
| `science` | 15 | Emergence, small worlds, power laws |
| `literature` | 13 | Essays, criticism, long-form craft |

---

## Architecture

```
Chrome Extension (Claude, observer)
        │  read_page (before/after)
        ▼
 Feed Page (browser, localhost:4242)
        │  fetch() — signals + feed polling
        ▼
 forage-server (Axum, thin)
        │  Rust function calls
        ▼
 forage-engine (library crate)
        │  MAB · schema · seed corpus
        ▼
   tidalDB (embedded, in-process)
```

See [architecture.md](architecture.md) for the full data flow, signal schema, preference evolution, and MAB implementation.

---

## What This Proves

This is the proof-of-concept for tidalDB's core thesis:

> A single embedded database can replace the 6-system content ranking stack — and the applications you can build on top of it could not exist any other way.

Forage's feedback loop requires a signal-to-re-rank latency under 100ms. On the 6-system stack (Redis → Kafka → feature store sync → ranking service), this is not achievable. tidalDB closes the loop in-process: signal write, preference vector update, and re-rank are the same operation.

The MAB exploration layer — finding things at the edge of your interest graph, learning your exploration tolerance — requires a live semantic preference model that updates with every interaction. This requires the vector index and the preference vector to be the same data structure, updated atomically. That is tidalDB.

Forage is not the product. Forage is the demo that makes someone say "I get it now."

---

## Embedding the Engine in Another App

```toml
# your-app/Cargo.toml
[dependencies]
forage-engine = { path = "path/to/forage/engine" }
```

```rust
use forage_engine::{ForageEngine, SignalType};
use std::path::Path;

let engine = ForageEngine::persistent(Path::new("/home/you/.forage/data"))?;
engine.seed_default_corpus()?;

engine.signal(user_id, item_id, SignalType::View)?;
engine.signal_dwell(user_id, item_id, 240_000)?;

let feed = engine.feed(user_id, 7)?;
for item in feed {
    println!("{} [{}]", item.title, item.label);
}
```

The Axum server is not required. The engine runs anywhere tidalDB runs — which is anywhere Rust runs.

---

## See Also

- [vision.md](vision.md) — What Forage is and why it matters
- [plan.md](plan.md) — Build phases and acceptance criteria
- [architecture.md](architecture.md) — Technical design and data flows
- [../../VISION.md](../../VISION.md) — tidalDB's thesis
- [../../API.md](../../API.md) — tidalDB API reference
