tidaldb/applications/forage/readme.md

Forage

A foraging engine. The Chrome extension browses, tidalDB learns, the feed sharpens.

---

This readme is an implementation-oriented summary. plan.md is the canonical build spec when details conflict.

---

What It Does

Forage is two things:

forage-engine — a library crate wrapping tidalDB with a foraging-specific signal schema, MAB exploration layer, and clean API. This is the reusable thing. Other applications embed it.

forage-server — a thin Axum HTTP server + feed page that demonstrates the engine. Click something → the next 7 items shift. Skip something → it does not come back. The preference model updates in under 100ms. The feed page handles all signal posting itself via browser fetch() — no external tooling required.

The Chrome extension's role is light: navigate to the page, snapshot the feed state before and after a session of interactions, report what changed. Three tool calls. The interesting loop runs without it.

---

Quickstart

# From repo root
cargo run -p forage-server --manifest-path applications/forage/server/Cargo.toml

# Feed is live at:
open http://localhost:4242

The server seeds 100 items across 8 categories on startup. Persistent by default at ~/.forage/data.

For throwaway runs:

cargo run -p forage-server --manifest-path applications/forage/server/Cargo.toml -- --ephemeral

To override the default data directory:

cargo run -p forage-server --manifest-path applications/forage/server/Cargo.toml -- --data-dir ~/.forage/data

---

API

Post a signal

curl -X POST http://localhost:4242/signal \
  -H "Content-Type: application/json" \
  -d '{ "user_id": 1, "item_id": 42, "signal_type": "view" }'

Signal types: view · dwell · save · skip · share

For dwell, include duration:

-d '{ "user_id": 1, "item_id": 42, "signal_type": "dwell", "duration_ms": 240000 }'

Get feed

curl "http://localhost:4242/feed?user=1&limit=7"

Each item in the response includes a label field:

• "match" — near a confirmed interest centroid
• "exploring" — from an underexplored category (the MAB exploration slot)
• "trending" — high velocity, regardless of personalization
• "resurfaced" — previously low-engagement content being re-checked

List all items

curl http://localhost:4242/items

---

The Exploration Budget

Every feed response includes approximately 1 exploration item (14% of 7 items, rounded up). This item comes from a category where the user has fewer than 5 signals. It is labeled "exploring".

When you engage with an exploration item, that category graduates toward the interest model. When you skip it, the MAB deprioritizes that exploration arm. Over time, exploration items land more often — the system learns your exploration tolerance.

---

Pre-built Users

Three users are seeded on startup:

User ID	State	Description
1	Cold start	No signals. Pure exploration feed.
2	Light signals	~10 signals across 2-3 categories. Partial preference model.
3	Converged	~50 signals, 2 dominant interest categories. Strong personalization.

Switch between users via the user dropdown in the feed page, or ?user=N in the API.

---

Running a Demo Session With Claude

In P0, Claude's role is observer only — the feed page posts its own signals. Ask Claude to:

"Navigate to localhost:4242 and snapshot the feed. I'm going to interact with it for a few minutes. After I'm done, read the page again and tell me how the feed composition shifted — which categories rose, which fell, what exploration items appeared."

Claude will:

1. navigate to localhost:4242
2. read_page — snapshot the initial feed (one call)
3. Stay idle while you interact with the feed normally in the browser
4. read_page again when you say you're done — snapshot the final feed (one call)
5. Report: category distribution before vs. after, which labels changed, what the MAB surfaced

Three MCP tool calls total. No token-burning click-by-click automation. The loop runs in the page.

This is the P0 acceptance test.

---

Categories

The seed corpus covers 8 categories:

Category	Items	Sample
tech	15	Distributed systems, CRDTs, WAL internals
music	10	Production, mixing, composition process
jazz	15	Coltrane changes, rhythm lineage, free jazz
cooking	12	Fermentation, sourdough chemistry, miso
fitness	10	Loaded carries, mobility, walking
travel	10	City guides, route essays, local craft cultures
science	15	Emergence, small worlds, power laws
literature	13	Essays, criticism, long-form craft

---

Architecture

Chrome Extension (Claude, observer)
        │  read_page (before/after)
        ▼
 Feed Page (browser, localhost:4242)
        │  fetch() — signals + feed polling
        ▼
 forage-server (Axum, thin)
        │  Rust function calls
        ▼
 forage-engine (library crate)
        │  MAB · schema · seed corpus
        ▼
   tidalDB (embedded, in-process)

See architecture.md for the full data flow, signal schema, preference evolution, and MAB implementation.

---

What This Proves

This is the proof-of-concept for tidalDB's core thesis:

A single embedded database can replace the 6-system content ranking stack — and the applications you can build on top of it could not exist any other way.

Forage's feedback loop requires a signal-to-re-rank latency under 100ms. On the 6-system stack (Redis → Kafka → feature store sync → ranking service), this is not achievable. tidalDB closes the loop in-process: signal write, preference vector update, and re-rank are the same operation.

The MAB exploration layer — finding things at the edge of your interest graph, learning your exploration tolerance — requires a live semantic preference model that updates with every interaction. This requires the vector index and the preference vector to be the same data structure, updated atomically. That is tidalDB.

Forage is not the product. Forage is the demo that makes someone say "I get it now."

---

Embedding the Engine in Another App

# your-app/Cargo.toml
[dependencies]
forage-engine = { path = "path/to/forage/engine" }

use forage_engine::{ForageEngine, SignalType};
use std::path::Path;

let engine = ForageEngine::persistent(Path::new("/home/you/.forage/data"))?;
engine.seed_default_corpus()?;

engine.signal(user_id, item_id, SignalType::View)?;
engine.signal_dwell(user_id, item_id, 240_000)?;

let feed = engine.feed(user_id, 7)?;
for item in feed {
    println!("{} [{}]", item.title, item.label);
}

The Axum server is not required. The engine runs anywhere tidalDB runs — which is anywhere Rust runs.

---

See Also

• vision.md — What Forage is and why it matters
• plan.md — Build phases and acceptance criteria
• architecture.md — Technical design and data flows
• ../../VISION.md — tidalDB's thesis
• ../../API.md — tidalDB API reference