jx12n 3bcfb3c576 feat: Bazel build, crate docs/ai-lookup, docker images, and engine hardening

- Add BUILD.bazel across tidal, tidal-net, tidal-server, tidalctl for bzlmod build
- Add tidal/ crate docs (README, CHANGELOG, CONTRIBUTING, AGENTS, CLAUDE, API, ARCHITECTURE) and ai-lookup reference
- Add docker standalone/cluster/deploy images, compose, and prometheus config
- Harden WAL (batch format, writer, dedup, diagnostics), text syncer/collectors, and vector registry
- Expand tidalctl CLI and tests; restructure WAL/visibility integration test suites
- Refine tidal-net transport/client/server and tidal-server cluster/scatter-gather

2026-06-07 18:29:38 -06:00

1.5 KiB

Raw Blame History

Milestone 5, Phase 4: Creator and People Search

Goal

Prove that the same SEARCH pipeline that indexes items can also index creator entities. After this phase, a developer can call db.search(&Search { entity_kind: Creator, query: "jazz" }) and receive BM25-ranked creators with optional vector fusion, filters, and sort modes.

Motivation

m5p1–p3 built a complete SEARCH pipeline for items. Creators are a first-class entity in tidalDB with their own storage engine, metadata, and embeddings. Extending the pipeline to creators validates the multi-entity-kind design and unlocks the people-search use case.

Tasks

Task	Title	Status
task-01	Creator Text Indexing	pending
task-02	Creator Vector Index	pending
task-03	Creator Search Executor	pending

Execution Order

task-01 (Creator Text Indexing)
    |
    v
task-02 (Creator Vector Index)
    |
    v
task-03 (Creator Search Executor)

All tasks are sequential.

Verification

cargo check
cargo clippy -- -D warnings
cargo test --lib
cargo test --test m5p4_creator_search
cargo bench --bench search -- bench_search_creator_text_200

Key assertions:

db.search(Search { entity_kind: Creator, query: "jazz" }) returns BM25-ranked creators
filter(verified = true) excludes non-verified creators
similar_to lookup triggers ANN on (EntityKind::Creator, "content") slot
bench_search_creator_text_200 < 20ms
All existing m5p3 item search tests still pass

1.5 KiB Raw Blame History Unescape Escape