tidaldb/API.md

API Reference

Quick API Reference: The examples below reflect the current implementation API. Use cargo doc --manifest-path tidal/Cargo.toml --open for full documentation.

How developers interact with tidalDB. This document covers initialization, schema definition, write operations, queries, and the feedback loop.

tidalDB has two interfaces:

1. Rust library — embed it in your process. No network overhead, no serialization. The API is Rust types and method calls.
2. HTTP server (tidal-server) — a standalone Axum-based server exposing a REST API for write, query, and health operations. Useful for polyglot stacks or when embedding isn't practical.

---

Table of Contents

• Initialization
• Schema Definition
  • Entity Types
  • Signal Definitions
  • Ranking Profiles
• Write Path
  • Ingesting Entities
  • Writing Embeddings
  • Writing Relationships
  • Writing Signals
• Query Language
  • RETRIEVE — Feeds, Browse, Related
  • SEARCH — Text + Semantic Retrieval
  • SUGGEST — Autocomplete and Suggestions
• Filters
• Sort Modes
• Diversity Constraints
• Pagination
• Response Format
• Lifecycle and Operations
• HTTP Server API
  • Authentication
  • Write Endpoints
  • Query Endpoints
  • Health Endpoints
  • Error Responses

---

Initialization

Open a database using the builder pattern. Define the schema first, then pass it to the builder.

use tidaldb::TidalDb;
use tidaldb::schema::{SchemaBuilder, EntityKind, DecaySpec, Window};
use std::time::Duration;

// 1. Define the schema (signal types, text fields, embedding slots).
let mut schema = SchemaBuilder::new();
let _ = schema.signal("view", EntityKind::Item, DecaySpec::Exponential {
    half_life: Duration::from_secs(7 * 24 * 3600),
})
    .windows(&[Window::OneHour, Window::TwentyFourHours, Window::ThirtyDays, Window::AllTime])
    .velocity(true)
    .add();
let schema = schema.build().expect("valid schema");

// 2a. Ephemeral (in-memory) -- no filesystem access, ideal for testing.
let db = TidalDb::builder()
    .ephemeral()
    .with_schema(schema.clone())
    .open()?;

// 2b. Persistent -- durable storage at the given path.
let db = TidalDb::builder()
    .with_data_dir("/var/lib/tidaldb/my_app")
    .with_schema(schema)
    .open()?;

The database is Send + Sync. Share it across threads with Arc<TidalDb>.

---

Schema Definition

Schema is defined before opening the database using SchemaBuilder. It declares signal types, text fields for full-text search, and embedding slots for vector search.

Entity Types

Entities are the nodes of the system. Three built-in types: Item, User, Creator. Entity metadata is stored as HashMap<String, String> key-value pairs.

Signal Definitions

Signals are typed, timestamped event streams. Decay, velocity, and windowed aggregation are declared in schema -- not computed in application code.

use tidaldb::schema::{SchemaBuilder, EntityKind, DecaySpec, Window, TextFieldType};
use std::time::Duration;

let mut schema = SchemaBuilder::new();

// View signal: exponential decay, 7-day half-life, three windows + velocity.
let _ = schema.signal("view", EntityKind::Item, DecaySpec::Exponential {
    half_life: Duration::from_secs(7 * 24 * 3600),
})
    .windows(&[Window::OneHour, Window::TwentyFourHours, Window::SevenDays, Window::ThirtyDays, Window::AllTime])
    .velocity(true)
    .add();

// Like signal: slower decay (14 days).
let _ = schema.signal("like", EntityKind::Item, DecaySpec::Exponential {
    half_life: Duration::from_secs(14 * 24 * 3600),
})
    .windows(&[Window::TwentyFourHours, Window::SevenDays, Window::AllTime])
    .velocity(true)
    .add();

// Skip signal: fast decay (1 day), no velocity.
let _ = schema.signal("skip", EntityKind::Item, DecaySpec::Exponential {
    half_life: Duration::from_secs(24 * 3600),
})
    .windows(&[Window::OneHour, Window::TwentyFourHours])
    .velocity(false)
    .add();

// Hide signal: permanent (never decays), no windows.
let _ = schema.signal("hide", EntityKind::Item, DecaySpec::Permanent).add();

// Share signal: for trending and social features.
let _ = schema.signal("share", EntityKind::Item, DecaySpec::Exponential {
    half_life: Duration::from_secs(3 * 24 * 3600),
})
    .windows(&[Window::OneHour, Window::TwentyFourHours, Window::AllTime])
    .velocity(true)
    .add();

// Completion signal: long-lived quality metric.
let _ = schema.signal("completion", EntityKind::Item, DecaySpec::Exponential {
    half_life: Duration::from_secs(30 * 24 * 3600),
})
    .windows(&[Window::AllTime])
    .velocity(false)
    .add();

// Text fields for BM25 full-text search.
schema.text_field("title", TextFieldType::Text);
schema.text_field("description", TextFieldType::Text);
schema.text_field("category", TextFieldType::Keyword);
schema.text_field("tags", TextFieldType::Keyword);

// Creator text fields for creator search.
schema.creator_text_field("name", TextFieldType::Text);
schema.creator_text_field("handle", TextFieldType::Keyword);

// Embedding slots for vector search (you provide the vectors).
schema.embedding_slot("content", EntityKind::Item, 128);
schema.embedding_slot("content", EntityKind::Creator, 128);

let schema = schema.build()?;

Decay types:

Decay	Behavior
Exponential { half_life }	Signal weight halves every half_life duration
Linear { lifetime }	Signal weight drops linearly to zero over lifetime
Permanent	Never decays -- hides, blocks, follows

The full signal reference is in USE_CASES.md Appendix C.

Ranking Profiles

tidalDB ships 25 built-in ranking profiles. The application says profile("trending"). The database executes the entire pipeline.

Built-in profiles include: trending, hot, new, for_you, following, related, notification, search, top_week, top_month, top_all_time, hidden_gems, controversial, most_viewed, most_liked, shuffle, cohort_trending, live, alphabetical_asc, alphabetical_desc, shortest, longest, most_commented, most_shared, date_saved.

See ai-lookup/services/ranking-profiles.md for the full list of built-in profiles.

Cohort Definitions

Cohorts are named predicates over user attributes. They define audience segments for scoped signal aggregation and trending.

use tidaldb::schema::EntityId;

// Define a cohort via db.define_cohort() after opening.
// Cohort signal aggregation happens at signal write time.
// Use RetrieveBuilder::cohort("us_young_music") to scope queries.

---

Write Path

Ingesting Entities

Items enter the system with metadata as HashMap<String, String> key-value pairs. The application provides the embedding -- tidalDB does not generate vectors.

use std::collections::HashMap;
use tidaldb::schema::EntityId;

let mut metadata = HashMap::new();
metadata.insert("title".to_string(), "Introduction to Jazz Piano".to_string());
metadata.insert("description".to_string(), "A beginner's guide...".to_string());
metadata.insert("category".to_string(), "music".to_string());
metadata.insert("tags".to_string(), "jazz,piano,tutorial,beginner".to_string());
metadata.insert("format".to_string(), "video".to_string());
metadata.insert("language".to_string(), "en".to_string());
metadata.insert("duration".to_string(), "1320".to_string()); // seconds
metadata.insert("creator_id".to_string(), "100".to_string());

db.write_item_with_metadata(EntityId::new(1), &metadata)?;

On commit, the item is:

1. Stored in the entity store
2. Text fields indexed in the inverted index (BM25)
3. Inserted into bitmap and range indexes for filtering
4. Added to the universe bitmap for RETRIEVE queries
5. Immediately queryable

Writing Embeddings

Embeddings are written separately from metadata. tidalDB L2-normalizes and indexes them into the HNSW vector index.

use tidaldb::schema::EntityId;

// Item embedding (you compute this externally).
let embedding: Vec<f32> = compute_embedding("Introduction to Jazz Piano");
db.write_item_embedding(EntityId::new(1), &embedding)?;

// Creator embedding.
let creator_embedding: Vec<f32> = compute_creator_embedding("Jazz Academy");
db.write_creator_embedding(EntityId::new(100), &creator_embedding)?;

Writing Relationships

Relationships are directional edges between entities (follows, blocks). Used for the following profile and blocked-creator filtering.

use tidaldb::schema::EntityId;
use tidaldb::schema::Timestamp;
use tidaldb::entities::RelationshipType;

// User follows a creator.
db.write_relationship(
    EntityId::new(123),            // from (user)
    RelationshipType::Follows,     // relationship type
    EntityId::new(100),            // to (creator)
    1.0,                           // weight
    Timestamp::now(),
)?;

Relationship types:

Variant	Meaning
Follows	User follows a creator
Blocks	User blocks a creator
InteractionWeight	Weighted interaction edge
Hide	User hides a creator
Mute	User mutes a creator

Writing Signals

Signals are how the feedback loop closes. A single signal write atomically updates:

1. The item's signal ledger (windowed aggregates, velocity, decay score)
2. The WAL (write-ahead log) for durability

use tidaldb::schema::{EntityId, Timestamp};

// User viewed an item.
db.signal("view", EntityId::new(1), 1.0, Timestamp::now())?;

// User completed 94% of the video.
db.signal("completion", EntityId::new(1), 0.94, Timestamp::now())?;

// User liked an item.
db.signal("like", EntityId::new(1), 1.0, Timestamp::now())?;

// User skipped after 3 seconds (strong negative).
db.signal("skip", EntityId::new(2), 1.0, Timestamp::now())?;

// User tapped "Not interested" (permanent negative on this item).
db.signal("hide", EntityId::new(2), 1.0, Timestamp::now())?;

For signals with user context (updates preference vectors, seen state, interaction weights):

use tidaldb::schema::{EntityId, Timestamp};

db.signal_with_context(
    "view",
    EntityId::new(1),       // item
    1.0,                     // weight
    Timestamp::now(),
    Some(123),               // for_user
    Some(100),               // creator_id
)?;

The next ranking query -- even 100ms later -- reflects the updated state.

---

Query Language

Three operations: RETRIEVE (feed generation, browse, related), SEARCH (text + semantic retrieval), SUGGEST (autocomplete).

All queries return ranked results with scores. The application renders -- it never re-ranks.

RETRIEVE — Feeds, Browse, Related

RETRIEVE generates ranked content lists. It handles personalized feeds, category browse, trending, following, related content, and every other surface described in USE_CASES.md.

use tidaldb::query::retrieve::Retrieve;
use tidaldb::schema::EntityId;

// Personalized For You feed.
let query = Retrieve::builder()
    .for_user(123)
    .profile("for_you")
    .limit(50)
    .build()?;
let results = db.retrieve(&query)?;

// Trending globally.
let query = Retrieve::builder()
    .profile("trending")
    .limit(25)
    .build()?;
let results = db.retrieve(&query)?;

use tidaldb::storage::indexes::filter::FilterExpr;

// Trending in a category.
let query = Retrieve::builder()
    .profile("trending")
    .filter(FilterExpr::eq("category", "jazz"))
    .limit(25)
    .build()?;
let results = db.retrieve(&query)?;

// Trending within a cohort -- what's hot among US young music fans.
let query = Retrieve::builder()
    .profile("cohort_trending")
    .cohort("us_young_music")
    .limit(25)
    .build()?;
let results = db.retrieve(&query)?;

// Following feed -- content from followed creators.
let query = Retrieve::builder()
    .for_user(123)
    .profile("following")
    .limit(50)
    .build()?;
let results = db.retrieve(&query)?;

use tidaldb::ranking::diversity::DiversityConstraints;

// Related content / Up Next -- anchored to a specific item.
let query = Retrieve::builder()
    .for_user(123)
    .profile("related")
    .similar_to(EntityId::new(1))
    .diversity(DiversityConstraints::new().max_per_creator(1))
    .limit(10)
    .build()?;
let results = db.retrieve(&query)?;

// Browse category with explicit sort mode.
let query = Retrieve::builder()
    .profile("top_week")
    .filter(FilterExpr::eq("category", "jazz"))
    .limit(20)
    .build()?;
let results = db.retrieve(&query)?;

// Hidden gems -- high quality, low reach.
let query = Retrieve::builder()
    .profile("hidden_gems")
    .limit(20)
    .build()?;
let results = db.retrieve(&query)?;

// Exclude previously seen items.
let query = Retrieve::builder()
    .for_user(123)
    .profile("for_you")
    .exclude(vec![EntityId::new(1), EntityId::new(2)])
    .limit(50)
    .build()?;
let results = db.retrieve(&query)?;

// Creator profile -- items from a specific creator.
let query = Retrieve::builder()
    .profile("new")
    .for_creator(EntityId::new(100))
    .limit(20)
    .build()?;
let results = db.retrieve(&query)?;

// Notification prioritization.
let query = Retrieve::builder()
    .for_user(123)
    .profile("notification")
    .limit(20)
    .build()?;
let results = db.retrieve(&query)?;

SEARCH — Text + Semantic Retrieval

Search combines full-text BM25 relevance with semantic similarity via RRF (Reciprocal Rank Fusion). Text relevance is the floor -- an irrelevant result never surfaces just because the user likes the creator.

use tidaldb::query::search::Search;

// Basic keyword search, personalized for this user.
let query = Search::builder()
    .query("rust tutorial beginner")
    .for_user(123)
    .limit(20)
    .build()?;
let results = db.search(&query)?;

// Hybrid search: text + vector.
let query_embedding: Vec<f32> = embed("rust tutorial beginner");
let query = Search::builder()
    .query("rust tutorial beginner")
    .vector(query_embedding)
    .for_user(123)
    .limit(20)
    .build()?;
let results = db.search(&query)?;

// Creator search.
use tidaldb::schema::EntityKind;

let query = Search::builder()
    .query("jazz piano")
    .entity_kind(EntityKind::Creator)
    .limit(10)
    .build()?;
let results = db.search(&query)?;

Query Composition — SEARCH within Scoped Results

SEARCH can be composed with scope constraints. This enables searching within trending, within a cohort, or within any candidate set.

use tidaldb::query::search::{Search, WithinScope};

// Search within globally trending items.
let query = Search::builder()
    .query("jazz piano")
    .within(WithinScope::Trending { window_hours: 24 })
    .limit(20)
    .build()?;
let results = db.search(&query)?;

// Search within cohort-scoped trending.
let query = Search::builder()
    .query("jazz piano")
    .within(WithinScope::CohortTrending {
        cohort: "us_young_music".into(),
        window_hours: 24,
    })
    .limit(20)
    .build()?;
let results = db.search(&query)?;

// Search within a user's following feed.
let query = Search::builder()
    .query("jazz piano")
    .for_user(123)
    .within(WithinScope::Following)
    .limit(20)
    .build()?;
let results = db.search(&query)?;

WithinScope:

Scope	Candidate Set
Trending { window_hours }	Items with high global velocity in window
CohortTrending { cohort, window_hours }	Items with high velocity among cohort members
Following	Items from followed creators (requires for_user)
Category { name }	Items in a category
Collection { id }	Items in a collection

SUGGEST — Autocomplete and Suggestions

use tidaldb::query::suggest::Suggest;
use tidaldb::schema::EntityId;

// Autocomplete on partial query.
let req = Suggest { prefix: "jazz pia".into(), for_user: None, limit: 5 };
let suggestions = db.suggest(&req)?;
// Returns Vec<Suggestion> with text and frequency.

// Personalized autocomplete.
let req = Suggest { prefix: "jazz pia".into(), for_user: Some(EntityId::new(123)), limit: 5 };
let suggestions = db.suggest(&req)?;

// Trending searches (empty prefix).
let req = Suggest { prefix: "".into(), for_user: None, limit: 10 };
let trending = db.suggest(&req)?;

Note: for_user is Option<EntityId>, not Option<u64>.

---

Filters

All filters are composable. Any combination of filters produces a valid, efficiently-executed query. Filters use the FilterExpr type from tidaldb::storage::indexes::filter::FilterExpr.

Content Attribute Filters

use tidaldb::storage::indexes::filter::FilterExpr;

FilterExpr::eq("category", "jazz")           // exact match on category
FilterExpr::eq("format", "video")            // exact match on format
FilterExpr::Tag("tutorial".to_string())      // tag match
FilterExpr::CreatorEq(100)                   // exact match on creator ID
FilterExpr::DurationMin(60)                  // minimum duration (seconds)
FilterExpr::DurationMax(600)                 // maximum duration (seconds)
FilterExpr::CreatedAfter(ts_nanos)           // created after timestamp (nanoseconds)
FilterExpr::CreatedBefore(ts_nanos)          // created before timestamp (nanoseconds)

Note: FilterExpr::eq() only routes "category" and "format" to typed variants. For tags, use FilterExpr::Tag(...) directly.

Engagement Threshold Filters

FilterExpr::MinSignal { signal: "view".into(), threshold: 10000.0 }
FilterExpr::MaxSignal { signal: "view".into(), threshold: 5000.0 }

Geographic Filters

FilterExpr::NearLocation { lat: 40.7128, lng: -74.0060, radius_km: 50.0 }

Collection Filters

use tidaldb::entities::CollectionId;

FilterExpr::InCollection(CollectionId::new(42))

See USE_CASES.md Appendix A for the complete filter reference.

---

Sort Modes

Sort modes are embedded in ranking profiles. The application names a profile. The database executes the ranking pipeline. 25 built-in profiles cover the most common sort needs.

Profile	Sort Mode
new	created_at DESC
trending	Engagement velocity
hot	Score / (age + 2)^gravity
top_week / top_month / top_all_time	Cumulative quality by window
most_viewed / most_liked	Signal count by window
most_commented / most_shared	Signal count (AllTime)
hidden_gems	High quality, low reach
controversial	max(positive * negative signals)
shuffle	Random, quality-weighted
live	Live viewer count DESC
date_saved	When user bookmarked DESC
alphabetical_asc / alphabetical_desc	Title A-Z / Z-A
shortest / longest	Duration ASC / DESC

See USE_CASES.md Appendix B for the complete sort mode reference.

---

Diversity Constraints

Diversity is a post-scoring pass. After candidates are scored, diversity constraints reorder the result set to enforce variety -- without reducing the result count.

use tidaldb::ranking::diversity::DiversityConstraints;

let diversity = DiversityConstraints::new()
    .max_per_creator(2)   // No more than 2 items per creator
    .format_mix(0.4);     // No format > 40% of results

let query = Retrieve::builder()
    .profile("for_you")
    .for_user(123)
    .diversity(diversity)
    .limit(50)
    .build()?;

Diversity is specified per query or per ranking profile. Query-level diversity overrides the profile default.

---

Pagination

Cursor-based pagination for stable result sets across pages.

use tidaldb::query::retrieve::Retrieve;

// First page.
let query = Retrieve::builder()
    .for_user(123)
    .profile("for_you")
    .limit(50)
    .build()?;
let page1 = db.retrieve(&query)?;

// Next page -- pass the cursor from the previous response.
if let Some(cursor) = page1.next_cursor {
    let query = Retrieve::builder()
        .for_user(123)
        .profile("for_you")
        .cursor(cursor)
        .limit(50)
        .build()?;
    let page2 = db.retrieve(&query)?;
}

Alternatively, use exclude to exclude previously returned items:

let seen_ids: Vec<_> = page1.items.iter().map(|r| r.entity_id).collect();
let query = Retrieve::builder()
    .for_user(123)
    .profile("for_you")
    .exclude(seen_ids)
    .limit(50)
    .build()?;
let page2 = db.retrieve(&query)?;

---

Response Format

RETRIEVE Response

pub struct Results {
    /// Ranked items with scores.
    pub items: Vec<RetrieveResult>,
    /// Cursor for fetching the next page.
    pub next_cursor: Option<Cursor>,
    /// Total candidate count before diversity/limit.
    pub total_candidates: usize,
    /// Whether all diversity constraints were satisfied.
    pub constraints_satisfied: bool,
    /// Warnings generated during query execution.
    pub warnings: Vec<String>,
    /// Session snapshot at query time (populated when `for_session` is set).
    pub session_snapshot: Option<SessionSnapshot>,
    /// The degradation level under which this query was executed.
    pub degradation_level: DegradationLevel,
    /// Per-query execution statistics (timing, candidate counts, profile name).
    pub stats: QueryStats,
}

pub struct RetrieveResult {
    /// Entity ID.
    pub entity_id: EntityId,
    /// Normalized score in [0.0, 1.0].
    pub score: f64,
    /// 1-based rank.
    pub rank: usize,
    /// Signal values that contributed to this score.
    pub signals: Vec<Signal>,
}

SEARCH Response

pub struct SearchResults {
    pub items: Vec<SearchResultItem>,
    pub next_cursor: Option<Cursor>,
    pub total_candidates: usize,
    pub constraints_satisfied: bool,
    pub warnings: Vec<String>,
    pub session_snapshot: Option<SessionSnapshot>,
    pub degradation_level: DegradationLevel,
    pub stats: QueryStats,
}

pub struct SearchResultItem {
    pub entity_id: EntityId,
    pub score: f64,
    pub rank: usize,
    pub bm25_score: Option<f32>,
    pub semantic_score: Option<f32>,
    pub signals: Vec<Signal>,
    pub metadata: Option<HashMap<String, String>>,
}

The application uses items to render the UI. It uses signals to display engagement counts (views, likes, etc.). It never re-ranks -- the order from tidalDB is the final order.

---

Lifecycle and Operations

Shutdown

// Graceful shutdown -- flushes WAL, checkpoints signal state, persists indexes.
db.close()?;
// Or equivalently:
db.shutdown()?;

Health Check

db.health_check()?; // Returns Ok(()) if operational.

Item Count

let count: u64 = db.item_count(); // Number of items in the universe bitmap.

Reading Signal State

use tidaldb::schema::{EntityId, Window};

// Read decay score (applies lazy decay to current time).
let score: Option<f64> = db.read_decay_score(EntityId::new(1), "view", 0)?;

// Read windowed event count.
let count: u64 = db.read_windowed_count(EntityId::new(1), "view", Window::OneHour)?;

// Read velocity (events per second).
let velocity: f64 = db.read_velocity(EntityId::new(1), "view", Window::OneHour)?;

Saved Searches

use tidaldb::schema::{EntityId, Timestamp};

// Save a search as a persistent feed.
db.save_search(EntityId::new(123), "Jazz tutorials", "jazz tutorial", None)?;

// Query a saved search for new results since a timestamp.
let results = db.retrieve_saved_search(EntityId::new(123), "Jazz tutorials", Some(since))?;

// List all saved searches for a user.
let searches = db.list_saved_searches(EntityId::new(123))?;

// Delete a saved search.
db.delete_saved_search(EntityId::new(123), "Jazz tutorials")?;

Collections

use tidaldb::schema::EntityId;
use tidaldb::entities::collection::Visibility;

// Create a user collection (playlist, board, etc.)
let collection_id = db.create_collection(EntityId::new(123), "Jazz Favorites", Visibility::Private)?;

// Add an item to a collection.
db.add_to_collection(collection_id, EntityId::new(1))?;

// Remove an item from a collection.
db.remove_from_collection(collection_id, EntityId::new(1))?;

// List collections for a user.
let collections = db.list_collections(EntityId::new(123))?;

Text Index Management

// Force a synchronous commit and reload of the text index.
// Useful in tests after writing items to make them immediately searchable.
db.flush_text_index()?;
db.flush_creator_text_index()?;

// Manual reload (for ephemeral mode).
db.reload_text_index()?;

---

Summary

Operation	What the Application Does	What tidalDB Does
Ingest content	Compute embedding, call write_item_with_metadata + write_item_embedding	Index text, insert vector, initialize signals, apply cold start
Record engagement	Call signal with event type	Update signal ledger, WAL-backed durability
Record engagement with context	Call signal_with_context with user/creator IDs	Update ledger + user preferences + interaction weights + cohort attribution
Serve a feed	Call retrieve with a profile name	Candidate retrieval, scoring, diversity enforcement, pagination
Search	Embed query, call search	BM25 + ANN + RRF fusion + personalization + diversity
Handle cold start	Nothing	Exploration budget, population priors -- automatic
Handle negative signals	Call signal with skip/hide	Preference decay, exclusion in future queries
Scope trending by cohort	Specify cohort name in retrieve query	Cohort-scoped signal aggregation, same ranking profile
Search within scope	Specify within on search query	Intersects text/vector retrieval with scoped candidate set
HTTP write	POST /items, /embeddings, /signals	Same as library write path, via JSON
HTTP query	GET /feed, /search	Same as library query, via query params

One process. One query interface. One operational model.

---

HTTP Server API

tidal-server is a standalone HTTP server wrapping the Rust library API. It exposes a REST interface for write, query, and health operations.

Running the Server

# Ephemeral (in-memory) mode on default port 9400.
tidal-server standalone

# Persistent mode with custom schema.
tidal-server standalone --data-dir /var/lib/tidaldb --schema config/schema.yaml

# Custom port and metrics endpoint.
PORT=8080 tidal-server standalone --metrics 127.0.0.1:9091

Flag / Env	Default	Description
--data-dir <PATH>	ephemeral	Persistent data directory
--schema <PATH>	bundled default	YAML schema file
--metrics <ADDR>	disabled	Bind Prometheus metrics server
PORT	9400	Listen port
TIDAL_API_KEY	unset (no auth)	Bearer token for protected routes

Authentication

When TIDAL_API_KEY is set, all write and query endpoints require a Bearer token:

Authorization: Bearer <your-api-key>

Health endpoints are always public. If the key is missing or invalid, the server returns:

HTTP/1.1 401 Unauthorized
WWW-Authenticate: Bearer

{"error": "missing or invalid api key"}

Middleware

Protected routes have these limits applied:

Layer	Behavior
Body limit	2 MB max request body (413 if exceeded)
Timeout	30 second wall-clock limit (408 if exceeded)
Concurrency	100 max in-flight requests (queued beyond that)

Health endpoints are exempt from timeout and concurrency limits so probes are never dropped under load.

Write Endpoints

POST /items

Create an item with metadata.

{
  "entity_id": 1,
  "metadata": {
    "title": "Introduction to Jazz Piano",
    "category": "music",
    "tags": "jazz,piano,tutorial"
  }
}

Response: 201 Created (no body)

POST /embeddings

Write an embedding vector for an item.

{
  "entity_id": 1,
  "values": [0.1, 0.2, 0.3, ...]
}

Response: 204 No Content

POST /signals

Record an engagement signal. user_id and creator_id are optional — when provided, the signal updates user preferences and interaction weights (equivalent to signal_with_context in the library API).

{
  "entity_id": 1,
  "signal": "view",
  "weight": 1.0,
  "user_id": 123,
  "creator_id": 100
}

Response: 204 No Content

Query Endpoints

GET /feed

Retrieve a ranked feed.

Parameter	Type	Default	Description
user_id	u64	—	User for personalization (optional)
profile	string	"for_you"	Ranking profile name
limit	u32	20	Max results (clamped to 1000)
similar_to	u64	—	Seed item for "more like this": with profile=related, sources candidates by ANN nearest-neighbor over this item's embedding
region	string	—	Target region (cluster mode only; rejected with 400 in standalone)

GET /feed?user_id=123&profile=trending&limit=25
GET /feed?profile=related&similar_to=42&limit=20

Response:

{
  "items": [
    {
      "entity_id": 42,
      "score": 0.95,
      "rank": 1,
      "signals": [
        {"name": "view", "value": 1523.0},
        {"name": "like", "value": 89.0}
      ]
    }
  ],
  "total_candidates": 1000,
  "region": null,
  "unavailable_shards": ["group-2"]
}

The signals field is omitted when empty. region is the region the feed was served from in cluster mode (null standalone). unavailable_shards is present only when a cross-shard read was degraded — it lists the shard groups that could not be reached, so a partial page (fewer items, lower total_candidates) is never silently indistinguishable from a complete one. It is omitted on a complete read.

GET /search

Text search with optional personalization.

Parameter	Type	Default	Description
query	string	—	Search text (required)
user_id	u64	—	User for personalization (optional)
limit	u32	20	Max results (clamped to 1000)
region	string	—	Target region (cluster mode only; rejected with 400 in standalone)

GET /search?query=jazz+piano&user_id=123&limit=10

Response:

{
  "items": [
    {
      "entity_id": 42,
      "score": 0.88,
      "rank": 1,
      "bm25_score": 12.5,
      "semantic_score": 0.92
    }
  ],
  "total_candidates": 50,
  "region": null,
  "unavailable_shards": ["group-2"]
}

bm25_score and semantic_score are omitted when not applicable. region and unavailable_shards behave exactly as on GET /feed: region is the serving region in cluster mode (null standalone), and unavailable_shards is present only when a cross-shard read was degraded (partial page) and omitted on a complete read.

POST /vector_search

Pure k-NN ANN probe over the item content vector slot — no profile scoring, fusion, or diversity. This is the recall-measurement / raw nearest-neighbor surface (used by the tidal-stress --verify-recall harness).

Request body:

{
  "vector": [0.013, -0.41, 0.22],
  "k": 10,
  "ef_search": 200
}

Field	Type	Default	Description
vector	f32[]	—	Dense query vector; must match the item content slot's dimensionality (required)
k	u32	10	Number of nearest neighbors to return (clamped to 1000)
ef_search	u32	—	Optional per-request HNSW beam-width override (the recall/latency knob); omitted = the slot's configured default

Response:

{
  "items": [
    {"entity_id": 42, "distance": 0.018},
    {"entity_id": 17, "distance": 0.041}
  ],
  "region": null,
  "unavailable_shards": ["group-2"]
}

Each match carries entity_id and distance (L2-squared distance from the query vector, lower = more similar; for the L2-normalized vectors tidalDB stores this lies in [0.0, 4.0] and is monotonic with cosine distance), ordered closest-first. region is always null here (the probe serves locally — standalone, or merged across a node's hosted shard groups in cluster mode). unavailable_shards is present only when a cross-shard probe was degraded (partial nearest-set).

Health Endpoints

Endpoint	Auth	Description
GET /health	No	Readiness probe — 200 when ready, 503 when shutting down
GET /health/startup	No	Startup probe — always 200
GET /health/live	No	Liveness probe — always 200
GET /openapi.json	No	Machine-readable OpenAPI 3.1 spec for this server (data + cluster routes)

These map directly to Kubernetes startup/liveness/readiness probes — see docs/runbooks/kubernetes.md.

GET /openapi.json returns the canonical, served HTTP API contract. It is generated from the handlers (so it never drifts from the code) and is exempt from auth — clients need it to learn how to authenticate. Load it into any OpenAPI viewer or generate a client:

curl -s http://localhost:9400/openapi.json | jq '.info, (.paths | keys)'

The cluster server serves a superset document that also includes the /cluster/* and /sharded/* routes. See docs/guides/server-deployment.md.

Cluster Endpoints

These routes exist only on cluster (region) nodes — they are absent from the standalone server. Status routes are read-only; the rebalance verbs require an admin API key. Full operational detail is in docs/guides/server-deployment.md and docs/runbooks/kubernetes.md.

Method	Path	Purpose
GET	/cluster/status	Aggregated cluster view — leader, relay high-water-mark, per-region lag, and a per-shard-group shards array
GET	/cluster/status/local	This node's local view — its hosted shard groups and which leaderships it holds
POST	/cluster/shards/{id}/transfer	Transfer shard group {id}'s leadership to a named replica (fenced promote; body {"region": "<name>"})
POST	/cluster/shards/{id}/replicas	Add or remove a replica of shard group {id} (body {"action": "add"|"remove", "name": ..., "grpc_addr": ..., "http_addr": ...}; addrs required for add)

The cluster node also exposes the membership/recovery verbs POST /cluster/join, /cluster/promote, /cluster/members{,/remove}, /cluster/partition, /cluster/heal, /cluster/catchup, /cluster/reseed, and /cluster/reconcile, plus the sharded data surface POST /sharded/{items,embeddings,signals} and GET /sharded/{feed,search}.

/sharded/* writes are SINGLE-COPY and require an explicit opt-in

POST /sharded/{items,embeddings,signals} hash-partitions by entity_id and applies the write to the owning region's local store with no WAL append. It does not ride the leader relay, so the data has redundancy 1 regardless of the replication factor — a cluster running RF3 with ack: quorum does not replicate these writes. That is by design: it buys parallel write throughput across shard owners (measured 3,669 signals/s vs ~90/s on the replicated path).

Because the endpoint previously answered 201/204 with nothing saying so, it now requires x-tidal-ack: local and returns 400 without it, with a body naming the header and the replicating alternative:

# Rejected — 400, single-copy durability was never acknowledged.
curl -X POST "$BASE/sharded/items" -H "authorization: Bearer $KEY" \
  -d '{"entity_id": 7, "metadata": {"title": "..."}}'

# Accepted — 201, and the caller has stated it accepts redundancy 1.
curl -X POST "$BASE/sharded/items" -H "authorization: Bearer $KEY" \
  -H 'x-tidal-ack: local' \
  -d '{"entity_id": 7, "metadata": {"title": "..."}}'

For a replicated write use POST /items, /embeddings, or /signals — those ride the leader WAL relay and honor x-tidal-ack: leader|quorum. local is rejected there (x-tidal-ack must be "leader" or "quorum"), because those routes always replicate.

GET /sharded/{feed,search} are reads and are unaffected — no header needed.

Cluster-Node-Only Data Endpoints

Method	Path	Purpose
POST	/hardnegs	Record a hide hard-negative for (user, item) (body {"user_id": ..., "item_id": ...}); 204 No Content. Cluster-node-only — converges via the /cluster/reconcile LWW snapshot

POST /vector_search (documented above) is served by both the standalone and cluster servers.

GET /health response:

{"ok": true, "service": "tidaldb", "mode": "standalone", "items": 1234}

During shutdown:

HTTP/1.1 503 Service Unavailable

{"ok": false, "service": "tidaldb", "cause": "shutting down"}

Error Responses

All errors return JSON with an error field:

{"error": "description of what went wrong"}

Condition	HTTP Status
Invalid input, bad schema	400 Bad Request
Missing/invalid API key	401 Unauthorized
Policy violation, expired session	403 Forbidden
Entity not found	404 Not Found
Request timeout	408 Request Timeout
Body too large	413 Payload Too Large
Backpressure, rate limited	429 Too Many Requests
Internal error	500 Internal Server Error