# 05 -- Cohort Specification

**Status:** Draft
**Authors:** tidalDB Engineering
**Date:** 2026-02-20
**Depends on:** Entity Model (02), Signal System (03), Query Engine
**Research:** `docs/research/tidaldb_signal_ledger.md` (Section 7: Cohort-Scoped Signal Aggregation)

---

## Table of Contents

1. [Overview](#1-overview)
2. [Cohort as a First-Class Primitive](#2-cohort-as-a-first-class-primitive)
3. [Cohort Types](#3-cohort-types)
4. [Cohort Definition Language](#4-cohort-definition-language)
5. [Membership Resolution](#5-membership-resolution)
6. [The Three-Layer Trending Model](#6-the-three-layer-trending-model)
7. [Integration Architecture](#7-integration-architecture)
8. [Cohort-Scoped Ranking Profiles](#8-cohort-scoped-ranking-profiles)
9. [Hierarchical Cohort Model](#9-hierarchical-cohort-model)
10. [Cohort Analytics](#10-cohort-analytics)
11. [API Surface](#11-api-surface)
12. [Worked Example](#12-worked-example)
13. [Accuracy Analysis](#13-accuracy-analysis)
14. [Configuration and Defaults](#14-configuration-and-defaults)
15. [Scale Considerations](#15-scale-considerations)
16. [Invariants and Correctness Guarantees](#16-invariants-and-correctness-guarantees)

---

## 1. Overview

A cohort is a dynamic predicate over user attributes that defines a population segment. Cohorts are not user groups. They are not lists. A user does not "join" a cohort -- they match its predicate based on their current attributes. When a user's attributes change, their cohort memberships change automatically.

Cohorts exist to answer a question that global signal aggregates cannot:

> "What is trending for users who look like this?"

The product owner's requirement is a three-layer model:

1. **Global trending** -- what is trending everywhere.
2. **Cohort trending** -- what is trending for users matching a profile (e.g., US users aged 18-24 who like jazz).
3. **Search within cohort trending** -- text and semantic search constrained to the cohort-trending candidate set.

Each layer builds on the previous. Global trending uses global signal aggregates (already designed in the Signal System spec, Section 6, Level 0). Cohort trending uses the hierarchical dimensional rollup system (Signal System spec, Section 7, Levels 1-2). Search within cohort trending composes text/semantic retrieval with the cohort-scoped candidate set.

This specification defines cohorts as a first-class primitive that connects the Entity Model's rich user attributes, the Signal System's dimensional rollup architecture, and the Query Engine's retrieval and ranking pipeline.

---

## 2. Cohort as a First-Class Primitive

### What a Cohort Is

A cohort is a **named predicate over user attributes** that resolves, at query time, to a set of user IDs. The predicate is evaluated against the User entity's metadata fields -- both application-set fields (region, locale, age_range) and database-computed fields (engagement_level, inferred_interests).

```
Cohort "young_us_jazz":
    Predicate: region:US AND age_range:18-24 AND inferred_interests CONTAINS jazz
    Resolution: bitmap of user IDs matching this predicate
    Signal scope: aggregate signals only from users in this bitmap
```

### What a Cohort Is Not

**Not a user group.** A cohort has no membership list that someone manages. Users match or do not match a predicate. There is no "add user to cohort" operation.

**Not a segment stored on the user.** Users do not carry a `cohorts` field. Membership is computed from attributes. If a user moves from the US to Japan, they stop matching `region:US` cohorts and start matching `region:JP` cohorts -- without any explicit membership update.

**Not a filter on items.** A cohort defines a population of users, not a subset of items. The items that "trend in a cohort" are items that users in that cohort engage with at high velocity. The cohort constrains the signal aggregation, not the item candidate set.

**Not an audience.** Cohorts are not used for targeting or ad delivery. They are used to scope signal aggregation for ranking queries. "What is trending among young US jazz fans" is a ranking question, not a targeting question.

### Why Cohorts Are Necessary

Global trending surfaces content that appeals to the broadest audience. This is useful but incomplete. A jazz video gaining rapid traction among 18-24 year old US users will never appear on a global trending list dominated by gaming and pop music. But for a user who matches that cohort, that jazz video is the most relevant trending result.

Without cohorts, the application must:
1. Maintain its own user segmentation system
2. Track per-segment signal aggregates in a feature store
3. Build custom trending logic per segment
4. Stitch these together with the ranking service

This is the feature-store pattern that tidalDB replaces. Cohorts are the mechanism by which it replaces it.

---

## 3. Cohort Types

### 3.1 Static Cohorts

Predicates over immutable or slow-changing user attributes. Membership changes rarely -- only when the user explicitly updates their profile.

```
DEFINE COHORT us_english AS region:US AND locale IN (en-US, en-GB)
DEFINE COHORT gen_z AS age_range IN (13-17, 18-24)
DEFINE COHORT premium AS account_type:premium
```

**Resolution strategy:** Pre-computed roaring bitmap, cached indefinitely. Invalidated and recomputed only when a user's matching attribute changes via `update_user()`. Because the underlying attributes are application-set and change infrequently, the bitmap is effectively static.

**Refresh cost:** O(1) per user attribute change (bitmap flip). Full recomputation is O(users) but only triggered on schema change.

### 3.2 Dynamic Cohorts

Predicates over database-computed attributes. Membership changes as user behavior changes, on the background computation schedule defined in the Entity Model spec.

```
DEFINE COHORT power_users AS engagement_level:power_user
DEFINE COHORT jazz_fans AS inferred_interests CONTAINS jazz
DEFINE COHORT binge_watchers AS session_pattern:binge AND content_format_preference:long
```

**Resolution strategy:** Roaring bitmap refreshed on the same schedule as the underlying computed field. `engagement_level` is recomputed every 6 hours (Entity Model spec, Section: Field Writability Model), so the `power_users` cohort bitmap is at most 6 hours stale. `inferred_interests` is recomputed hourly (incremental) and daily (full), so `jazz_fans` reflects interests within the last hour.

**Refresh cost:** Piggybacks on the existing computed field refresh. No additional computation -- the bitmap is updated as a side effect of the computed field update.

### 3.3 Hybrid Cohorts

Predicates combining static and dynamic attributes. The most common cohort type in practice.

```
DEFINE COHORT young_us_jazz AS
    region:US AND age_range:18-24 AND inferred_interests CONTAINS jazz
```

**Resolution strategy:** Bitmap intersection of the static components (region:US, age_range:18-24) with the dynamic component (inferred_interests CONTAINS jazz). The static bitmaps are cached. The dynamic bitmap is refreshed on schedule. Intersection is computed on demand or cached with the staleness of the most-stale component.

### 3.4 Ad-hoc Cohorts

Inline predicates in a query, not named or saved. Used for exploratory queries and one-off analytics.

```
RETRIEVE items
USING PROFILE trending
FOR COHORT region:JP AND age_range:25-34
WINDOW 24h
LIMIT 25
```

**Resolution strategy:** Computed at query time from the predicate. Bitmaps for individual attribute values are always available (they are the term-to-bitmap indexes from the Entity Model spec, Section: Cohort-Ready Design). The compound bitmap is the intersection of these per-value bitmaps. Resolution cost depends on predicate complexity but is bounded by the bitmap intersection performance target (<5ms for compound predicates).

**Caching:** Ad-hoc cohort bitmaps are not cached between queries. If the same ad-hoc predicate appears frequently, the application should define it as a named cohort to benefit from caching.

---

## 4. Cohort Definition Language

### 4.1 Predicate Syntax

Cohort predicates are boolean expressions over user attribute fields. Every field on the User entity (both application-set and database-computed) is a valid predicate dimension.

**Simple equality:**
```
region:US
engagement_level:power_user
account_type:premium
```

**Set membership (IN):**
```
locale IN (en-US, en-GB, en-AU)
age_range IN (18-24, 25-34)
```

**Contains (for keywords fields):**
```
inferred_interests CONTAINS jazz
explicit_interests CONTAINS cooking
primary_categories CONTAINS music
```

**Range predicates (for numeric fields):**
```
birth_year:1995-2005
platform_tenure_days > 365
daily_active_hours >= 4.0
followed_creator_count:100-1000
```

**Negation:**
```
NOT engagement_level:dormant
NOT account_type:admin
```

**Compound predicates (AND/OR/NOT with grouping):**
```
region:US AND age_range:18-24 AND inferred_interests CONTAINS jazz
(region:US OR region:CA) AND age_range:18-24
region:US AND NOT engagement_level:dormant
(locale IN (en-US, en-GB) OR language:en) AND engagement_level:power_user
```

### 4.2 Named Cohort Definition

Named cohorts are defined in schema and persist across queries. They are the recommended approach for any cohort used more than once.

```rust
db.define_cohort(CohortDef {
    name: "young_us_jazz",
    predicate: Predicate::and(vec![
        Predicate::eq("region", "US"),
        Predicate::eq("age_range", "18-24"),
        Predicate::contains("inferred_interests", "jazz"),
    ]),
})?;

db.define_cohort(CohortDef {
    name: "latam_power_users",
    predicate: Predicate::and(vec![
        Predicate::in_set("region", &["BR", "MX", "AR", "CO", "CL"]),
        Predicate::eq("engagement_level", "power_user"),
    ]),
})?;

db.define_cohort(CohortDef {
    name: "long_form_enthusiasts",
    predicate: Predicate::and(vec![
        Predicate::eq("content_format_preference", "long"),
        Predicate::gt("daily_active_hours", 2.0),
        Predicate::not(Predicate::eq("engagement_level", "dormant")),
    ]),
})?;
```

**Text DSL equivalent (for query strings and configuration):**
```
DEFINE COHORT young_us_jazz AS region:US AND age_range:18-24 AND inferred_interests CONTAINS jazz
DEFINE COHORT latam_power_users AS region IN (BR, MX, AR, CO, CL) AND engagement_level:power_user
DEFINE COHORT long_form_enthusiasts AS content_format_preference:long AND daily_active_hours > 2.0 AND NOT engagement_level:dormant
```

### 4.3 Predicate Validation Rules

1. Every field referenced in a predicate must exist on the User entity. Referencing a non-existent field returns `SchemaError::UnknownField`.
2. Predicate operators must match the field type. `>` on a keyword field returns `SchemaError::TypeMismatch`. `CONTAINS` on a non-keywords field returns `SchemaError::TypeMismatch`.
3. Cohort names must be unique. Redefining a cohort with the same name replaces the previous definition (the bitmap is recomputed on the next refresh cycle).
4. Maximum predicate depth is 8 levels of nesting. This prevents pathological evaluation but allows all practical cohort definitions.
5. Maximum 500 named cohorts. This is a practical limit on the schema catalog, not on query-time ad-hoc cohorts which are unlimited.

### 4.4 Predicate Type Reference

| Operator | Applicable Field Types | Bitmap Operation | Example |
|----------|----------------------|------------------|---------|
| `:` (equality) | keyword, computed(keyword) | Direct bitmap lookup | `region:US` |
| `IN` | keyword, computed(keyword) | Union of bitmaps per value | `region IN (US, CA, MX)` |
| `CONTAINS` | keywords, computed(keywords) | Direct bitmap lookup per value | `inferred_interests CONTAINS jazz` |
| `>`, `>=`, `<`, `<=` | i64, f64, computed(i64), computed(f64) | Range scan on sorted numeric index | `platform_tenure_days > 365` |
| `range` (a-b) | i64, f64, computed(i64), computed(f64) | Range scan on sorted numeric index | `birth_year:1995-2005` |
| `NOT` | any predicate | Bitmap complement | `NOT engagement_level:dormant` |
| `AND` | predicates | Bitmap intersection | `region:US AND age_range:18-24` |
| `OR` | predicates | Bitmap union | `region:US OR region:CA` |

---

## 5. Membership Resolution

### 5.1 Resolution Mechanism

Cohort membership is resolved using the roaring bitmap indexes maintained by the Entity Model (spec 02, Section: Cohort-Ready Design). Every keyword and keywords field on the User entity has a term-to-bitmap index. Every numeric field has a sorted numeric index that supports range predicate resolution to bitmaps.

```
Resolution of "region:US AND age_range:18-24 AND inferred_interests CONTAINS jazz":

Step 1: region_bitmap["US"]              --> bitmap A  (all US users)
Step 2: age_range_bitmap["18-24"]        --> bitmap B  (all 18-24 users)
Step 3: interests_bitmap["jazz"]         --> bitmap C  (all jazz-interested users)
Step 4: A AND B AND C                    --> bitmap D  (the cohort)

Bitmap D is the cohort's resolved membership.
Cardinality: |D| = roaring::cardinality(D)
```

### 5.2 Resolution Latency Targets

| Cohort Type | Resolution Target | Mechanism |
|-------------|------------------|-----------|
| Named static cohort | < 1ms | Pre-computed bitmap, cached in memory |
| Named dynamic cohort | < 1ms | Pre-computed bitmap, refreshed on schedule |
| Named hybrid cohort | < 2ms | Intersection of cached static + cached dynamic |
| Ad-hoc, 1 predicate term | < 1ms | Single bitmap lookup |
| Ad-hoc, 2-3 predicate terms (AND) | < 2ms | 2-3 bitmap intersections |
| Ad-hoc, 4+ predicate terms | < 5ms | Multiple bitmap operations |
| Ad-hoc with range predicates | < 5ms | Range scan + bitmap intersection |
| Ad-hoc with NOT | < 3ms | Bitmap complement + intersection |

These targets assume 10M users and the bitmap memory budget of ~630 MB from the Entity Model spec.

### 5.3 Bitmap Caching Strategy

**Named cohorts:** The resolved bitmap is cached in memory alongside the cohort definition. Cache lifetime depends on cohort type:

| Cohort Type | Cache Lifetime | Invalidation Trigger |
|-------------|---------------|---------------------|
| Static | Indefinite | Any `update_user()` that changes a matching field |
| Dynamic | Matches computed field refresh interval | Background materializer recomputes the underlying field |
| Hybrid | Min(static lifetime, dynamic refresh interval) | Either trigger above |

**Invalidation mechanism for static cohorts:** When `update_user()` modifies a field referenced by any named cohort's predicate, the affected cohort bitmaps are marked dirty. Recomputation is deferred to the next read (lazy) or the next background cycle (eager, default). The choice is configurable:

```rust
CohortConfig {
    // Eager: recompute bitmap immediately on user attribute change.
    // Higher write-path cost, always-fresh bitmaps.
    // Lazy: mark dirty, recompute on next query.
    // Lower write-path cost, first query after change pays recomputation.
    invalidation: CohortInvalidation::Eager,  // default
}
```

In practice, for static cohorts, the invalidation cost is trivial: flipping one bit in a roaring bitmap per user update. Eager invalidation is the right default.

**Dynamic cohort refresh:** Dynamic cohort bitmaps are refreshed by the background materializer as a side effect of computed field updates. When `engagement_level` is recomputed for a batch of users, every named cohort with `engagement_level` in its predicate has its bitmap updated in the same pass. No separate cohort refresh job is needed.

### 5.4 Integration with Signal System Dimensional Hierarchy

The Signal System spec (Section 7) defines a three-level dimensional hierarchy for cohort-scoped signal aggregation:

```
Level 0: GLOBAL                           -- one counter per item per signal per window
Level 1: PRIMARY DIMENSIONS               -- region (~20), language (~30), age_group (6)
Level 2: BEHAVIORAL SEGMENTS              -- up to 100 application-defined segments
Level 3: COMPOSITE (query-time estimate)  -- intersection of Level 1 and Level 2
```

Cohort membership resolution feeds directly into this hierarchy:

| Cohort Predicate | Dimensional Level | Signal Aggregation Path |
|-----------------|-------------------|------------------------|
| Single Level 1 dimension (e.g., `region:US`) | Level 1 | Exact rollup lookup |
| Single Level 2 segment (e.g., `engagement_level:power_user`) | Level 2 | Exact rollup lookup |
| Multiple Level 1 dimensions (e.g., `region:US AND age_range:18-24`) | Level 3 | Independence estimation from Level 1 rollups |
| Level 1 + Level 2 (e.g., `region:US AND jazz_fans`) | Level 3 | Independence estimation from Level 1 + Level 2 |
| Named cohort registered as Level 2 segment | Level 2 | Exact rollup lookup |

**The key design decision:** Any named cohort can optionally be registered as a Level 2 behavioral segment, which activates exact counter tracking at signal write time. This trades write amplification for query accuracy. The threshold for when to promote a cohort to Level 2 is discussed in Section 13 (Accuracy Analysis).

```rust
db.define_cohort(CohortDef {
    name: "young_us_jazz",
    predicate: Predicate::and(vec![
        Predicate::eq("region", "US"),
        Predicate::eq("age_range", "18-24"),
        Predicate::contains("inferred_interests", "jazz"),
    ]),
    // Promote to Level 2 segment for exact signal tracking.
    // Costs ~1 additional counter increment per signal write
    // from users matching this cohort, but provides exact
    // cohort-scoped signal aggregates instead of estimates.
    exact_tracking: true,
})?;
```

---

## 6. The Three-Layer Trending Model

This is the organizing principle of the entire cohort system. Every feature, every API extension, and every storage decision exists to serve this three-layer model.

### 6.1 Layer 1: Global Trending

**What is trending everywhere?**

```
RETRIEVE items
USING PROFILE trending
WINDOW 24h
LIMIT 25
```

This query uses Level 0 (global) signal aggregates. It is already fully specified in the Signal System spec. No cohort resolution is involved. The ranking profile `trending` reads global velocity signals (share velocity, view velocity, engagement ratio) and ranks by pure signal momentum.

**Signal path:** Global counters in the hot tier and warm tier. O(1) per entity per signal. Exact.

**Latency target:** < 20ms for 25 results.

### 6.2 Layer 2: Cohort Trending

**What is trending for users matching a profile?**

```
RETRIEVE items
USING PROFILE trending
FOR COHORT young_us_jazz
WINDOW 24h
LIMIT 25
```

This query scopes signal aggregation to users matching the `young_us_jazz` cohort predicate. Instead of reading global view velocity, the query engine reads the cohort-scoped view velocity: "how many views did this item receive in the last 24 hours from users in this cohort?"

**Signal path depends on how the cohort maps to the dimensional hierarchy:**

**Case A -- Single primary dimension (exact):**
```
RETRIEVE items USING PROFILE trending FOR COHORT region:US WINDOW 24h LIMIT 25
```
Maps to Level 1 rollup for `region:US`. Direct counter lookup. Exact.

**Case B -- Named cohort registered as Level 2 segment (exact):**
```
RETRIEVE items USING PROFILE trending FOR COHORT young_us_jazz WINDOW 24h LIMIT 25
```
If `young_us_jazz` has `exact_tracking: true`, it is a Level 2 behavioral segment with its own counters. Direct counter lookup. Exact.

**Case C -- Composite query (estimated):**
```
RETRIEVE items USING PROFILE trending FOR COHORT region:US AND age_range:18-24 WINDOW 24h LIMIT 25
```
No exact counters for this intersection. Estimated from Level 1 rollups using the independence assumption:
```
C(region:US AND age_range:18-24) ~= C(region:US) * C(age_range:18-24) / C(global)
```
Accuracy: ~85-95% for weakly correlated dimensions (Section 13).

**Latency target:** < 50ms for 25 results (includes cohort resolution + signal aggregation + ranking).

### 6.3 Layer 3: Search Within Cohort Trending

**Text or semantic search constrained to what is trending in a cohort.**

```
SEARCH items
QUERY "piano"
WITHIN TRENDING FOR COHORT young_us_jazz
WINDOW 24h
LIMIT 20
```

This is the most complex query in the system. It composes three operations:

1. **Cohort resolution:** Resolve `young_us_jazz` to a user bitmap.
2. **Cohort trending candidate generation:** Identify items with high cohort-scoped velocity in the 24h window. This produces a candidate set (e.g., the top 500 items trending in this cohort).
3. **Search within candidates:** Apply BM25 and/or semantic search for "piano" within the candidate set only. Rank by text relevance, re-weighted by cohort trending score.

**Execution plan:**

```
Step 1: Resolve cohort "young_us_jazz"           --> bitmap D (user set)
            Cost: < 2ms (cached bitmap intersection)

Step 2: Generate cohort trending candidates
            Read cohort-scoped velocity for all items with cohort tracking active
            Filter to items with velocity above threshold
            Sort by cohort velocity
            Take top 500 candidates
            Cost: < 20ms (scan 100K cohort-tracked items)

Step 3: Apply text search "piano" within 500 candidates
            BM25 score against inverted index, intersected with candidate set
            Optional: semantic search with query embedding
            Hybrid fusion (RRF or weighted) if both text and vector
            Cost: < 10ms (inverted index lookup + candidate intersection)

Step 4: Final ranking
            Combine text relevance score with cohort velocity score
            Apply diversity constraints
            Return top 20
            Cost: < 5ms

Total: < 37ms (within 50ms budget)
```

**Query semantics:** `WITHIN TRENDING` means "restrict the candidate set to items that are currently trending in this scope." It is not a filter (which would eliminate items from an existing candidate set) -- it is a candidate generation strategy. Items not trending in the cohort are never considered, regardless of their text relevance.

**Latency target:** < 50ms for 20 results.

---

## 7. Integration Architecture

### How Cohorts Connect the Three Subsystems

```
                    ┌──────────────────────────────────────────────┐
                    │              QUERY ENGINE                     │
                    │                                              │
                    │  RETRIEVE items                              │
                    │  USING PROFILE trending                      │
                    │  FOR COHORT young_us_jazz        ┌────────┐ │
                    │  WINDOW 24h                      │ Result │ │
                    │  LIMIT 25                        │  Set   │ │
                    │                                  └────┬───┘ │
                    └──────────┬───────────────────────────┬┘─────┘
                               │                           │
                    ┌──────────▼───────────┐    ┌──────────▼──────────┐
                    │   ENTITY MODEL       │    │   SIGNAL SYSTEM     │
                    │                      │    │                     │
                    │  User attributes:    │    │  Dimensional rollups:│
                    │  - region: "US"      │    │  Level 0: global    │
                    │  - age_range: "18-24"│    │  Level 1: region,   │
                    │  - inferred_interests│    │    language, age     │
                    │    ["jazz", ...]     │    │  Level 2: segments  │
                    │                      │    │  Level 3: composite │
                    │  Bitmap indexes:     │    │    (estimated)      │
                    │  region["US"] → bmp  │    │                     │
                    │  age["18-24"] → bmp  │    │  Cohort-scoped      │
                    │  interest["jazz"]→bmp│    │  velocity per item  │
                    │                      │    │                     │
                    │  Cohort resolution:  │    │  Write-time cohort  │
                    │  A ∩ B ∩ C → bitmap D│    │  attribution:       │
                    │                      │    │  user memberships → │
                    │  UserCohortMembership│    │  counter increments │
                    │  cached per user     │    │                     │
                    └──────────────────────┘    └─────────────────────┘
                               │                           ▲
                               │    UserCohortMemberships  │
                               └───────────────────────────┘
                                 (cached on user, used at
                                  signal write time for
                                  cohort counter attribution)
```

### Data Flow: Signal Write with Cohort Attribution

When a signal event arrives (e.g., `user_123 views item_abc`):

```
1. Load user_123's UserCohortMemberships from hot-tier cache
       {region: US, language: en, age_group: 18-24, segments: [jazz_fans, power_users]}

2. Check if item_abc has cohort tracking active
       (global signal rate > COHORT_ACTIVATION_THRESHOLD)

3. If cohort tracking active:
       a. Increment global counter (Level 0)                       -- always
       b. Increment region:US counter (Level 1)                    -- from membership
       c. Increment language:en counter (Level 1)                  -- from membership
       d. Increment age_group:18-24 counter (Level 1)             -- from membership
       e. Increment jazz_fans segment counter (Level 2)            -- from membership
       f. Increment power_users segment counter (Level 2)          -- from membership
       g. If young_us_jazz has exact_tracking:                     -- named cohort
          Increment young_us_jazz segment counter (Level 2)

4. If cohort tracking not active:
       a. Increment global counter only (Level 0)
       b. Check if global counter crossed activation threshold
          If yes, activate cohort tracking for item_abc
```

### Data Flow: Cohort Trending Query

When a `FOR COHORT` query arrives:

```
1. Resolve cohort predicate to query plan
       Parse "young_us_jazz" → lookup named cohort definition
       Determine dimensional mapping:
         - If exact_tracking: true → Level 2 segment lookup
         - If single Level 1 dimension → Level 1 rollup lookup
         - If composite → independence estimation

2. For each candidate item (items with cohort tracking active):
       Read cohort-scoped signal aggregates per the query plan
       Compute velocity within the requested window

3. Rank candidates by cohort-scoped velocity
       Apply ranking profile (trending: velocity-dominant)
       Apply diversity constraints
       Return top-K results
```

---

## 8. Cohort-Scoped Ranking Profiles

### 8.1 Cohort Trending as a Boost

Ranking profiles can reference cohort trending as a boost signal. This enables "For You, weighted toward what is trending among people like you."

```rust
db.define_profile(ProfileDef {
    name: "for_you_cohort_aware",
    version: 1,
    candidate: Candidate::Ann {
        query_vector: VectorSource::UserPreference,
        index: EntityKind::Item,
        top_k: 500,
    },
    boosts: vec![
        Boost::signal("view", Window::hours(24), Velocity, 0.3),
        Boost::relationship("interaction_weight", 0.2),
        Boost::social_proof(0.15),
        // New: boost items trending in the querying user's cohort
        Boost::cohort_trending("auto", Window::hours(24), 0.2),
    ],
    // ...
})?;
```

The `Boost::cohort_trending("auto", ...)` computes the querying user's primary cohort automatically from their attributes (region + age_range + top inferred interest) and boosts items trending in that cohort. The `"auto"` parameter means "derive the cohort from the querying user's attributes." A specific cohort name can also be used:

```rust
Boost::cohort_trending("young_us_jazz", Window::hours(24), 0.2)
```

### 8.2 Cohort-Relative Scoring

A powerful discovery signal: "this item is trending MORE in this cohort than globally." An item with global velocity of 100/hour and cohort velocity of 500/hour has a cohort-relative score of 5.0 -- it is 5x more popular among this cohort than the general population. This surfaces content that is specifically resonant with a population segment.

```rust
Boost::cohort_relative("young_us_jazz", Window::hours(24), 0.25)
```

The cohort-relative score is computed as:

```
cohort_relative_score = cohort_velocity / max(global_velocity, floor)
```

Where `floor` prevents division by zero and dampens noise for low-traffic items. Default floor: 10.0 events/hour.

### 8.3 Cohort Trending as Candidate Generation

Instead of using ANN or scan for candidate generation, a ranking profile can use cohort trending as its candidate source:

```rust
db.define_profile(ProfileDef {
    name: "trending_for_you",
    version: 1,
    candidate: Candidate::CohortTrending {
        cohort: CohortSource::Auto, // derive from querying user
        window: Window::hours(24),
        top_k: 200,
    },
    boosts: vec![
        // Re-rank by user preference match
        Boost::preference_match(0.3),
        Boost::signal("completion", Window::all_time(), Value, 0.2),
    ],
    // ...
})?;
```

This generates candidates from "items trending in the user's cohort" and then re-ranks by personal preference. It answers the question: "Of the things trending among people like me, which ones match my specific taste?"

### 8.4 CohortSource Enum

```rust
pub enum CohortSource {
    /// Derive cohort from the querying user's attributes.
    /// Uses the user's region, age_range, and top inferred interest
    /// to construct an automatic cohort predicate.
    Auto,

    /// Use a specific named cohort.
    Named(String),

    /// Use an inline predicate (ad-hoc cohort).
    Predicate(Predicate),
}
```

---

## 9. Hierarchical Cohort Model

### 9.1 Natural Hierarchy

Cohorts form a natural hierarchy that mirrors the signal system's dimensional hierarchy:

```
Global (all users)
├── Region (US, EU, APAC, LATAM, ...)
│   ├── Locale (en-US, en-GB, es-MX, ...)
│   └── Region + Age (US:18-24, US:25-34, ...)
│       └── Region + Age + Interest (US:18-24:jazz, ...)
├── Language (en, es, ja, ...)
│   └── Language + Age (en:18-24, ...)
├── Age Group (13-17, 18-24, 25-34, ...)
└── Behavioral Segments (power_users, jazz_fans, ...)
    └── Region + Segment (US:jazz_fans, ...)
```

### 9.2 Roll-up and Drill-down

The hierarchy enables efficient navigation:

**Roll-up:** "Trending in US" is the parent of "Trending in US among 18-24." If the child cohort is too small to produce reliable trending data (fewer than 1000 active users), the system falls back to the parent cohort and applies a weaker cohort-relative boost.

**Drill-down:** "Trending in US" can be decomposed into "Trending in US among 18-24" vs "Trending in US among 25-34" for analytics or A/B comparison.

### 9.3 Mapping to Signal System Levels

| Hierarchy Level | Signal System Level | Counter Type | Accuracy |
|----------------|---------------------|--------------|----------|
| Global | Level 0 | Always maintained | Exact |
| Single primary dimension | Level 1 | Always maintained for active items | Exact |
| Single behavioral segment | Level 2 | Maintained for registered segments | Exact |
| Two primary dimensions | Level 3 | Estimated at query time | ~85-95% |
| Primary + behavioral | Level 3 | Estimated at query time | ~75-90% |
| Named cohort with exact_tracking | Level 2 | Maintained as explicit segment | Exact |

### 9.4 Minimum Population Threshold

Cohort-scoped trending is only meaningful when the cohort has sufficient active users to produce statistically reliable signal velocity. A cohort of 10 users cannot have meaningful "trending" content.

**Minimum population for cohort trending queries:**

| Query Type | Minimum Cohort Size | Rationale |
|-----------|-------------------|-----------|
| Cohort trending (top 25) | 1,000 active users in window | Statistical reliability of velocity |
| Cohort trending (top 10) | 500 active users in window | Smaller result set needs less data |
| Search within cohort trending | 2,000 active users in window | Needs enough trending candidates to search within |
| Cohort-relative scoring | 500 active users in window | Ratio needs denominator stability |

"Active users in window" means users in the cohort who have generated at least one signal event within the query window.

When a cohort is below the minimum population threshold, the query engine:
1. Returns a warning in the response: `CohortWarning::InsufficientPopulation { cohort, size, minimum }`.
2. Falls back to the nearest parent cohort in the hierarchy that meets the threshold.
3. Applies a cohort-relative boost from the original cohort (if any exact data exists) as a secondary signal.

---

## 10. Cohort Analytics

Platform operators need inverse queries -- not "what is trending in this cohort" but "what cohorts is this item trending in." These are operator-facing analytics, not end-user queries.

### 10.1 Item Cohort Performance

**"Which cohorts is this item performing best in?"**

```rust
let analysis = db.analyze_item_cohorts(AnalyzeItemCohorts {
    item: "item_abc",
    signal: "view",
    window: Window::hours(24),
    // Return cohorts where this item's velocity is highest
    sort: CohortAnalysisSort::AbsoluteVelocity,
    limit: 20,
})?;

// Returns:
// [
//   { cohort: "region:BR", velocity: 1200/h, relative: 3.2 },
//   { cohort: "age_range:18-24", velocity: 800/h, relative: 2.1 },
//   { cohort: "jazz_fans", velocity: 600/h, relative: 8.5 },
//   ...
// ]
```

This query iterates over all Level 1 and Level 2 dimensional rollups for the given item and signal, ranks by velocity, and returns the top cohorts. It answers: "who is this content resonating with?"

### 10.2 Cohort Velocity Anomalies

**"What cohorts are showing unusual velocity for this category?"**

```rust
let anomalies = db.detect_cohort_anomalies(CohortAnomalyDetection {
    filter: Filter::eq("category", "jazz"),
    signal: "view",
    window: Window::hours(6),
    // Detect cohorts where category velocity is > 2 standard deviations
    // above that cohort's historical baseline for this category
    threshold: AnomalyThreshold::StdDev(2.0),
})?;

// Returns:
// [
//   { cohort: "region:JP", category: "jazz", velocity: 5000/h,
//     baseline: 800/h, z_score: 3.2, since: "2h ago" },
//   ...
// ]
```

This enables alerting on unusual engagement patterns -- "jazz content is suddenly blowing up in Japan" -- which is valuable for editorial teams and content strategy.

### 10.3 Cohort Comparison

**"How does this item's performance in cohort A compare to cohort B?"**

```rust
let comparison = db.compare_cohorts(CohortComparison {
    item: "item_abc",
    cohort_a: "young_us_jazz",
    cohort_b: "gen_z",  // broader cohort
    signals: vec!["view", "like", "share", "completion"],
    window: Window::hours(24),
})?;

// Returns:
// {
//   cohort_a: { view: 600/h, like: 120/h, share: 45/h, completion: 0.82 },
//   cohort_b: { view: 200/h, like: 30/h, share: 8/h, completion: 0.65 },
//   ratios: { view: 3.0, like: 4.0, share: 5.6, completion: 1.26 },
// }
```

This supports A/B analysis of content performance across audience segments.

---

## 11. API Surface

### 11.1 Schema Operations

**Define a named cohort:**

```rust
db.define_cohort(CohortDef {
    name: "young_us_jazz",
    predicate: Predicate::and(vec![
        Predicate::eq("region", "US"),
        Predicate::eq("age_range", "18-24"),
        Predicate::contains("inferred_interests", "jazz"),
    ]),
    exact_tracking: true,   // register as Level 2 segment
})?;
```

**Text DSL:**
```
DEFINE COHORT young_us_jazz
    AS region:US AND age_range:18-24 AND inferred_interests CONTAINS jazz
    WITH EXACT TRACKING
```

**List cohorts:**
```rust
let cohorts = db.list_cohorts()?;
// Returns: Vec<CohortInfo> with name, predicate, type, cardinality, tracking mode
```

**Describe cohort:**
```rust
let info = db.describe_cohort("young_us_jazz")?;
// Returns: CohortInfo {
//   name: "young_us_jazz",
//   predicate: "region:US AND age_range:18-24 AND inferred_interests CONTAINS jazz",
//   cohort_type: CohortType::Hybrid,
//   cardinality: 42_350,
//   exact_tracking: true,
//   created_at: ...,
//   last_refreshed: ...,
// }
```

**Drop cohort:**
```rust
db.drop_cohort("young_us_jazz")?;
```

Dropping a cohort removes the definition and bitmap from the schema catalog. If the cohort had `exact_tracking: true`, the corresponding Level 2 segment counters are deallocated on the next background materializer cycle. Historical cohort-scoped signal data is retained in rollups but no longer receives new counter increments.

### 11.2 Query Extensions

**FOR COHORT clause in RETRIEVE:**

```rust
// Named cohort
let results = db.retrieve(Retrieve {
    entity: EntityKind::Item,
    profile: "trending",
    for_cohort: Some(CohortRef::Named("young_us_jazz")),
    window: Some(Window::hours(24)),
    limit: 25,
    ..Default::default()
})?;

// Ad-hoc cohort
let results = db.retrieve(Retrieve {
    entity: EntityKind::Item,
    profile: "trending",
    for_cohort: Some(CohortRef::Predicate(
        Predicate::and(vec![
            Predicate::eq("region", "JP"),
            Predicate::eq("age_range", "25-34"),
        ])
    )),
    window: Some(Window::hours(24)),
    limit: 25,
    ..Default::default()
})?;
```

**Text DSL:**
```
RETRIEVE items
USING PROFILE trending
FOR COHORT young_us_jazz
WINDOW 24h
LIMIT 25

RETRIEVE items
USING PROFILE trending
FOR COHORT region:JP AND age_range:25-34
WINDOW 24h
LIMIT 25
```

**WITHIN TRENDING FOR COHORT in SEARCH:**

```rust
let results = db.search(Search {
    query: "piano",
    within_trending: Some(WithinTrending {
        cohort: CohortRef::Named("young_us_jazz"),
        window: Window::hours(24),
        min_velocity: None,     // use default threshold
        max_candidates: 500,    // trending candidate pool size
    }),
    for_user: Some("user_123"),
    profile: "search",
    limit: 20,
    ..Default::default()
})?;
```

**Text DSL:**
```
SEARCH items
QUERY "piano"
WITHIN TRENDING FOR COHORT young_us_jazz
WINDOW 24h
FOR USER @user_123
USING PROFILE search
LIMIT 20
```

### 11.3 Write Path

**No explicit cohort writes.** There is no `write_cohort_membership()` or `add_user_to_cohort()` API. Membership is resolved from user attributes. The only write that affects cohort membership is `update_user()` (which changes attributes) and the background materializer (which recomputes computed fields).

Signal writes interact with cohorts through the cohort attribution mechanism (Section 7): the user's `UserCohortMemberships` struct determines which cohort counters are incremented.

### 11.4 Admin Operations

```rust
// List all named cohorts with cardinality
let cohorts = db.list_cohorts()?;

// Describe a specific cohort (predicate, type, cardinality, freshness)
let info = db.describe_cohort("young_us_jazz")?;

// Force refresh a cohort bitmap (normally happens on schedule)
db.refresh_cohort("young_us_jazz")?;

// Drop a named cohort
db.drop_cohort("young_us_jazz")?;

// Get cohort cardinality without full resolution (approximate, from cached bitmap)
let size = db.cohort_cardinality("young_us_jazz")?;
// Returns: 42_350

// Validate a predicate without defining a cohort
// (useful for UI that lets operators build cohort predicates)
let validation = db.validate_predicate(Predicate::and(vec![
    Predicate::eq("region", "US"),
    Predicate::eq("nonexistent_field", "value"),
]))?;
// Returns: Err(SchemaError::UnknownField("nonexistent_field"))
```

---

## 12. Worked Example

### "Trending Jazz Among Young US Users" -- End to End

This traces the complete lifecycle of a cohort query, from schema definition through signal writes to query execution and result delivery.

### Step 1: Define the Cohort

```rust
db.define_cohort(CohortDef {
    name: "young_us_jazz",
    predicate: Predicate::and(vec![
        Predicate::eq("region", "US"),
        Predicate::eq("age_range", "18-24"),
        Predicate::contains("inferred_interests", "jazz"),
    ]),
    exact_tracking: true,
})?;
```

The database:
1. Validates the predicate (all fields exist on User entity, types match operators).
2. Resolves the initial bitmap: `region_bitmap["US"] AND age_range_bitmap["18-24"] AND inferred_interests_bitmap["jazz"]` = 42,350 users.
3. Caches the bitmap in memory.
4. Registers `young_us_jazz` as a Level 2 behavioral segment in the signal system.
5. Updates `UserCohortMemberships` for all 42,350 matching users to include the `young_us_jazz` segment bit.

### Step 2: Signal Events Flow In

Over the next hour, users interact with content. Consider one signal event:

```rust
db.signal(Signal {
    kind: "view",
    item: "jazz_piano_video_42",
    user: "user_8847",  // a 22-year-old US user who likes jazz
    timestamp: Utc::now(),
    weight: 1.0,
    context: None,
})?;
```

The signal write path:
1. Load `user_8847`'s `UserCohortMemberships`: `{region: US, language: en, age_group: 18-24, segments: [jazz_fans, power_users, young_us_jazz]}`.
2. Check if `jazz_piano_video_42` has cohort tracking active. It does (it crossed the 100 events/hour threshold 3 hours ago).
3. Increment counters:
   - Level 0: global view counter for `jazz_piano_video_42` (**+1**)
   - Level 1: region:US counter (**+1**)
   - Level 1: language:en counter (**+1**)
   - Level 1: age_group:18-24 counter (**+1**)
   - Level 2: jazz_fans segment counter (**+1**)
   - Level 2: power_users segment counter (**+1**)
   - Level 2: young_us_jazz segment counter (**+1**) -- exact tracking

Total counter increments for this event: 7 (write amplification: 7x for this event, but only because cohort tracking is active and the user is in 3 segments).

### Step 3: Query Execution

An application serves a "trending jazz for you" surface:

```
RETRIEVE items
USING PROFILE trending
FOR COHORT young_us_jazz
WINDOW 24h
LIMIT 25
```

**Query plan:**

```
Phase 1: Candidate Identification
    Source: all items with cohort tracking active (~100K items)
    Filter: items with young_us_jazz segment velocity > 0 in 24h window
    Result: ~2,400 candidate items with non-zero cohort velocity

Phase 2: Signal Read
    For each candidate, read from the young_us_jazz Level 2 segment counters:
    - view.velocity(24h) in young_us_jazz
    - share.velocity(24h) in young_us_jazz
    - like.velocity(24h) in young_us_jazz
    - engagement_ratio in young_us_jazz (likes + comments + shares / views)
    Cost: ~2,400 items * 4 signal reads * ~200ns = ~1.9ms

Phase 3: Ranking
    Apply trending profile scoring:
    - share_velocity weight 0.5
    - view_velocity weight 0.3
    - engagement_ratio weight 0.2
    Score each candidate
    Cost: ~2,400 * 50ns = ~120us

Phase 4: Diversity and Result Assembly
    Sort by score
    Apply max_per_creator:1
    Take top 25
    Cost: < 100us

Total: < 5ms for signal reads + < 1ms for ranking + < 2ms for candidate scan
     = ~8ms total (well within 50ms budget)
```

**Result:**

```rust
Results {
    results: vec![
        RankedItem {
            id: "jazz_piano_video_42",
            score: 0.89,
            signals: SignalSnapshot {
                values: {
                    "view": {"24h": 3420, "1h": 580},
                    "share": {"24h": 245, "1h": 67},
                    "like": {"24h": 890, "1h": 156},
                },
            },
            cohort_signals: Some(CohortSignalSnapshot {
                cohort: "young_us_jazz",
                values: {
                    "view": {"24h": 1850, "1h": 312},
                    "share": {"24h": 178, "1h": 52},
                    "like": {"24h": 620, "1h": 108},
                },
            }),
        },
        // ... 24 more items
    ],
    next_cursor: Some(...),
    total_candidates: 2400,
    cohort_info: Some(CohortQueryInfo {
        name: "young_us_jazz",
        cardinality: 42_350,
        active_in_window: 8_920,
        accuracy: CohortAccuracy::Exact,
    }),
}
```

### Step 4: Search Within Cohort Trending

The user types "piano" in the search bar on the same surface:

```
SEARCH items
QUERY "piano"
WITHIN TRENDING FOR COHORT young_us_jazz
WINDOW 24h
LIMIT 20
```

**Query plan:**

```
Phase 1: Cohort Trending Candidate Generation
    Same as Phase 1-2 above but with larger pool:
    Take top 500 items trending in young_us_jazz (24h window)
    Cost: ~10ms

Phase 2: Text Retrieval Within Candidates
    BM25 search for "piano" in inverted index
    Intersect BM25 result set with 500 trending candidates
    Matching items: ~35 (items containing "piano" that are also trending in cohort)
    Cost: ~3ms (inverted index lookup + bitmap intersection)

Phase 3: Hybrid Ranking
    For each of the 35 matching items:
    - text_relevance (BM25 score) * 0.5
    - cohort_trending_velocity * 0.3
    - cohort_relative_score * 0.2 (how much more popular in this cohort vs global)
    Cost: < 1ms

Phase 4: Diversity and Result Assembly
    Sort by hybrid score, apply diversity, take top 20
    Cost: < 1ms

Total: ~15ms (well within 50ms budget)
```

---

## 13. Accuracy Analysis

### 13.1 Exact vs Estimated Cohort Aggregates

The accuracy of cohort-scoped signal aggregates depends on how the cohort maps to the dimensional hierarchy:

| Scenario | Accuracy | Error Source | Mitigation |
|----------|----------|-------------|------------|
| Global (Level 0) | Exact | None | N/A |
| Single Level 1 dimension | Exact | None | N/A |
| Single Level 2 segment | Exact | None | N/A |
| Named cohort with exact_tracking | Exact | None | N/A |
| Two Level 1 dimensions (AND) | ~85-95% | Independence assumption | Promote to Level 2 |
| Three Level 1 dimensions (AND) | ~75-90% | Independence assumption compounds | Promote to Level 2 |
| Level 1 + Level 2 (AND) | ~80-92% | Cross-level independence assumption | Promote to Level 2 |
| OR predicates | ~90-98% | Inclusion-exclusion estimation | Exact union where possible |

### 13.2 Independence Assumption Error Analysis

The composite estimation formula assumes independence between dimensions:

```
C(A AND B) ~= C(A) * C(B) / C(global)
```

When dimensions are correlated, the estimate diverges from the true count. The direction of error depends on the correlation:

**Positive correlation** (e.g., region:US and language:en): The estimate **overcounts**. More US users speak English than the independence assumption predicts, so the true intersection is larger than the estimate of the broader population but the ratio of signal events attributed is correct to within the correlation factor.

**Negative correlation** (e.g., region:JP and language:en): The estimate **undercounts**. Fewer Japanese users speak English than independence predicts.

**Empirical correlation bounds for common dimension pairs:**

| Dimension Pair | Correlation Strength | Estimated Error | Direction |
|---------------|---------------------|-----------------|-----------|
| region + language | Moderate-strong | 15-25% | Overcount for matching pairs (US+en), undercount for mismatched |
| region + age_range | Weak | 5-10% | Slight variation by region demographics |
| age_range + engagement_level | Moderate | 10-20% | Younger users skew toward power_user |
| language + age_range | Weak | 5-10% | Minimal correlation |
| region + inferred_interests | Moderate | 10-20% | Cultural preferences vary by region |
| age_range + inferred_interests | Moderate | 10-15% | Age influences interest patterns |

### 13.3 When to Promote to Exact Tracking

A named cohort should be promoted to exact tracking (`exact_tracking: true`) when:

1. **The cohort is queried frequently.** If a cohort trending query runs more than 10 times per minute, the estimation overhead and accuracy loss justify the write-time cost of exact tracking.

2. **The cohort combines correlated dimensions.** A cohort like `region:US AND language:en` has strong correlation and will have 15-25% estimation error. Exact tracking eliminates this.

3. **The cohort is used for business-critical surfaces.** The "trending for you" surface on a homepage warrants exact tracking. An internal analytics dashboard does not.

4. **The cohort is small.** Small cohorts (< 10,000 users) amplify estimation error because the independence assumption has higher relative variance with smaller populations.

**The cost of exact tracking:** One additional counter increment per signal write from a matching user to a cohort-tracked item. For a cohort of 42,350 users and a platform with 50,000 signal events/second, approximately 0.4% of events (213/second) come from this cohort. Each event adds one counter increment. This is negligible write amplification.

**Practical limit on exact-tracked cohorts:** The Signal System spec (Section 7) allows up to 100 Level 2 behavioral segments. Named cohorts with exact_tracking consume segments from this pool. With 100 total segments minus the base behavioral segments (engagement_level: 5, content_format_preference: 3, session_pattern: 3 = 11), approximately **89 slots** are available for exact-tracked named cohorts. This is sufficient for all high-value cohort definitions.

### 13.4 Error Impact on Ranking

Estimation error affects the absolute signal counts for a cohort but has a smaller effect on **relative ranking** within the cohort. If the estimation error is a roughly uniform multiplier across all items (which it is when the correlation factor is stable), then the ranking order of items by cohort velocity is preserved even with 15-25% absolute count error.

The scenario where estimation error distorts ranking is when different items have different cohort composition within the estimated population. For example, if item A is popular specifically among US English speakers and item B is popular among US Spanish speakers, and the cohort is estimated as `region:US AND language:en`, item B's signal counts will be overestimated (because the US population includes Spanish speakers, and the independence assumption does not subtract them). In practice, this distortion is small because the dimensional rollups already separate by language (Level 1), and the estimation only applies to the cross-dimension intersection.

---

## 14. Configuration and Defaults

### 14.1 Cohort System Configuration

```rust
pub struct CohortConfig {
    /// Maximum number of named cohorts.
    /// Default: 500.
    pub max_named_cohorts: usize,

    /// Maximum predicate depth (nesting levels).
    /// Default: 8.
    pub max_predicate_depth: usize,

    /// Cohort bitmap invalidation strategy.
    /// Eager: recompute bitmap on user attribute change.
    /// Lazy: mark dirty, recompute on next query.
    /// Default: Eager.
    pub invalidation: CohortInvalidation,

    /// Minimum cohort population for trending queries.
    /// Queries against cohorts smaller than this return a warning
    /// and fall back to the nearest parent cohort.
    /// Default: 1000.
    pub min_trending_population: u32,

    /// Maximum ad-hoc predicate terms per query.
    /// Limits query-time computation for inline cohort predicates.
    /// Default: 10.
    pub max_adhoc_predicate_terms: usize,

    /// Floor for cohort-relative scoring.
    /// Prevents division by near-zero global velocity.
    /// Default: 10.0 events per hour.
    pub relative_score_floor: f64,

    /// Maximum candidates for WITHIN TRENDING candidate generation.
    /// Default: 500.
    pub max_trending_candidates: usize,
}
```

### 14.2 Per-Cohort Configuration

```rust
pub struct CohortDef {
    /// Unique cohort name.
    pub name: String,

    /// Predicate over user attributes.
    pub predicate: Predicate,

    /// Whether to register as a Level 2 segment for exact signal tracking.
    /// Default: false.
    /// When true, consumes one Level 2 segment slot (max 89 available).
    pub exact_tracking: bool,
}
```

### 14.3 Default Thresholds

| Parameter | Default | Rationale |
|-----------|---------|-----------|
| Cohort activation threshold (item level) | 100 events/hour | From Signal System spec Section 7. Below this, cohort breakdown adds no useful information. |
| Minimum cohort population for trending | 1,000 active users | Statistical reliability. With < 1000 users, velocity signals are too noisy for meaningful trending. |
| Maximum named cohorts | 500 | Schema catalog practical limit. Each cohort adds one bitmap (~few KB compressed) to memory. |
| Maximum Level 2 segments (exact tracking) | 89 available (100 total minus 11 base behavioral) | Signal System spec Section 7. Write amplification scales with segment count. |
| Relative score floor | 10.0 events/hour | Prevents extreme ratios from low-traffic items. An item with 1 cohort view / 0.1 global views should not score 10x. |
| WITHIN TRENDING candidate pool | 500 | Balances search recall with query latency. 500 candidates searched in < 5ms. |
| Bitmap cache refresh (dynamic cohorts) | Matches underlying field refresh | Hourly for inferred_interests, 6-hourly for engagement_level. No separate refresh cycle. |

---

## 15. Scale Considerations

### 15.1 Resource Budget Summary

| Resource | Value | Source |
|----------|-------|--------|
| Named cohort definitions | Up to 500 | Configuration limit |
| Level 2 exact-tracked cohorts | Up to 89 | Signal System spec (100 segments minus 11 base) |
| Level 1 primary dimension values | ~56 (20 regions + 30 languages + 6 age groups) | Signal System spec Section 7 |
| Bitmap memory (10M users) | ~630 MB | Entity Model spec Section: Cohort-Ready Design |
| UserCohortMemberships cache (10M users) | ~220 MB (22 bytes per user) | Signal System spec Section 7 |
| Dimensional rollup storage (7-day retention) | ~316 GB | Signal System spec Section 7 |
| Write amplification (average) | ~1.13x | Signal System spec Section 7 |
| Items with active cohort tracking | ~100K | Signal System spec Section 7 (threshold-gated) |

### 15.2 Query Latency Budget

| Operation | Budget | Components |
|-----------|--------|------------|
| Cohort resolution (named, cached) | < 1ms | Bitmap lookup from cache |
| Cohort resolution (ad-hoc, 3 terms) | < 3ms | 3 bitmap lookups + 2 intersections |
| Cohort trending (25 results) | < 50ms | Resolution (1ms) + candidate scan (20ms) + signal reads (10ms) + ranking (5ms) + diversity (1ms) |
| Search within cohort trending (20 results) | < 50ms | Resolution (1ms) + candidate gen (15ms) + text search (10ms) + ranking (5ms) + diversity (1ms) |
| Cohort analytics (item cohort analysis) | < 200ms | Scan all Level 1 + Level 2 dimensions for one item |
| Cohort comparison (2 cohorts, 4 signals) | < 20ms | 8 signal reads per item (2 cohorts * 4 signals) |

### 15.3 Write Path Impact

The cohort system's primary write-path cost is counter attribution at signal write time. The cost depends on:

1. **Whether the target item has cohort tracking active.** 99% of items do not (below threshold). For these items, the cohort system adds zero write-path cost.

2. **How many cohort memberships the user has.** Average: 3 Level 1 dimensions + 5-10 Level 2 segments = 8-13 counter increments per event (for cohort-tracked items only).

3. **Whether any named exact-tracked cohorts match.** Each matching exact-tracked cohort adds 1 counter increment.

**Blended write amplification at 50,000 events/second:**
- 99% of events: 1x (global counter only) = 49,500 increments
- 1% of events targeting cohort-tracked items: ~14x average = 7,000 increments
- Total: 56,500 increments for 50,000 events = **1.13x** write amplification

This matches the Signal System spec's analysis and is well within the performance budget.

---

## 16. Invariants and Correctness Guarantees

### Membership Invariants

**INV-COH-1: Bitmap consistency.** A named cohort's cached bitmap is consistent with the underlying attribute indexes at the time of its last refresh. Formally: for any user U, if `bitmap.contains(U)` then `predicate.evaluate(attributes(U)) == true` as of the last refresh timestamp. The converse (predicate match implies bitmap membership) holds only for static cohorts and may lag by the refresh interval for dynamic cohorts.

**INV-COH-2: No stale membership in signal attribution.** A user's `UserCohortMemberships` is refreshed before any signal event from that user is attributed to cohort counters. A user who was in cohort C but is no longer (due to attribute change) does not contribute to C's counters after the membership update propagates.

**INV-COH-3: Monotonic cardinality.** The reported cardinality of a cohort bitmap matches the number of set bits. `db.cohort_cardinality(name)` equals `bitmap.cardinality()`.

### Signal Attribution Invariants

**INV-COH-4: Attribution completeness.** Every signal event from a user in cohort C targeting a cohort-tracked item increments C's counter exactly once. No double-counting, no missed attribution.

**INV-COH-5: Level consistency.** Exact-tracked cohort counters (Level 2) are consistent with what would be computed by filtering the global event stream by cohort membership. Formally: `counter(item, signal, cohort, window) == count({event in events(item, signal, window) : event.user in cohort})`.

**INV-COH-6: Estimation bound.** Composite cohort estimates (Level 3) satisfy: `|estimate - true_count| / true_count < max_relative_error` where `max_relative_error` is bounded by the mutual information between the constituent dimensions. The system does not guarantee a specific error bound but reports `CohortAccuracy::Estimated { confidence }` in query responses.

### Query Invariants

**INV-COH-7: Threshold enforcement.** If a cohort's active population is below `min_trending_population`, the query engine never returns results ranked solely by that cohort's signal aggregates. It must fall back to a parent cohort or return the `CohortWarning::InsufficientPopulation` warning.

**INV-COH-8: WITHIN TRENDING candidate containment.** When a `SEARCH ... WITHIN TRENDING FOR COHORT C` query executes, every result item was a member of the cohort trending candidate set. No item outside the trending set appears in results, regardless of text relevance.

### Property Tests

```rust
// P1: Bitmap matches predicate evaluation for all users.
proptest! {
    fn bitmap_matches_predicate(
        users in arb_user_set(100),
        predicate in arb_predicate(),
    ) {
        let bitmap = resolve_cohort(&users, &predicate);
        for user in &users {
            let in_bitmap = bitmap.contains(user.id);
            let matches = predicate.evaluate(&user.attributes);
            prop_assert_eq!(in_bitmap, matches,
                "user {} bitmap={} predicate={}", user.id, in_bitmap, matches);
        }
    }
}

// P2: Exact-tracked counter matches filtered event count.
proptest! {
    fn exact_counter_matches_events(
        events in arb_signal_events(1000),
        cohort in arb_cohort(),
    ) {
        let counter = cohort_counter(&events, &cohort);
        let filtered = events.iter()
            .filter(|e| cohort.contains(e.user_id))
            .count();
        prop_assert_eq!(counter, filtered as u64);
    }
}

// P3: Composite estimate is within expected error bounds.
proptest! {
    fn composite_estimate_bounded(
        events in arb_signal_events(10000),
        dim_a in arb_level1_dimension(),
        dim_b in arb_level1_dimension(),
    ) {
        let count_a = dimensional_count(&events, &dim_a);
        let count_b = dimensional_count(&events, &dim_b);
        let count_global = events.len() as f64;

        let estimate = count_a * count_b / count_global;
        let actual = events.iter()
            .filter(|e| dim_a.matches(e) && dim_b.matches(e))
            .count() as f64;

        // Allow up to 30% relative error for this test
        // (real error depends on correlation)
        if actual > 100.0 {
            let relative_error = (estimate - actual).abs() / actual;
            prop_assert!(relative_error < 0.30,
                "estimate={}, actual={}, error={:.1}%",
                estimate, actual, relative_error * 100.0);
        }
    }
}

// P4: WITHIN TRENDING results are subset of trending candidates.
proptest! {
    fn search_within_trending_containment(
        items in arb_items(500),
        cohort in arb_cohort(),
        query in arb_search_query(),
    ) {
        let trending_candidates = cohort_trending_candidates(&items, &cohort);
        let search_results = search_within_trending(&query, &cohort, &items);

        for result in &search_results {
            prop_assert!(trending_candidates.contains(&result.id),
                "result {} not in trending candidates", result.id);
        }
    }
}
```

---

## Appendix A: Glossary

| Term | Definition |
|------|------------|
| **Cohort** | A named predicate over user attributes that defines a population segment |
| **Predicate** | A boolean expression over user attribute fields (equality, range, set membership, compound) |
| **Static cohort** | A cohort whose predicate references only slow-changing app-set attributes (region, age_range) |
| **Dynamic cohort** | A cohort whose predicate references database-computed attributes (engagement_level, inferred_interests) |
| **Hybrid cohort** | A cohort combining static and dynamic predicate terms |
| **Ad-hoc cohort** | An inline predicate in a query, not named or saved |
| **Cohort resolution** | The process of evaluating a predicate against user attribute bitmaps to produce a user set |
| **Exact tracking** | Registering a cohort as a Level 2 behavioral segment with dedicated signal counters |
| **Dimensional rollup** | Pre-aggregated signal counters per dimension value per item (Level 1 and Level 2 in the signal hierarchy) |
| **Independence assumption** | The estimation that P(A AND B) = P(A) * P(B) used for composite cohort queries |
| **Cohort-relative score** | Ratio of cohort velocity to global velocity for an item, measuring cohort-specific resonance |
| **WITHIN TRENDING** | A query clause that restricts search candidates to items trending in a specified cohort |
| **Cohort activation threshold** | The global signal rate above which an item begins tracking per-cohort counters (default: 100 events/hour) |
| **Minimum population threshold** | The minimum number of active cohort users required for cohort trending queries (default: 1,000) |

## Appendix B: References

1. Signal System Specification, Section 7: Cohort-Scoped Signal Aggregation. `docs/specs/03-signal-system.md`.
2. Entity Model Specification, Section: Cohort-Ready Design. `docs/specs/02-entity-model.md`.
3. Chambi, S., Lemire, D., Kaser, O., Godin, R. "Better bitmap performance with Roaring bitmaps." Software: Practice and Experience, 2016.
4. Cormode, G., Garofalakis, M., Haas, P.J., Jermaine, C. "Synopses for Massive Data: Samples, Histograms, Wavelets, Sketches." Foundations and Trends in Databases, 2012.
