Add multi-vector preference entity (per-signal-type preference vectors with event-time decay) feeding ANN candidate generation in the query executor. - entities: multi_preference vectors + event-time-aware preference updates - query/executor: ANN candidate-gen + personalization/pipeline integration - storage/keys, db ops, state_rebuild: persist & rebuild multi-vector prefs - ranking: profile + builtins support for multi-vector scoring - tidal-server/config: expose multi-preference knobs - tests/bench: m12_preference_event_time integration + multi_preference bench - docs: multi-vector-preference research, ROADMAP/ARCHITECTURE refresh, legal/tidaldb-patent-proposal - .codex/agents: codex agent definitions - chore: gitignore tool-regenerated .agents/ mirror (doc-guard rejects it)
288 lines
12 KiB
Rust
288 lines
12 KiB
Rust
//! Ranking profile type system.
|
|
//!
|
|
//! A `RankingProfile` is the unit of ranking configuration. It declares how
|
|
//! candidates are sourced, scored, gated, penalized, and diversified. Profiles
|
|
//! are versioned and registered in the `ProfileRegistry`.
|
|
//!
|
|
//! Every type here is `Serialize + Deserialize` so profiles can be stored in
|
|
//! the schema layer and exchanged over the API.
|
|
|
|
use serde::{Deserialize, Serialize};
|
|
|
|
use crate::schema::Window;
|
|
|
|
// ── Core profile ────────────────────────────────────────────────────────────
|
|
|
|
/// A complete ranking profile -- the unit of ranking configuration.
|
|
///
|
|
/// Profiles combine:
|
|
/// - **Candidate strategy**: how to source candidates (ANN, scan, signal-ranked)
|
|
/// - **Boosts**: signal-based score multipliers
|
|
/// - **Decay**: time-decay weighting
|
|
/// - **Gates**: minimum thresholds that filter candidates
|
|
/// - **Penalties**: negative score adjustments
|
|
/// - **Excludes**: hard filters that remove candidates
|
|
/// - **Diversity**: per-creator and format-mix constraints
|
|
/// - **Exploration**: fraction of random candidates injected for discovery
|
|
/// - **Sort**: the primary scoring formula
|
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
|
pub struct RankingProfile {
|
|
pub name: String,
|
|
pub version: u32,
|
|
pub candidate_strategy: CandidateStrategy,
|
|
pub boosts: Vec<Boost>,
|
|
pub decay: Option<ProfileDecay>,
|
|
pub gates: Vec<Gate>,
|
|
pub penalties: Vec<Penalty>,
|
|
pub excludes: Vec<Exclude>,
|
|
pub diversity: DiversitySpec,
|
|
pub exploration: f64,
|
|
pub sort: Option<Sort>,
|
|
pub is_builtin: bool,
|
|
}
|
|
|
|
// ── Sort modes ──────────────────────────────────────────────────────────────
|
|
|
|
/// Primary sort mode. Determines the scoring formula applied to candidates.
|
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
|
pub enum Sort {
|
|
Hot {
|
|
gravity: f64,
|
|
},
|
|
Trending,
|
|
Rising,
|
|
Controversial,
|
|
HiddenGems,
|
|
Shuffle,
|
|
New,
|
|
TopWindow {
|
|
window: Window,
|
|
},
|
|
MostViewed {
|
|
window: Window,
|
|
},
|
|
MostLiked {
|
|
window: Window,
|
|
},
|
|
/// Sort creators by total follow count (`AllTime` follow signal value).
|
|
/// Uses the "follow" signal's `AllTime` windowed count as a proxy for
|
|
/// follower count. Degrades gracefully when "follow" signal is absent.
|
|
MostFollowed,
|
|
/// Sort creators by engagement rate proxy (view + like velocity over 24h).
|
|
/// Higher combined velocity = higher engagement rate.
|
|
CreatorEngagementRate,
|
|
/// Sort alphabetically by item "title" metadata field (A-Z, case-insensitive).
|
|
/// Items without a title are sorted last.
|
|
AlphabeticalAsc,
|
|
/// Sort reverse-alphabetically by item "title" metadata field (Z-A, case-insensitive).
|
|
/// Items without a title are sorted last.
|
|
AlphabeticalDesc,
|
|
/// Sort by item "duration" metadata field in seconds, shortest first.
|
|
/// Items without a duration are sorted last.
|
|
Shortest,
|
|
/// Sort by item "duration" metadata field in seconds, longest first.
|
|
/// Items without a duration are sorted last.
|
|
Longest,
|
|
/// Sort by windowed count of "comment" signal.
|
|
MostCommented {
|
|
window: Window,
|
|
},
|
|
/// Sort by windowed count of "share" signal.
|
|
MostShared {
|
|
window: Window,
|
|
},
|
|
/// Sort by current decayed score of `viewer_count` signal.
|
|
/// Items without the signal score 0.0.
|
|
LiveViewerCount,
|
|
/// Sort by timestamp when the querying user saved the item.
|
|
/// Requires `for_user` context; returns `QueryError` if absent.
|
|
DateSaved,
|
|
}
|
|
|
|
// ── Candidate strategy ──────────────────────────────────────────────────────
|
|
|
|
/// How candidates are sourced for ranking.
|
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
|
pub enum CandidateStrategy {
|
|
Ann {
|
|
slot: String,
|
|
limit: usize,
|
|
/// Multi-vector (PinnerSage-style) query fan-out width: the number of the
|
|
/// user's top interest clusters to issue ANN queries against, merging by
|
|
/// best distance (`docs/research/multi-vector-preference.md` §4). The
|
|
/// queries run **sequentially** on the query thread today (the loop is
|
|
/// embarrassingly parallel and reserved for parallelization).
|
|
/// `min(K_active, top_clusters)` clusters are queried; 1 reduces to the
|
|
/// single-vector path. Defaults to
|
|
/// [`DEFAULT_TOP_M`](crate::entities::multi_preference::DEFAULT_TOP_M) (3)
|
|
/// via serde so profiles serialized before multi-vector deserialize cleanly.
|
|
#[serde(default = "default_top_clusters")]
|
|
top_clusters: usize,
|
|
},
|
|
Scan {
|
|
sort_field: String,
|
|
},
|
|
SignalRanked {
|
|
signal: String,
|
|
window: Window,
|
|
},
|
|
Hybrid,
|
|
Relationship,
|
|
CohortTrending,
|
|
}
|
|
|
|
/// Serde default for `top_clusters` on [`CandidateStrategy::Ann`]: the
|
|
/// `PinnerSage` serve-time count (3). A free function because
|
|
/// `serde(default = "...")` requires a callable path.
|
|
const fn default_top_clusters() -> usize {
|
|
crate::entities::multi_preference::DEFAULT_TOP_M
|
|
}
|
|
|
|
// ── Signal aggregation ──────────────────────────────────────────────────────
|
|
|
|
/// Which aggregation to read from a signal for scoring/gating.
|
|
///
|
|
/// - `Value`: raw windowed count
|
|
/// - `Velocity`: rate of change within a window
|
|
/// - `DecayScore`: exponentially decayed running score in `[0.0, ~1.0]`
|
|
/// - `Ratio`: ratio of two signals (planned for M3; currently returns 0.0 with a warning)
|
|
/// - `RelativeVelocity`: velocity relative to a baseline (planned for M3; currently returns 0.0 with a warning)
|
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
|
pub enum SignalAgg {
|
|
Value,
|
|
Velocity,
|
|
Ratio,
|
|
DecayScore,
|
|
RelativeVelocity,
|
|
}
|
|
|
|
impl SignalAgg {
|
|
/// Whether this aggregation has a working executor implementation.
|
|
///
|
|
/// `Ratio` and `RelativeVelocity` are declared in the type system but not
|
|
/// yet computed by the executor (planned for M3 cross-signal reads). A
|
|
/// profile that references them registers cleanly but would silently fail
|
|
/// every gate / contribute 0.0 to every boost at query time, so the
|
|
/// registry rejects them up front via [`ProfileError::UnsupportedAggregation`].
|
|
///
|
|
/// [`ProfileError::UnsupportedAggregation`]: crate::ranking::registry::ProfileError::UnsupportedAggregation
|
|
#[must_use]
|
|
pub const fn is_implemented(&self) -> bool {
|
|
!matches!(self, Self::Ratio | Self::RelativeVelocity)
|
|
}
|
|
|
|
/// Stable, lowercase label for diagnostics and error messages.
|
|
#[must_use]
|
|
pub const fn label(&self) -> &'static str {
|
|
match self {
|
|
Self::Value => "Value",
|
|
Self::Velocity => "Velocity",
|
|
Self::Ratio => "Ratio",
|
|
Self::DecayScore => "DecayScore",
|
|
Self::RelativeVelocity => "RelativeVelocity",
|
|
}
|
|
}
|
|
}
|
|
|
|
// ── Boost ───────────────────────────────────────────────────────────────────
|
|
|
|
/// A signal-based score multiplier. Adds `weight * agg(signal, window)` to the
|
|
/// candidate's score.
|
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
|
pub struct Boost {
|
|
pub signal: String,
|
|
pub agg: SignalAgg,
|
|
pub window: Window,
|
|
pub weight: f64,
|
|
}
|
|
|
|
// ── Decay ───────────────────────────────────────────────────────────────────
|
|
|
|
/// Profile-level time-decay. Applies an exponential decay factor to the score
|
|
/// based on the named signal's age.
|
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
|
pub struct ProfileDecay {
|
|
pub signal: String,
|
|
pub half_life_secs: u64,
|
|
pub weight: f64,
|
|
}
|
|
|
|
// ── Gate ────────────────────────────────────────────────────────────────────
|
|
|
|
/// A minimum-threshold filter. Candidates with `agg(signal, window) < min_threshold`
|
|
/// are excluded from the result set.
|
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
|
pub struct Gate {
|
|
pub signal: String,
|
|
pub agg: SignalAgg,
|
|
pub window: Window,
|
|
pub min_threshold: f64,
|
|
}
|
|
|
|
// ── Penalty ─────────────────────────────────────────────────────────────────
|
|
|
|
/// A negative score adjustment. Subtracts `weight * agg(signal, window)` from
|
|
/// the candidate's score.
|
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
|
pub struct Penalty {
|
|
pub signal: String,
|
|
pub agg: SignalAgg,
|
|
pub window: Window,
|
|
pub weight: f64,
|
|
}
|
|
|
|
// ── Exclude ─────────────────────────────────────────────────────────────────
|
|
|
|
/// A hard filter. Candidates with `agg(signal, window) > above` are removed
|
|
/// entirely.
|
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
|
pub struct Exclude {
|
|
pub signal: String,
|
|
pub agg: SignalAgg,
|
|
pub window: Window,
|
|
pub above: f64,
|
|
}
|
|
|
|
// ── Diversity ───────────────────────────────────────────────────────────────
|
|
|
|
/// Diversity constraints applied after scoring and before final result assembly.
|
|
///
|
|
/// Both fields are optional. When `None`, the corresponding constraint is not enforced.
|
|
/// M2 enforces `max_per_creator` and `format_mix_max_fraction` via the `DiversitySelector`.
|
|
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
|
|
pub struct DiversitySpec {
|
|
/// Maximum number of items from any single creator in the result set.
|
|
/// `None` means no per-creator limit.
|
|
pub max_per_creator: Option<usize>,
|
|
/// Maximum fraction of the result set that any single content format may occupy.
|
|
/// For example, `Some(0.5)` means no single format can exceed 50% of results.
|
|
/// `None` means no format-mix constraint.
|
|
pub format_mix_max_fraction: Option<f64>,
|
|
}
|
|
|
|
// ── Tests ───────────────────────────────────────────────────────────────────
|
|
|
|
#[cfg(test)]
|
|
#[allow(clippy::unwrap_used)]
|
|
mod tests {
|
|
use super::*;
|
|
// Shared profile skeleton (DRY-S, M0-M10 review): the per-module
|
|
// `minimal_profile` 14-field literal now lives in `ranking::test_fixtures`.
|
|
use crate::ranking::test_fixtures::minimal_profile;
|
|
|
|
#[test]
|
|
fn profile_serializes_to_json() {
|
|
let profile = minimal_profile("test_profile");
|
|
let json = serde_json::to_string(&profile).unwrap();
|
|
let deserialized: RankingProfile = serde_json::from_str(&json).unwrap();
|
|
assert_eq!(deserialized.name, "test_profile");
|
|
}
|
|
|
|
#[test]
|
|
fn diversity_spec_default_is_none() {
|
|
let spec = DiversitySpec::default();
|
|
assert!(spec.max_per_creator.is_none());
|
|
assert!(spec.format_mix_max_fraction.is_none());
|
|
}
|
|
}
|