6385425a92
Some checks failed
ci/woodpecker/push/woodpecker Pipeline failed
`score_hot` hardcoded `age_hours = 24.0`, so the divisor in `log10(max(views,1)) / (age_hours + 2)^gravity` was constant across the candidate set and `Sort::Hot` reduced EXACTLY to `log10(max(views, 1))` -- a view-count ranking wearing a recency sort's name. Four built-in profiles use it (`hot`, `for_you`, `following`, `brief`); anyone tuning `gravity` was tuning a no-op. The in-code comment justified this by saying a per-entity `created_at` lookup needs an `EntityId -> created_at_ns` reverse map that "is not built". That was stale, and it was the load-bearing claim: `created_at` has been materialized INTO item metadata on every write since `Items::metadata_with_created_at`, the executor has held an `EntityId -> metadata` map since M6p3, and the replication record carries the materialized map so replicas cannot diverge. No index, storage change, schema change or migration -- the scorer reads the map it already had, exactly the way `read_duration` does three lines away. `Sort::New` used `entity_id as f64`. Wrong twice: it assumed IDs are assigned in creation order, and it used the ID's MAGNITUDE as the base score, so on a catalog of N items the sort contributed ~N against a boost sum in single digits. Recency did not participate in the ranking, it annihilated every boost. Now negated age in hours -- same ordering, boost-comparable scale. Three more instances of the same defect class, found by auditing rather than assuming the report was complete: 1. Both age sorts were missing from `needs_metadata_for_sort`, so a profile with no session and no diversity never loaded the map the fix depends on. 2. Every metadata sort was DEAD on the SEARCH path. Its metadata pre-load was gated on `session_context.is_some()` and never consulted `profile.sort`, AND the `ProfileExecutor` it built never had `with_item_metadata` called at all -- the map it did compute went only to the keyword-hint argument, which the sort scorers do not read. `shortest`/`longest` scored NEG_INFINITY and the alphabetical sorts the missing-title sentinel, for every candidate, silently. 3. Under `ReducedCandidates` load the candidate cap kept the highest entity IDs, correct only while `Sort::New` meant "highest ID". Left alone it would discard the genuinely newest items BEFORE scoring -- wrong only when degraded, the hardest case to notice. Now keyed off the `created_at` index via the new `RangeIndex::top_n_descending`. The decision "which sorts read item metadata" now lives on `Sort` itself as an exhaustive match. It was a `matches!` in one executor while a second executor had its own different copy, which is precisely how a metadata-reading sort came to be omitted from both. MEASURED, not inferred: - Real server, 10 items, equal views, ages 2-20 days: before every score was 0.5 (all-equal set folded to the normalizer's midpoint) and the feed returned oldest-first forever; after, 1.0 -> 0.0 strictly descending, newest first. - `new` with zero signals returns the exact REVERSE of candidate-scan order. - `alphabetical_asc`, `shortest`, `longest` verified end to end with title and duration order both opposing entity id. - Metadata point-read cost at 2,000 candidates (the ceiling: `scan_candidates` caps at `max(limit*10, 200)` and `limit > 500` is rejected): 7.25ms, 3.6us per candidate. Guarded at 250ms. THE BUG REPORT'S CENTRAL PROMISE IS FALSE and the changelog says so. §7 claimed this fix lets a zero-signal corpus rank newest-first so a consumer could delete its workaround. It arithmetically cannot: the numerator `log10(max(views,1))` is exactly 0.0 for 0 OR 1 views, so the age divisor has nothing to scale and every candidate still ties -- confirmed on the live server, all ten scores 0.5. Age-awareness begins at the second view. Fixing cold-start needs recency to be ADDITIVE rather than a pure divisor, which reorders every existing Hot consumer, so it is a separate decision. `sort_hot_zero_view_corpus_still_ties_regardless_of_ age` pins the limit so it cannot be rediscovered by accident. Three existing tests asserted the old entity-ID behaviour. Inverted to assert real recency, not loosened -- and each fixture now makes id order and creation order DISAGREE, because an ordering assertion where the two candidate orderings agree is satisfied by the defect too. Three of my own new tests were vacuous for exactly that reason and were caught by mutation-testing; one was also flaky (it passed in a 12-test run and failed run alone, because retrieval order for exactly-tied vectors is not deterministic). Every new assertion is mutation-proven against the implementation it replaces. Full lib suite 2130 passed. Clippy 66 warnings vs 66 at baseline, zero added.
124 lines
5.4 KiB
YAML
124 lines
5.4 KiB
YAML
# tidalDB STANDALONE instance backing thepeach's "discover" post feed.
|
|
# Apply with:
|
|
# kubectl apply -k k8s/discover/
|
|
#
|
|
# ---------------------------------------------------------------------------
|
|
# STANDALONE, NEVER CLUSTER. A cluster node's POST /signals applies
|
|
# signal/entity/weight and never reads user_id/creator_id
|
|
# (tidal-server/src/cluster/node.rs:7461-7503) while still answering 204. A
|
|
# cluster deployment would therefore accept every behavioural signal, learn
|
|
# nothing, and leave no wire evidence. The consumer asserts
|
|
# mode:"standalone" from /health at boot for exactly this reason. Do not
|
|
# rebase this overlay onto k8s/cluster/.
|
|
# ---------------------------------------------------------------------------
|
|
#
|
|
# WHY THE MANIFESTS ARE VENDORED HERE INSTEAD OF `resources: [..]`:
|
|
# kustomize refuses both ways of reaching k8s/'s manifests from a directory
|
|
# nested inside k8s/ (verified with kubectl v1.35.0 / kustomize v5.7.1):
|
|
# - `resources: [..]` -> "cycle detected: candidate root .../k8s
|
|
# contains visited root .../k8s/discover"
|
|
# - `resources: [../statefulset.yaml]` (or a symlink to it)
|
|
# -> "security; file .../k8s/statefulset.yaml
|
|
# is not in or below .../k8s/discover"
|
|
# The sibling overlays (cluster-local-kind, cluster-t4-kind) work only because
|
|
# k8s/cluster/ is not their parent. So statefulset.yaml, service.yaml and
|
|
# poddisruptionbudget.yaml here are BYTE-IDENTICAL copies of the ones in k8s/,
|
|
# and every discover-specific difference lives in this file plus
|
|
# statefulset-patch.yaml. That keeps the divergence auditable:
|
|
#
|
|
# for f in statefulset service poddisruptionbudget; do
|
|
# diff -u "k8s/$f.yaml" "k8s/discover/$f.yaml" || echo "DRIFT: $f"
|
|
# done # must print nothing; refresh with `cp k8s/$f.yaml k8s/discover/`
|
|
#
|
|
# k8s/namespace.yaml is deliberately NOT vendored: thepeach-staging is created
|
|
# and owned by thepeach's own tofu, and kustomize's namespace transformer
|
|
# rewrites a vendored Namespace's metadata.name, which would make this overlay
|
|
# claim ownership of thepeach's namespace. k8s/schema-configmap.yaml is not
|
|
# vendored either — the discover schema below replaces it.
|
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
kind: Kustomization
|
|
|
|
# Lives beside thepeach's own workloads so the api and companion-worker pods
|
|
# reach it by in-cluster DNS with no cross-namespace NetworkPolicy work. NOT
|
|
# tidalDB's own `tidaldb` namespace.
|
|
namespace: thepeach-staging
|
|
|
|
resources:
|
|
- service.yaml
|
|
- statefulset.yaml
|
|
- poddisruptionbudget.yaml
|
|
|
|
# The schema is kept as a plain, reviewable YAML file and turned into a
|
|
# ConfigMap here rather than hand-embedded in a ConfigMap literal, so it stays
|
|
# diffable against thepeach's copy (thepeach: tidaldb_config/discover-schema.yaml)
|
|
# and loadable by a local `tidal-server standalone --schema` run.
|
|
# disableNameSuffixHash: the StatefulSet's schema volume and the tofu-side
|
|
# operator runbooks name this ConfigMap explicitly; a rolling hash would break
|
|
# both. The schema is read once at boot, so roll the StatefulSet to apply edits.
|
|
configMapGenerator:
|
|
- name: tidaldb-discover-schema
|
|
files:
|
|
- schema.yaml
|
|
options:
|
|
disableNameSuffixHash: true
|
|
|
|
# The base image `tidaldb:deploy` is the LOCAL kind-loaded tag and would
|
|
# ImagePullBackOff on a real cluster. Pinned to the current published amd64
|
|
# PLATFORM-manifest digest (tag m12-poisonfix-20260831), the same image the live
|
|
# cluster set runs (k8s/cluster/statefulset.yaml:103). One image serves every
|
|
# subcommand; the StatefulSet's `standalone` args select the mode.
|
|
images:
|
|
- name: tidaldb
|
|
newName: registry.threesix.ai/tidal/server
|
|
digest: sha256:93a2929d2473c753ed2322aac547dcc5a2d8598fd00aada5104553d7b8632078
|
|
|
|
labels:
|
|
# Mirrors the label k8s/kustomization.yaml applies to the base set.
|
|
- pairs:
|
|
app.kubernetes.io/part-of: tidaldb
|
|
includeSelectors: false
|
|
# includeSelectors: the base selects purely on app.kubernetes.io/name=tidaldb.
|
|
# In a shared namespace that would cross-match any other tidalDB instance
|
|
# (e.g. a future companions instance), silently pointing this Service at the
|
|
# wrong pods. Pinning `instance: discover` into the Service, PDB and
|
|
# StatefulSet selectors plus the pod template makes the match exact.
|
|
- pairs:
|
|
app.kubernetes.io/instance: discover
|
|
includeSelectors: true
|
|
|
|
patches:
|
|
# Order matters: this patch targets the base name `tidaldb`, so it must run
|
|
# before the renames below.
|
|
- path: statefulset-patch.yaml
|
|
target:
|
|
kind: StatefulSet
|
|
name: tidaldb
|
|
# Rename the base's `tidaldb` triple to `tidaldb-discover`. Done with explicit
|
|
# patches rather than `nameSuffix: -discover` because nameSuffix also rewrites
|
|
# GENERATED resources, turning the ConfigMap above into
|
|
# `tidaldb-discover-schema-discover` (verified). The StatefulSet's
|
|
# spec.serviceName is set to match in statefulset-patch.yaml — a patch-driven
|
|
# rename is opaque to kustomize's nameReference transformer, so nothing
|
|
# rewires it for us.
|
|
- target:
|
|
kind: Service
|
|
name: tidaldb
|
|
patch: |-
|
|
- op: replace
|
|
path: /metadata/name
|
|
value: tidaldb-discover
|
|
- target:
|
|
kind: PodDisruptionBudget
|
|
name: tidaldb
|
|
patch: |-
|
|
- op: replace
|
|
path: /metadata/name
|
|
value: tidaldb-discover
|
|
- target:
|
|
kind: StatefulSet
|
|
name: tidaldb
|
|
patch: |-
|
|
- op: replace
|
|
path: /metadata/name
|
|
value: tidaldb-discover
|