# Task 01: User + Creator Entity Types and Storage

## Context

**Milestone:** 3 -- Personalized Ranking
**Phase:** m3p1 -- User and Creator Entities with Relationships
**Depends On:** m1p1 (types), m1p3 (storage), m1p5 (entity write pattern), m2p1 (vector index for embedding slots)
**Blocks:** Task 02 (Relationship Graph), Task 03 (User-State Bitmap Indexes), m3p2 (Feedback Loop)
**Complexity:** M

## Objective

Deliver `UserEntity` and `CreatorEntity` types with write/read APIs that store user and creator data in their respective fjall keyspaces. Users have a mutable preference embedding slot (1536-dim, managed by the database). Creators have a catalog embedding slot (1536-dim, aggregated from items, provided by the application). Both entity types have a metadata map stored under `Tag::Meta`.

M1p5 established the `write_item` pattern for items. This task extends `TidalDb` with `write_user`, `read_user`, `write_creator`, `read_creator` methods that follow the same pattern but route to the users and creators keyspaces via `FjallStorage::backend(EntityKind::User)` and `FjallStorage::backend(EntityKind::Creator)`.

The `StorageBox` enum in `db/mod.rs` currently only exposes `items_engine()`. This task adds `users_engine()` and `creators_engine()` methods. For the `Memory` variant, three separate `InMemoryBackend` instances are used (one per entity kind) to maintain isolation.

## Requirements

- `UserEntity` struct: `user_id: EntityId`, `metadata: HashMap<String, String>`, `embedding: Option<Vec<f32>>`
- `CreatorEntity` struct: `creator_id: EntityId`, `metadata: HashMap<String, String>`, `embedding: Option<Vec<f32>>`
- `db.write_user(user_id, metadata, embedding)` stores in users keyspace
- `db.read_user(user_id)` returns `Option<UserEntity>`
- `db.write_creator(creator_id, metadata, embedding)` stores in creators keyspace
- `db.read_creator(creator_id)` returns `Option<CreatorEntity>`
- Metadata stored under `[entity_id][0x00][META]` in the respective keyspace
- Embedding stored under `[entity_id][0x00][META][EMB:default]` suffix in the respective keyspace
- `StorageBox` extended with `users_engine()`, `creators_engine()`
- `StorageBox::Memory` variant holds three `InMemoryBackend` instances
- User and creator entities are `Send + Sync`
- Entities persist across shutdown and restart (fjall backend)
- `CreatorItemsBitmap`: maintained in a `DashMap<EntityId, RoaringBitmap>` mapping creator_id to the set of their item IDs. Updated when items are written via `write_item_with_metadata`.

## Technical Design

### Module Structure

```
tidal/src/
  entities/
    mod.rs        -- pub mod user; pub mod creator; re-exports
    user.rs       -- UserEntity, serialize/deserialize
    creator.rs    -- CreatorEntity, serialize/deserialize
  db/
    mod.rs        -- StorageBox extended, new public API methods
```

### Public API

```rust
// === entities/user.rs ===

use std::collections::HashMap;
use crate::schema::EntityId;

/// A user entity with preference embedding and metadata.
#[derive(Debug, Clone)]
pub struct UserEntity {
    pub user_id: EntityId,
    pub metadata: HashMap<String, String>,
    /// Preference embedding managed by the database.
    /// `None` for cold-start users (no engagement history).
    /// Updated atomically on signal writes in m3p2.
    pub embedding: Option<Vec<f32>>,
}

/// Serialize a UserEntity to bytes for storage.
///
/// Format:
/// ```text
/// [has_embedding: 1 byte (0 or 1)]
/// [embedding_len: 4 bytes LE (0 if no embedding)]
/// [embedding: embedding_len * 4 bytes, f32 LE]
/// [metadata_count: 4 bytes LE]
/// for each metadata entry:
///   [key_len: 4 bytes LE][key bytes]
///   [val_len: 4 bytes LE][value bytes]
/// ```
pub fn serialize_user_entity(entity: &UserEntity) -> Vec<u8>;

/// Deserialize a UserEntity from bytes.
pub fn deserialize_user_entity(user_id: EntityId, data: &[u8]) -> Option<UserEntity>;
```

```rust
// === entities/creator.rs ===

use std::collections::HashMap;
use crate::schema::EntityId;

/// A creator entity with catalog embedding and metadata.
#[derive(Debug, Clone)]
pub struct CreatorEntity {
    pub creator_id: EntityId,
    pub metadata: HashMap<String, String>,
    /// Catalog embedding aggregated from the creator's item embeddings.
    /// Provided by the application, not managed by the database.
    pub embedding: Option<Vec<f32>>,
}

pub fn serialize_creator_entity(entity: &CreatorEntity) -> Vec<u8>;
pub fn deserialize_creator_entity(creator_id: EntityId, data: &[u8]) -> Option<CreatorEntity>;
```

### StorageBox Extension

```rust
// === db/mod.rs (extended) ===

pub(crate) enum StorageBox {
    Memory {
        items: InMemoryBackend,
        users: InMemoryBackend,
        creators: InMemoryBackend,
    },
    Fjall(crate::storage::FjallStorage),
}

impl StorageBox {
    fn items_engine(&self) -> &dyn StorageEngine {
        match self {
            Self::Memory { items, .. } => items,
            Self::Fjall(f) => f.backend(EntityKind::Item),
        }
    }

    fn users_engine(&self) -> &dyn StorageEngine {
        match self {
            Self::Memory { users, .. } => users,
            Self::Fjall(f) => f.backend(EntityKind::User),
        }
    }

    fn creators_engine(&self) -> &dyn StorageEngine {
        match self {
            Self::Memory { creators, .. } => creators,
            Self::Fjall(f) => f.backend(EntityKind::Creator),
        }
    }
}
```

### TidalDb API Extensions

```rust
// === db/mod.rs (new methods on TidalDb) ===

impl TidalDb {
    /// Write (or overwrite) a user entity.
    pub fn write_user(
        &self,
        user_id: EntityId,
        metadata: &HashMap<String, String>,
        embedding: Option<&[f32]>,
    ) -> crate::Result<()>;

    /// Read a user entity by ID.
    pub fn read_user(&self, user_id: EntityId) -> crate::Result<Option<UserEntity>>;

    /// Write (or overwrite) a creator entity.
    pub fn write_creator(
        &self,
        creator_id: EntityId,
        metadata: &HashMap<String, String>,
        embedding: Option<&[f32]>,
    ) -> crate::Result<()>;

    /// Read a creator entity by ID.
    pub fn read_creator(&self, creator_id: EntityId) -> crate::Result<Option<CreatorEntity>>;
}
```

### CreatorItemsBitmap

```rust
// === entities/mod.rs ===

use dashmap::DashMap;
use roaring::RoaringBitmap;
use crate::schema::EntityId;

/// Maps creator_id -> set of item IDs belonging to that creator.
///
/// Maintained by `write_item_with_metadata` and used by `FollowsBitmap`
/// to efficiently resolve "items from followed creators".
pub struct CreatorItemsBitmap {
    inner: DashMap<EntityId, RoaringBitmap>,
}

impl CreatorItemsBitmap {
    pub fn new() -> Self;

    /// Add an item to a creator's item set.
    pub fn add_item(&self, creator_id: EntityId, item_id: EntityId);

    /// Remove an item from a creator's item set.
    pub fn remove_item(&self, creator_id: EntityId, item_id: EntityId);

    /// Get the bitmap of all items for a creator.
    pub fn items_for_creator(&self, creator_id: EntityId) -> RoaringBitmap;

    /// Get the union bitmap of items for multiple creators.
    pub fn items_for_creators(&self, creator_ids: &[EntityId]) -> RoaringBitmap;
}
```

### Embedding Storage

Embeddings are stored alongside the entity metadata in the same key-value pair. The serialization format includes the embedding bytes inline. This avoids a separate key lookup per entity read and keeps embedding + metadata co-located for cache efficiency.

For vector index integration, the embedding is also inserted into the appropriate USearch/BruteForce index via the existing `EmbeddingSlotRegistry` from m2p1. User preference vectors go into a `user_preference` slot. Creator embeddings go into a `creator_catalog` slot.

## Test Strategy

### Unit Tests

```rust
#[test]
fn user_entity_serialize_roundtrip() {
    let user = UserEntity {
        user_id: EntityId::new(42),
        metadata: HashMap::from([
            ("locale".into(), "en-US".into()),
            ("age_range".into(), "18-24".into()),
        ]),
        embedding: Some(vec![0.1; 1536]),
    };
    let bytes = serialize_user_entity(&user);
    let recovered = deserialize_user_entity(EntityId::new(42), &bytes).unwrap();
    assert_eq!(recovered.user_id, user.user_id);
    assert_eq!(recovered.metadata, user.metadata);
    assert_eq!(recovered.embedding.unwrap().len(), 1536);
}

#[test]
fn user_entity_serialize_no_embedding() {
    let user = UserEntity {
        user_id: EntityId::new(1),
        metadata: HashMap::new(),
        embedding: None,
    };
    let bytes = serialize_user_entity(&user);
    let recovered = deserialize_user_entity(EntityId::new(1), &bytes).unwrap();
    assert!(recovered.embedding.is_none());
}

#[test]
fn creator_entity_serialize_roundtrip() {
    let creator = CreatorEntity {
        creator_id: EntityId::new(7),
        metadata: HashMap::from([
            ("name".into(), "Jazz Academy".into()),
            ("verified".into(), "true".into()),
        ]),
        embedding: Some(vec![0.5; 1536]),
    };
    let bytes = serialize_creator_entity(&creator);
    let recovered = deserialize_creator_entity(EntityId::new(7), &bytes).unwrap();
    assert_eq!(recovered.creator_id, creator.creator_id);
    assert_eq!(recovered.metadata, creator.metadata);
}

#[test]
fn write_read_user_ephemeral() {
    let db = TidalDb::builder().ephemeral().with_schema(test_schema()).open().unwrap();
    let mut meta = HashMap::new();
    meta.insert("locale".into(), "en-US".into());
    db.write_user(EntityId::new(1), &meta, None).unwrap();
    let user = db.read_user(EntityId::new(1)).unwrap().unwrap();
    assert_eq!(user.metadata["locale"], "en-US");
    assert!(user.embedding.is_none());
}

#[test]
fn write_read_creator_ephemeral() {
    let db = TidalDb::builder().ephemeral().with_schema(test_schema()).open().unwrap();
    let mut meta = HashMap::new();
    meta.insert("name".into(), "Jazz Academy".into());
    let emb = vec![0.1; 1536];
    db.write_creator(EntityId::new(1), &meta, Some(&emb)).unwrap();
    let creator = db.read_creator(EntityId::new(1)).unwrap().unwrap();
    assert_eq!(creator.metadata["name"], "Jazz Academy");
    assert_eq!(creator.embedding.unwrap().len(), 1536);
}

#[test]
fn read_nonexistent_user_returns_none() {
    let db = TidalDb::builder().ephemeral().with_schema(test_schema()).open().unwrap();
    assert!(db.read_user(EntityId::new(999)).unwrap().is_none());
}

#[test]
fn write_user_overwrites_previous() {
    let db = TidalDb::builder().ephemeral().with_schema(test_schema()).open().unwrap();
    let meta1 = HashMap::from([("locale".into(), "en-US".into())]);
    let meta2 = HashMap::from([("locale".into(), "ja-JP".into())]);
    db.write_user(EntityId::new(1), &meta1, None).unwrap();
    db.write_user(EntityId::new(1), &meta2, None).unwrap();
    let user = db.read_user(EntityId::new(1)).unwrap().unwrap();
    assert_eq!(user.metadata["locale"], "ja-JP");
}

#[test]
fn creator_items_bitmap_add_and_query() {
    let bitmap = CreatorItemsBitmap::new();
    bitmap.add_item(EntityId::new(1), EntityId::new(100));
    bitmap.add_item(EntityId::new(1), EntityId::new(101));
    bitmap.add_item(EntityId::new(2), EntityId::new(200));

    let creator_1_items = bitmap.items_for_creator(EntityId::new(1));
    assert!(creator_1_items.contains(100));
    assert!(creator_1_items.contains(101));
    assert!(!creator_1_items.contains(200));

    let union = bitmap.items_for_creators(&[EntityId::new(1), EntityId::new(2)]);
    assert_eq!(union.len(), 3);
}

#[test]
fn storage_box_memory_isolates_entity_kinds() {
    // Ensure same key in items vs users does not collide
    let storage = StorageBox::memory();
    let key = encode_key(EntityId::new(1), Tag::Meta, b"");
    storage.items_engine().put(&key, b"item_data").unwrap();
    storage.users_engine().put(&key, b"user_data").unwrap();

    let item_val = storage.items_engine().get(&key).unwrap();
    let user_val = storage.users_engine().get(&key).unwrap();
    assert_eq!(item_val.as_deref(), Some(b"item_data".as_slice()));
    assert_eq!(user_val.as_deref(), Some(b"user_data".as_slice()));
}
```

### Property Tests

```rust
use proptest::prelude::*;

proptest! {
    #[test]
    fn user_serialize_roundtrip_any_metadata(
        id in 1u64..10000,
        keys in proptest::collection::vec("[a-z]{1,20}", 0..10),
        vals in proptest::collection::vec("[a-zA-Z0-9]{0,50}", 0..10),
    ) {
        let metadata: HashMap<String, String> = keys.into_iter()
            .zip(vals.into_iter())
            .collect();
        let user = UserEntity {
            user_id: EntityId::new(id),
            metadata,
            embedding: None,
        };
        let bytes = serialize_user_entity(&user);
        let recovered = deserialize_user_entity(EntityId::new(id), &bytes);
        prop_assert!(recovered.is_some());
        let recovered = recovered.unwrap();
        prop_assert_eq!(recovered.metadata, user.metadata);
    }

    #[test]
    fn creator_items_bitmap_union_is_superset(
        creator_a_items in proptest::collection::vec(1u32..1000, 0..50),
        creator_b_items in proptest::collection::vec(1u32..1000, 0..50),
    ) {
        let bitmap = CreatorItemsBitmap::new();
        for &id in &creator_a_items {
            bitmap.add_item(EntityId::new(1), EntityId::new(u64::from(id)));
        }
        for &id in &creator_b_items {
            bitmap.add_item(EntityId::new(2), EntityId::new(u64::from(id)));
        }
        let union = bitmap.items_for_creators(&[EntityId::new(1), EntityId::new(2)]);
        let a_set = bitmap.items_for_creator(EntityId::new(1));
        let b_set = bitmap.items_for_creator(EntityId::new(2));
        prop_assert!(a_set.is_subset(&union));
        prop_assert!(b_set.is_subset(&union));
    }
}
```

## Acceptance Criteria

- [ ] `UserEntity` and `CreatorEntity` structs with `Debug`, `Clone`
- [ ] `serialize_user_entity` / `deserialize_user_entity` roundtrip for all metadata sizes
- [ ] `serialize_creator_entity` / `deserialize_creator_entity` roundtrip
- [ ] `db.write_user()` stores in users keyspace via `StorageBox::users_engine()`
- [ ] `db.read_user()` retrieves from users keyspace, returns `None` for missing
- [ ] `db.write_creator()` stores in creators keyspace
- [ ] `db.read_creator()` retrieves from creators keyspace
- [ ] `StorageBox::Memory` uses three separate `InMemoryBackend` instances
- [ ] `StorageBox::Fjall` routes to correct keyspace per entity kind
- [ ] Entity kind isolation: same EntityId in items and users does not collide
- [ ] `CreatorItemsBitmap` correctly maps creators to their item sets
- [ ] `CreatorItemsBitmap::items_for_creators` returns correct union
- [ ] Embedding stored inline with metadata in serialization format
- [ ] Overwrite semantics: writing the same entity_id replaces the previous value
- [ ] `cargo clippy -- -D warnings` passes
- [ ] All unit tests and property tests pass

## Research References

- [docs/research/tidaldb_signal_ledger.md](../../../research/tidaldb_signal_ledger.md) -- Subject-prefix key encoding pattern
- [thoughts.md](../../../../thoughts.md) -- Part V.12 (subject-prefix keys), Part V.16 (user preference vector)

## Implementation Notes

- The `StorageBox::Memory` variant must change from `Memory(InMemoryBackend)` to `Memory { items, users, creators }`. This is a breaking change to the enum. Update all match arms in `db/mod.rs` and `db/builder.rs`.
- `write_item_with_metadata` (from m2p5) must be extended to update the `CreatorItemsBitmap` when a creator_id is present in the metadata. The bitmap is stored as a field on `TidalDb`.
- Embedding dimensions (1536) are validated against the schema's embedding slot configuration. If the provided embedding has wrong dimensions, return `LumenError::Schema`.
- Do NOT implement relationship edges in this task. That is Task 02.
- Do NOT implement user-state bitmap indexes in this task. That is Task 03.
- The `entities/` module is a new top-level module in `tidal/src/`. Add `pub mod entities;` to `lib.rs`.
- For persistent mode, user and creator entities survive restart because they are stored via `StorageEngine::put()` to the respective fjall keyspace. No additional checkpoint logic is needed (fjall handles persistence).
- Metadata serialization reuses the same binary format as `serialize_metadata` in `db/mod.rs`. Consider extracting the shared codec into a utility module or `entities/mod.rs`.
