29400d48db
Implements the foundation of tidalDB's data pipeline: **Phase 1 – Schema primitives** - EntityId newtype (u64, big-endian ordering) - SignalTypeDefinition with pre-computed decay λ, deduped/sorted windows - SchemaBuilder with full constraint validation (duplicates, identifiers, half-life, windows, velocity) - LumenError wrapping all subsystems with required From impls **Phase 2 – Write-Ahead Log** - Length-prefixed, BLAKE3-protected entry format - Group-commit writer (batch up to 100 events / 10 ms) - Double-buffered content-hash deduplication - Checkpoint, truncation, and crash-recovery with full replay - Integration, property, and UAT tests (incl. 5,500-event deterministic UAT) - Proptest coverage scaled to 10 000 events/run (was ≤500) to meet acceptance criterion; cases reduced 100→10 to keep runtime comparable **Phase 3 – Storage engine** - StorageEngine trait (get/put/delete/scan/batch/flush) - Key encoding: [EntityId][0x00][Tag][suffix] with ordering/prefix helpers - InMemoryBackend (BTreeMap + RwLock) - FjallStorage with three isolated keyspaces and atomic batch helper - Property tests for key ordering and round-trip correctness Also adds planning docs for phases 4-5, research docs, architecture overview, and roadmap updates. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
86 lines
2.0 KiB
Rust
86 lines
2.0 KiB
Rust
/// A single operation within a write batch.
|
|
#[derive(Debug, Clone)]
|
|
pub(crate) enum BatchOp {
|
|
Put { key: Vec<u8>, value: Vec<u8> },
|
|
Delete { key: Vec<u8> },
|
|
}
|
|
|
|
/// An atomic batch of write operations.
|
|
///
|
|
/// Collects put and delete operations that are applied atomically
|
|
/// to a storage backend via `StorageEngine::write_batch`.
|
|
#[derive(Debug, Clone, Default)]
|
|
pub struct WriteBatch {
|
|
pub(crate) ops: Vec<BatchOp>,
|
|
}
|
|
|
|
impl WriteBatch {
|
|
#[must_use]
|
|
pub const fn new() -> Self {
|
|
Self { ops: Vec::new() }
|
|
}
|
|
|
|
/// Pre-allocate capacity for `n` operations.
|
|
#[must_use]
|
|
pub fn with_capacity(n: usize) -> Self {
|
|
Self {
|
|
ops: Vec::with_capacity(n),
|
|
}
|
|
}
|
|
|
|
/// Add a put operation to the batch.
|
|
pub fn put(&mut self, key: Vec<u8>, value: Vec<u8>) {
|
|
self.ops.push(BatchOp::Put { key, value });
|
|
}
|
|
|
|
/// Add a delete operation to the batch.
|
|
pub fn delete(&mut self, key: Vec<u8>) {
|
|
self.ops.push(BatchOp::Delete { key });
|
|
}
|
|
|
|
/// Number of operations in the batch.
|
|
#[must_use]
|
|
pub const fn len(&self) -> usize {
|
|
self.ops.len()
|
|
}
|
|
|
|
/// Whether the batch is empty.
|
|
#[must_use]
|
|
pub const fn is_empty(&self) -> bool {
|
|
self.ops.is_empty()
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn new_batch_is_empty() {
|
|
let batch = WriteBatch::new();
|
|
assert!(batch.is_empty());
|
|
assert_eq!(batch.len(), 0);
|
|
}
|
|
|
|
#[test]
|
|
fn put_and_delete() {
|
|
let mut batch = WriteBatch::new();
|
|
batch.put(b"key1".to_vec(), b"val1".to_vec());
|
|
batch.delete(b"key2".to_vec());
|
|
assert_eq!(batch.len(), 2);
|
|
assert!(!batch.is_empty());
|
|
}
|
|
|
|
#[test]
|
|
fn with_capacity() {
|
|
let batch = WriteBatch::with_capacity(100);
|
|
assert!(batch.is_empty());
|
|
}
|
|
|
|
#[test]
|
|
fn default_is_empty() {
|
|
let batch = WriteBatch::default();
|
|
assert!(batch.is_empty());
|
|
}
|
|
}
|