diff --git a/.claude/guides/backend/api-endpoints.md b/.claude/guides/backend/api-endpoints.md index a162974..c379973 100644 --- a/.claude/guides/backend/api-endpoints.md +++ b/.claude/guides/backend/api-endpoints.md @@ -76,6 +76,81 @@ Rules: - Always return `Result<_, ApiError>` for consistent error responses - Use `tag = "..."` to group related endpoints in docs +### 2.1. Query Parameters with Arrays + +**CRITICAL:** If your query parameter DTO contains `Vec` or `Option>`, you MUST use `QsQuery` instead of standard `Query`. + +#### Use `QsQuery` for Array Parameters ✅ + +```rust +use crate::extractors::QsQuery; + +#[derive(Deserialize, ToSchema)] +pub struct FilterRequest { + /// List of sources to filter by + pub sources: Option>, // ← Array type + pub limit: usize, +} + +#[utoipa::path( + get, + path = "/my-endpoint", + params( + ("sources" = Option>, Query, description = "Filter sources"), + ("limit" = usize, Query, description = "Max results"), + ), + // ... +)] +pub async fn my_handler( + State(state): State, + QsQuery(params): QsQuery, // ✅ CORRECT +) -> Result> { + // Dashboard sends: ?sources[]=rfc&sources[]=community&limit=10 + // params.sources = Some(vec!["rfc", "community"]) +} +``` + +#### Never Use Standard `Query` with Arrays ❌ + +```rust +// ❌ WRONG - Will silently fail! +pub async fn broken_handler( + State(state): State, + Query(params): Query, // sources will always be None! +) -> Result> { + // Dashboard sends: ?sources[]=rfc + // Result: params.sources = None (filter doesn't work!) +} +``` + +#### Use Standard `Query` for Scalar Parameters Only ✅ + +```rust +use axum::extract::Query; + +#[derive(Deserialize, ToSchema)] +pub struct SimpleRequest { + pub limit: usize, + pub offset: usize, + pub category: Option, // All scalars, no arrays +} + +pub async fn simple_handler( + State(state): State, + Query(params): Query, // ✅ CORRECT +) -> Result> { + // ?limit=10&offset=0&category=security +} +``` + +**Why:** The StemeDB Dashboard uses JavaScript's `URLSearchParams` which generates bracket notation for arrays (`?filters[]=a&filters[]=b`). Standard `axum::extract::Query` uses `serde_urlencoded` which doesn't support brackets. `QsQuery` uses `serde_qs` which does. + +**Quick Check:** +- DTO has `Vec` or `Option` field? → Use `QsQuery` +- All fields are scalars? → Use `Query` or `AxumQuery` + +See `crates/stemedb-api/src/extractors.rs` for detailed documentation. + ### 3. Register the Route Add to the `OpenApiRouter` in `crates/stemedb-api/src/lib.rs`: @@ -118,6 +193,7 @@ Map internal errors to API errors in the handler layer. Never leak internal erro - [ ] DTO types in `dto.rs` with `Serialize, Deserialize, ToSchema` - [ ] `From<>` conversions between DTOs and internal types - [ ] Handler annotated with `#[utoipa::path]` +- [ ] **Query parameters:** Use `QsQuery` if DTO has `Vec` fields, otherwise use `Query` - [ ] Error responses documented in `responses(...)` - [ ] Route registered on `OpenApiRouter` - [ ] Tests cover happy path and error cases diff --git a/CLAUDE.md b/CLAUDE.md index 0e8a2db..5d1e5c2 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -208,6 +208,7 @@ const MAX_POOL_SIZE: u32 = 50; - **Zero-Copy:** Use `rkyv` for serialization. ALWAYS use `stemedb_core::serde::{serialize, deserialize}` — NEVER use raw `AllocSerializer` in production code. - **Instrument Critical Paths:** Use `#[instrument]` on public methods in WAL, storage, ingestion, and lens code. Include meaningful fields (key_len, payload_len, offset, candidates_count, lens). - **Structured Logging:** Use `tracing` (info!, warn!, error!) instead of `println!`/`eprintln!`. Clippy enforces via `print_stdout`/`print_stderr` at warn level. CLI binaries (e.g., `stemedb-sim`) may use `#![allow()]` for user-facing output. +- **Query Parameter Arrays:** In API handlers, use `QsQuery` extractor (not standard `Query`) for any DTO with `Vec` or `Option>` fields. Dashboard uses bracket notation (`?sources[]=a&sources[]=b`) which requires `serde_qs`. Standard `Query` silently fails on array params. See `crates/stemedb-api/src/extractors.rs` for details. - **Document Changes:** Update `ai-lookup/` when adding new types/concepts. Keep skills in sync with code. - **No Git Operations:** NEVER use git stash, git branch, git checkout, or any git operations unless the user explicitly tells you to. - **No GitHub Workflows:** We use pre-commit hooks, not GitHub Actions CI. diff --git a/crates/stemedb-api/README.md b/crates/stemedb-api/README.md index e0a2bad..ee65771 100644 --- a/crates/stemedb-api/README.md +++ b/crates/stemedb-api/README.md @@ -11,6 +11,71 @@ The API follows the standard axum pattern: - **State** (`state.rs`) - Shared application state (Journal, Store) - **Router** (`lib.rs`) - axum router with OpenAPI support via utoipa +## Query Parameter Patterns + +### When to Use QsQuery vs Query + +The API uses two different query parameter extractors depending on whether array parameters are needed: + +#### Use `QsQuery` for Array Parameters + +**Required when:** Your request DTO contains `Vec` or `Option>` fields. + +```rust +use crate::extractors::QsQuery; + +#[derive(Deserialize)] +struct MyRequest { + sources: Option>, // Array parameter + limit: usize, +} + +async fn my_handler( + State(state): State, + QsQuery(params): QsQuery, // ✅ Correct +) -> Result> { + // Dashboard sends: ?sources[]=rfc&sources[]=community&limit=10 + // params.sources = Some(vec!["rfc", "community"]) +} +``` + +**Why:** The StemeDB Dashboard uses JavaScript's `URLSearchParams` which generates bracket notation for arrays (`?filters[]=a&filters[]=b`). Standard `axum::extract::Query` uses `serde_urlencoded` which doesn't support bracket notation. `QsQuery` uses `serde_qs` which does. + +**Warning:** If you use standard `Query` with array parameters, the dashboard filters will **silently fail** (returning all results instead of filtered results). + +#### Use Standard `Query` for Scalar Parameters + +**Required when:** All query parameters are scalars (no arrays/vectors). + +```rust +use axum::extract::Query; + +#[derive(Deserialize)] +struct SimpleRequest { + limit: usize, + offset: usize, + category: Option, +} + +async fn simple_handler( + State(state): State, + Query(params): Query, // ✅ Correct +) -> Result> { + // Standard URL: ?limit=10&offset=0&category=security +} +``` + +**When to use alias:** If your handler file also imports `stemedb_query::Query`, use `use axum::extract::Query as AxumQuery` to avoid name collision. + +### Quick Reference + +| DTO Field Types | Extractor | Example | +|-----------------|-----------|---------| +| All scalars (String, usize, Option) | `Query` or `AxumQuery` | `handlers/meter.rs:60` | +| Contains Vec or Option | `QsQuery` | `handlers/aphoria/corpus.rs:41` | + +See `src/extractors.rs` for detailed documentation and examples. + ## Write Path ``` diff --git a/crates/stemedb-api/src/extractors.rs b/crates/stemedb-api/src/extractors.rs index 66e0c66..206563a 100644 --- a/crates/stemedb-api/src/extractors.rs +++ b/crates/stemedb-api/src/extractors.rs @@ -31,9 +31,9 @@ impl IntoResponse for QsQueryRejection { /// Query string extractor that supports bracket notation (e.g., `?sources[]=value1&sources[]=value2`). /// -/// This extractor uses `serde_qs` instead of `serde_urlencoded` to properly handle -/// array parameters with bracket notation, which is the standard format used by -/// JavaScript's URLSearchParams and the StemeDB Dashboard. +/// This extractor uses `serde_qs` in **non-strict mode** to properly handle +/// array parameters with bracket notation (both literal `[]` and URL-encoded `%5B%5D`), +/// which is the standard format used by JavaScript's URLSearchParams and web browsers. /// /// # When to Use QsQuery vs Query /// @@ -43,7 +43,7 @@ impl IntoResponse for QsQueryRejection { /// - You need bracket notation support: `?filters[]=a&filters[]=b` /// /// **Use standard `axum::extract::Query` when:** -/// - All query parameters are scalars (String, usize, bool, Option, etc.) +/// - All query parameters are scalars (String, usize, bool, `Option`, etc.) /// - No array/vector parameters needed /// - Simpler and lighter weight for non-array cases /// @@ -99,7 +99,11 @@ where async fn from_request_parts(parts: &mut Parts, _state: &S) -> Result { let query = parts.uri.query().unwrap_or_default(); - let value = serde_qs::from_str(query).map_err(|err| QsQueryRejection { + + // Use non-strict mode to accept both encoded (%5B%5D) and literal ([]) brackets. + // Browsers URL-encode brackets, so sources[] becomes sources%5B%5D in the query string. + let config = serde_qs::Config::new(5, false); + let value = config.deserialize_str(query).map_err(|err| QsQueryRejection { message: err.to_string(), })?; Ok(QsQuery(value)) @@ -184,4 +188,30 @@ mod tests { } ); } + + #[tokio::test] + async fn test_encoded_brackets() { + // Test URL-encoded brackets (%5B = '[', %5D = ']') + // This is what browsers send when using URLSearchParams + let uri: Uri = + "http://example.com?sources%5B%5D=rfc&sources%5B%5D=owasp&sources%5B%5D=community&limit=100" + .parse() + .unwrap(); + let mut parts = Request::builder().uri(uri).body(()).unwrap().into_parts().0; + + let QsQuery(params): QsQuery = + QsQuery::from_request_parts(&mut parts, &()).await.unwrap(); + + assert_eq!( + params, + TestParams { + sources: Some(vec![ + "rfc".to_string(), + "owasp".to_string(), + "community".to_string() + ]), + limit: Some(100), + } + ); + } }