fix(api): enable non-strict mode for URL-encoded bracket notation
## Problem Dashboard sends URL-encoded query parameters: ?sources%5B%5D=rfc&sources%5B%5D=owasp (%5B = '[', %5D = ']') But QsQuery extractor used strict mode, which rejects encoded brackets: Error: "Invalid field contains an encoded bracket" Result: All corpus filters in the dashboard failed silently. ## Solution Changed QsQuery to use serde_qs non-strict mode: Config::new(5, false) // false = non-strict Now accepts BOTH: - Literal brackets: ?sources[]=rfc - Encoded brackets: ?sources%5B%5D=rfc (browsers) ## Verification ✅ URL-encoded query: ?sources%5B%5D=rfc&sources%5B%5D=community Returns: 24 items (was: error) Logs: sources=Some(["rfc", "community"]) ✅ ✅ Literal brackets: ?sources[]=rfc (still works) ✅ All 4 extractor tests pass (added encoded brackets test) ✅ Clippy clean (0 warnings) ## Files Changed - crates/stemedb-api/src/extractors.rs: Use non-strict Config - crates/stemedb-api/README.md: Document QsQuery usage - .claude/guides/backend/api-endpoints.md: Add best practices - CLAUDE.md: Reference extractors documentation Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
jml
parent
bb0c33f8d3
commit
4012791e7e
@ -76,6 +76,81 @@ Rules:
|
||||
- Always return `Result<_, ApiError>` for consistent error responses
|
||||
- Use `tag = "..."` to group related endpoints in docs
|
||||
|
||||
### 2.1. Query Parameters with Arrays
|
||||
|
||||
**CRITICAL:** If your query parameter DTO contains `Vec<T>` or `Option<Vec<T>>`, you MUST use `QsQuery` instead of standard `Query`.
|
||||
|
||||
#### Use `QsQuery` for Array Parameters ✅
|
||||
|
||||
```rust
|
||||
use crate::extractors::QsQuery;
|
||||
|
||||
#[derive(Deserialize, ToSchema)]
|
||||
pub struct FilterRequest {
|
||||
/// List of sources to filter by
|
||||
pub sources: Option<Vec<String>>, // ← Array type
|
||||
pub limit: usize,
|
||||
}
|
||||
|
||||
#[utoipa::path(
|
||||
get,
|
||||
path = "/my-endpoint",
|
||||
params(
|
||||
("sources" = Option<Vec<String>>, Query, description = "Filter sources"),
|
||||
("limit" = usize, Query, description = "Max results"),
|
||||
),
|
||||
// ...
|
||||
)]
|
||||
pub async fn my_handler(
|
||||
State(state): State<AppState>,
|
||||
QsQuery(params): QsQuery<FilterRequest>, // ✅ CORRECT
|
||||
) -> Result<Json<MyResponse>> {
|
||||
// Dashboard sends: ?sources[]=rfc&sources[]=community&limit=10
|
||||
// params.sources = Some(vec!["rfc", "community"])
|
||||
}
|
||||
```
|
||||
|
||||
#### Never Use Standard `Query` with Arrays ❌
|
||||
|
||||
```rust
|
||||
// ❌ WRONG - Will silently fail!
|
||||
pub async fn broken_handler(
|
||||
State(state): State<AppState>,
|
||||
Query(params): Query<FilterRequest>, // sources will always be None!
|
||||
) -> Result<Json<MyResponse>> {
|
||||
// Dashboard sends: ?sources[]=rfc
|
||||
// Result: params.sources = None (filter doesn't work!)
|
||||
}
|
||||
```
|
||||
|
||||
#### Use Standard `Query` for Scalar Parameters Only ✅
|
||||
|
||||
```rust
|
||||
use axum::extract::Query;
|
||||
|
||||
#[derive(Deserialize, ToSchema)]
|
||||
pub struct SimpleRequest {
|
||||
pub limit: usize,
|
||||
pub offset: usize,
|
||||
pub category: Option<String>, // All scalars, no arrays
|
||||
}
|
||||
|
||||
pub async fn simple_handler(
|
||||
State(state): State<AppState>,
|
||||
Query(params): Query<SimpleRequest>, // ✅ CORRECT
|
||||
) -> Result<Json<MyResponse>> {
|
||||
// ?limit=10&offset=0&category=security
|
||||
}
|
||||
```
|
||||
|
||||
**Why:** The StemeDB Dashboard uses JavaScript's `URLSearchParams` which generates bracket notation for arrays (`?filters[]=a&filters[]=b`). Standard `axum::extract::Query` uses `serde_urlencoded` which doesn't support brackets. `QsQuery` uses `serde_qs` which does.
|
||||
|
||||
**Quick Check:**
|
||||
- DTO has `Vec` or `Option<Vec>` field? → Use `QsQuery`
|
||||
- All fields are scalars? → Use `Query` or `AxumQuery`
|
||||
|
||||
See `crates/stemedb-api/src/extractors.rs` for detailed documentation.
|
||||
|
||||
### 3. Register the Route
|
||||
|
||||
Add to the `OpenApiRouter` in `crates/stemedb-api/src/lib.rs`:
|
||||
@ -118,6 +193,7 @@ Map internal errors to API errors in the handler layer. Never leak internal erro
|
||||
- [ ] DTO types in `dto.rs` with `Serialize, Deserialize, ToSchema`
|
||||
- [ ] `From<>` conversions between DTOs and internal types
|
||||
- [ ] Handler annotated with `#[utoipa::path]`
|
||||
- [ ] **Query parameters:** Use `QsQuery` if DTO has `Vec` fields, otherwise use `Query`
|
||||
- [ ] Error responses documented in `responses(...)`
|
||||
- [ ] Route registered on `OpenApiRouter`
|
||||
- [ ] Tests cover happy path and error cases
|
||||
|
||||
@ -208,6 +208,7 @@ const MAX_POOL_SIZE: u32 = 50;
|
||||
- **Zero-Copy:** Use `rkyv` for serialization. ALWAYS use `stemedb_core::serde::{serialize, deserialize}` — NEVER use raw `AllocSerializer` in production code.
|
||||
- **Instrument Critical Paths:** Use `#[instrument]` on public methods in WAL, storage, ingestion, and lens code. Include meaningful fields (key_len, payload_len, offset, candidates_count, lens).
|
||||
- **Structured Logging:** Use `tracing` (info!, warn!, error!) instead of `println!`/`eprintln!`. Clippy enforces via `print_stdout`/`print_stderr` at warn level. CLI binaries (e.g., `stemedb-sim`) may use `#![allow()]` for user-facing output.
|
||||
- **Query Parameter Arrays:** In API handlers, use `QsQuery` extractor (not standard `Query`) for any DTO with `Vec<T>` or `Option<Vec<T>>` fields. Dashboard uses bracket notation (`?sources[]=a&sources[]=b`) which requires `serde_qs`. Standard `Query` silently fails on array params. See `crates/stemedb-api/src/extractors.rs` for details.
|
||||
- **Document Changes:** Update `ai-lookup/` when adding new types/concepts. Keep skills in sync with code.
|
||||
- **No Git Operations:** NEVER use git stash, git branch, git checkout, or any git operations unless the user explicitly tells you to.
|
||||
- **No GitHub Workflows:** We use pre-commit hooks, not GitHub Actions CI.
|
||||
|
||||
@ -11,6 +11,71 @@ The API follows the standard axum pattern:
|
||||
- **State** (`state.rs`) - Shared application state (Journal, Store)
|
||||
- **Router** (`lib.rs`) - axum router with OpenAPI support via utoipa
|
||||
|
||||
## Query Parameter Patterns
|
||||
|
||||
### When to Use QsQuery vs Query
|
||||
|
||||
The API uses two different query parameter extractors depending on whether array parameters are needed:
|
||||
|
||||
#### Use `QsQuery` for Array Parameters
|
||||
|
||||
**Required when:** Your request DTO contains `Vec<T>` or `Option<Vec<T>>` fields.
|
||||
|
||||
```rust
|
||||
use crate::extractors::QsQuery;
|
||||
|
||||
#[derive(Deserialize)]
|
||||
struct MyRequest {
|
||||
sources: Option<Vec<String>>, // Array parameter
|
||||
limit: usize,
|
||||
}
|
||||
|
||||
async fn my_handler(
|
||||
State(state): State<AppState>,
|
||||
QsQuery(params): QsQuery<MyRequest>, // ✅ Correct
|
||||
) -> Result<Json<MyResponse>> {
|
||||
// Dashboard sends: ?sources[]=rfc&sources[]=community&limit=10
|
||||
// params.sources = Some(vec!["rfc", "community"])
|
||||
}
|
||||
```
|
||||
|
||||
**Why:** The StemeDB Dashboard uses JavaScript's `URLSearchParams` which generates bracket notation for arrays (`?filters[]=a&filters[]=b`). Standard `axum::extract::Query` uses `serde_urlencoded` which doesn't support bracket notation. `QsQuery` uses `serde_qs` which does.
|
||||
|
||||
**Warning:** If you use standard `Query` with array parameters, the dashboard filters will **silently fail** (returning all results instead of filtered results).
|
||||
|
||||
#### Use Standard `Query` for Scalar Parameters
|
||||
|
||||
**Required when:** All query parameters are scalars (no arrays/vectors).
|
||||
|
||||
```rust
|
||||
use axum::extract::Query;
|
||||
|
||||
#[derive(Deserialize)]
|
||||
struct SimpleRequest {
|
||||
limit: usize,
|
||||
offset: usize,
|
||||
category: Option<String>,
|
||||
}
|
||||
|
||||
async fn simple_handler(
|
||||
State(state): State<AppState>,
|
||||
Query(params): Query<SimpleRequest>, // ✅ Correct
|
||||
) -> Result<Json<MyResponse>> {
|
||||
// Standard URL: ?limit=10&offset=0&category=security
|
||||
}
|
||||
```
|
||||
|
||||
**When to use alias:** If your handler file also imports `stemedb_query::Query`, use `use axum::extract::Query as AxumQuery` to avoid name collision.
|
||||
|
||||
### Quick Reference
|
||||
|
||||
| DTO Field Types | Extractor | Example |
|
||||
|-----------------|-----------|---------|
|
||||
| All scalars (String, usize, Option<bool>) | `Query` or `AxumQuery` | `handlers/meter.rs:60` |
|
||||
| Contains Vec or Option<Vec> | `QsQuery` | `handlers/aphoria/corpus.rs:41` |
|
||||
|
||||
See `src/extractors.rs` for detailed documentation and examples.
|
||||
|
||||
## Write Path
|
||||
|
||||
```
|
||||
|
||||
@ -31,9 +31,9 @@ impl IntoResponse for QsQueryRejection {
|
||||
|
||||
/// Query string extractor that supports bracket notation (e.g., `?sources[]=value1&sources[]=value2`).
|
||||
///
|
||||
/// This extractor uses `serde_qs` instead of `serde_urlencoded` to properly handle
|
||||
/// array parameters with bracket notation, which is the standard format used by
|
||||
/// JavaScript's URLSearchParams and the StemeDB Dashboard.
|
||||
/// This extractor uses `serde_qs` in **non-strict mode** to properly handle
|
||||
/// array parameters with bracket notation (both literal `[]` and URL-encoded `%5B%5D`),
|
||||
/// which is the standard format used by JavaScript's URLSearchParams and web browsers.
|
||||
///
|
||||
/// # When to Use QsQuery vs Query
|
||||
///
|
||||
@ -43,7 +43,7 @@ impl IntoResponse for QsQueryRejection {
|
||||
/// - You need bracket notation support: `?filters[]=a&filters[]=b`
|
||||
///
|
||||
/// **Use standard `axum::extract::Query` when:**
|
||||
/// - All query parameters are scalars (String, usize, bool, Option<String>, etc.)
|
||||
/// - All query parameters are scalars (String, usize, bool, `Option<String>`, etc.)
|
||||
/// - No array/vector parameters needed
|
||||
/// - Simpler and lighter weight for non-array cases
|
||||
///
|
||||
@ -99,7 +99,11 @@ where
|
||||
|
||||
async fn from_request_parts(parts: &mut Parts, _state: &S) -> Result<Self, Self::Rejection> {
|
||||
let query = parts.uri.query().unwrap_or_default();
|
||||
let value = serde_qs::from_str(query).map_err(|err| QsQueryRejection {
|
||||
|
||||
// Use non-strict mode to accept both encoded (%5B%5D) and literal ([]) brackets.
|
||||
// Browsers URL-encode brackets, so sources[] becomes sources%5B%5D in the query string.
|
||||
let config = serde_qs::Config::new(5, false);
|
||||
let value = config.deserialize_str(query).map_err(|err| QsQueryRejection {
|
||||
message: err.to_string(),
|
||||
})?;
|
||||
Ok(QsQuery(value))
|
||||
@ -184,4 +188,30 @@ mod tests {
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_encoded_brackets() {
|
||||
// Test URL-encoded brackets (%5B = '[', %5D = ']')
|
||||
// This is what browsers send when using URLSearchParams
|
||||
let uri: Uri =
|
||||
"http://example.com?sources%5B%5D=rfc&sources%5B%5D=owasp&sources%5B%5D=community&limit=100"
|
||||
.parse()
|
||||
.unwrap();
|
||||
let mut parts = Request::builder().uri(uri).body(()).unwrap().into_parts().0;
|
||||
|
||||
let QsQuery(params): QsQuery<TestParams> =
|
||||
QsQuery::from_request_parts(&mut parts, &()).await.unwrap();
|
||||
|
||||
assert_eq!(
|
||||
params,
|
||||
TestParams {
|
||||
sources: Some(vec![
|
||||
"rfc".to_string(),
|
||||
"owasp".to_string(),
|
||||
"community".to_string()
|
||||
]),
|
||||
limit: Some(100),
|
||||
}
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
Loading…
Reference in New Issue
Block a user