stemedb/applications/aphoria/dogfood/msgqueue/docs/sources/rabbitmq-docs.md

RabbitMQ Best Practices - Key Excerpts for Message Queue Consumers

Authority Tier: Tier 2 (Vendor) Source: https://www.rabbitmq.com/best-practices.html Relevance: RabbitMQ's official best practices provide vendor-specific guidance on consumer configuration, performance tuning, and common pitfalls. This is authoritative for production deployments.

---

Consumer Prefetch Configuration

Set a prefetch count that balances throughput and memory usage. A good starting point is 10-100 messages depending on message size and processing time. Setting it too high can lead to memory issues, too low reduces throughput.

Key Claim:

• msgqueue/consumer/prefetch_count :: recommended_range = 10..100
• Consequence: prefetch_count < 10 reduces throughput; prefetch_count > 100 risks OOM

---

Consumer Timeout Configuration

Use heartbeats to detect dead TCP connections. The recommended heartbeat interval is 30-60 seconds. Too short (< 10s) can cause false positives under network congestion, too long (> 300s) delays failure detection.

Key Claim:

• msgqueue/connection/heartbeat_interval :: recommended_value = 30
• Consequence: No heartbeat fails to detect connection loss; too short causes false positives

---

Connection Pooling

Create a small number of connections (1-10) and multiplex channels over them. Each connection is a TCP connection with handshake overhead. Too many connections exhaust broker file descriptors.

Key Claim:

• msgqueue/connection/max_connections :: recommended_range = 1..10
• Consequence: Unbounded connections exhaust broker resources (file descriptors, memory)

---

Manual Acknowledgment

Use manual acknowledgments (basic.ack) to ensure messages are not lost. Only acknowledge a message after it has been successfully processed. Acknowledging before processing risks data loss on consumer crashes.

Key Claim:

• msgqueue/consumer/ack_mode :: recommended_value = "manual"
• Consequence: Auto-ack before processing causes data loss on crashes

---

Dead Letter Queue

Configure a dead letter exchange (DLX) for messages that fail processing multiple times. Without a DLX, poison messages can block queue processing indefinitely.

Key Claim:

• msgqueue/consumer/dead_letter_queue :: required = true
• Consequence: No DLX means poison messages block queue forever

---

Backpressure Handling

Implement backpressure by limiting the in-memory queue size and pausing consumption when the queue is full. Without backpressure, fast producers can overwhelm slow consumers, leading to OOM.

Key Claim:

• msgqueue/queue/max_size :: recommended_range = 100..10000
• Consequence: Unbounded queue causes OOM under sustained load

---

TLS Configuration

Always enable TLS for production deployments. Disable certificate verification only for local development. In production, missing TLS validation allows MITM attacks.

Key Claim:

• msgqueue/tls/certificate_validation :: required = true
• Consequence: Disabled TLS validation allows attackers to intercept message traffic

---

Extraction Guide

1. Navigate to Best Practices:

   # Fetch RabbitMQ docs
   curl https://www.rabbitmq.com/best-practices.html > rabbitmq-best-practices.html
   

2. Search for:

   • Consumer configuration (prefetch, ack mode)
   • Connection management (pooling, heartbeat)
   • Error handling (DLX, requeue limits)
   • Security (TLS, authentication)
   • Performance tuning (backpressure, batch size)

3. Extract Official Recommendations:

   • Note recommended ranges (e.g., "10-100 messages")
   • Document performance trade-offs
   • Map vendor warnings to consequences

4. Map to Concept Paths:

   Prefetch → msgqueue/consumer/prefetch_count
   Heartbeat → msgqueue/connection/heartbeat_interval
   Pooling → msgqueue/connection/max_connections
   

5. Note Consequences from Warnings:

   • What does RabbitMQ say will go wrong?
   • What performance issues arise?
   • What security risks exist?