Memory Estimates (per collection, no quantization)

Medium (1M objects):    ~6GB
Large (25M objects):    ~150GB
XLarge (500M objects):  ~3TB

💡 Quantize vectors for significant reductions (e.g. 50%-75%) to memory requirements.

Common Anti-patterns

Architecture Anti-patterns

Many collections (>50) sharing settings

• May be inefficient if the number of collections keep growing
• Consider using a multi-tenant collection instead

Cross-references with high cardinality

• Performance penalties incurred
• Consider denormalizing data

HNSW without vector compression without specific retrieval quality needs

• Higher memory footprint & resource requirements
• Evaluate compression algorithms

Data Design Anti-patterns

Vectorization of all text, including non-semantic data (IDs, URLs, etc.)

• Adds to vectorization cost for little benefit
• Consider what semantic value those properties are adding

Key Takeaways

1. Plan for scale early - Some decisions are expensive to change later
2. Denormalize when possible - Cross-references are slow at scale
3. Use compression wisely - Balance memory savings with search quality
4. Test your patterns - Use staging environments that mirror production
5. Monitor and iterate - Vector databases benefit from continuous optimization

Architectural decisions in vector databases require balancing multiple concerns: performance, cost, scalability, and operational complexity. Use this guide's decision patterns as starting points, but always validate choices against your specific requirements and constraints.

Next Steps

• Set up a development environment that mirrors your planned production architecture
• Implement monitoring to track key metrics (query performance, memory usage, costs)
• Plan regular architectural reviews as your usage patterns evolve
• Stay updated with Weaviate's evolving features and best practices