RAG Agents in Prod: 10 Lessons We Learned — Douwe Kiela, creator of RAG

8 March 2026 · Original source →

RAG Agents in Prod: 10 Lessons We Learned —Douwe Kiela, creator of RAG

ONE SENTENCE SUMMARY

Context is king; successful RAG in production hinges on system design, specialization, data, speed, observability, and ambitious, scalable deployment success.

MAIN POINTS

  1. Enterprise ROI hinges on context-aware Rag systems, not just big language models.
  2. Specialize to capture enterprise expertise; AGI is not always necessary.
  3. Data is the company; scale AI on noisy data, not just cleaning.
  4. Production over pilots; design for production from day one.
  5. Speed over perfection; early barely functional iterations win.
  6. Make engineers focused on business value, not low-level plumbing.
  7. Ease of consumption and workflow integration drive adoption.
  8. Observability and attribution are essential for handling inaccuracies.
  9. Start with ambitious use cases; ROI comes from big impact.
  10. Data-centric mindset: data retention and governance enable scalable AI.

TAKEAWAYS

  1. Build systems, not just models; Rag pipelines around LLMs matter more than model size.
  2. Specialize on enterprise expertise to unlock real ROI.
  3. Design for production from start; pilots are easy but production is hard.
  4. Prioritize speed, adoption, and workflow integration to achieve usage.
  5. Emphasize observability and attribution to manage inaccuracies and compliance.