Skip to content

#026 — RAG or an AI Agent: Which to Choose for Document Search? ​

Date: November 6, 2026 | 19:00 London / 21:00 Kyiv
Format: Solo episode · live experiment · ~90 minutes
Guest: —
YouTube: https://youtube.com/live/aSMQZ5RQU3Q

Participants ​

Oleh Levchenko — Senior Full Stack JavaScript Developer (React / Node.js / AWS)
The AA · Cardiff, UK
YouTube @zloyleva · itfriday.community

What we'll cover ​

You need AI search over your own documents. Do you build a vector index, or give an agent plain grep? We test it live on real data: 25 IT Friday streams, 58 hours of transcripts, speech-recognition errors included.

Three contenders, the same questions, and scoring rules written down in advance:

  • RAG — embeddings and the top 8 chunks
  • AI agent — only Grep, Glob and Read, no index
  • Hybrid — the same agent, plus semantic search through MCP

Topics ​

  • Building RAG without a vector database: chunking, local embeddings, search
  • What a header on each chunk changes: the same search with and without it
  • How an agent works without an index, and how many tokens it uses
  • Hybrid: connecting RAG to an agent as a tool through a custom MCP server
  • Exact facts, rephrased questions, multi-stream questions and counting
  • Traps: questions with no answer, and terms that speech recognition got wrong
  • The cost of an answer for each contender
  • How to choose for your own project
  • Your questions from chat, put to the contenders live

Key takeaways ​

Added after the stream.

Timecodes ​

Added after the stream.

From the chat ​

The best questions and ideas from the live chat, translated from Ukrainian. Don't want your message here? Message us on Telegram and we'll remove it.

@kabanenko5319: Isn't document search a symbiosis of the agent approach and RAG? And depending on the domain, the share of RAG vs. agent changes?

→ Covered live: all three contenders work with the same data, only the search method differs. The hybrid is exactly that symbiosis.

@kabanenko5319: What if we used a couple of chunking algorithms, clustered the results, and picked the chunking algorithm by clustering quality?

→ Open question. Let's continue on Discord.

@kabanenko5319: How would you build a gold dataset for RAG? I came up with my own approach, curious to compare it with yours.

→ Waiting for the pipeline write-up on Discord.

@blazheiko777: Calling a provider's API for embeddings would be much cheaper on a small project.

→ Fair point. The demo uses local Ollama for simplicity and zero cost. For production, an API is a perfectly good choice.

@blazheiko777: Build a pet project that indexes all the streams.

→ Idea noted. Want to join? Write on Discord.

Resources ​

IT Friday · @zloyleva