My Library

Day 3: RAG with a Knowledge Base

What You'll Build

Today you'll give your AI advisor access to a custom knowledge base. Instead of just relying on what's in its training data, it will use content from your advisors - Theo, Primeagen, and Brian.

The Limitation of Day 2

Our advisor stays in its lane now, but it only knows what was in its training data (which has a cutoff date). It can't answer questions like "What does Theo think about TypeScript?" because that's not in Gemini's training data.

We need to give it custom knowledge. That's RAG.

How RAG Works

RAG stands for Retrieval Augmented Generation. It's a simple but powerful pattern:

  1. Retrieve: Get relevant content from your knowledge base
  2. Augment: Add that content to the prompt
  3. Generate: Let the AI respond using your content

That's it. You're just stuffing extra context into the prompt. The "magic" is in what you retrieve and how you format it.

Step 1: Understand the Knowledge Base

Look at data/knowledge-base.json. This is where our advisor content lives:

[
  {
    "id": "primeagen-vim-workflow",
    "topic": "Developer Workflow",
    "advisor": "Primeagen",
    "keywords": ["workflow", "vim", "editor", "terminal"],
    "content": "Your tools should be invisible. Learn your editor deeply..."
  },
  {
    "id": "theo-typescript",
    "topic": "TypeScript Best Practices",
    "advisor": "Theo",
    "keywords": ["typescript", "types", "type safety"],
    "content": "If you're using 'any', you're doing it wrong..."
  },
  {
    "id": "brian-testing",
    "topic": "Testing Strategy",
    "advisor": "Brian Jenney",
    "keywords": ["testing", "tests", "tdd", "unit tests"],
    "content": "Test what matters. Not everything needs 100% coverage..."
  }
]

Each entry has:

  • id: Unique identifier
  • topic: What this entry is about
  • advisor: Who said it (Primeagen, Theo, Brian Jenney)
  • keywords: Terms for matching (we'll use these tomorrow)
  • content: The actual expert opinion to inject

Step 2: Import the Knowledge Base

Open app/api/chat/route.ts. Add the import at the top:

import { NextRequest, NextResponse } from 'next/server';
import { GoogleGenerativeAI } from '@google/generative-ai';
import knowledgeBase from '@/data/knowledge-base.json' with { type: 'json' };

The with { type: 'json' } syntax tells TypeScript this is a JSON import.

Step 3: Update Your System Prompt

Now update your POST function to include the knowledge base in the system prompt:

export async function POST(request: NextRequest) {
  try {
    const { message } = await request.json();

    // Get unique list of advisors
    const experts = new Set(knowledgeBase.map((item: any) => item.advisor));

    const SYSTEM_PROMPT = `
    You have access to a knowledge base of coding experts.
    The experts are: ${Array.from(experts).join(', ')}.

    If the question is not related to coding, you should say that you are not sure.
    If the user asks about an advisor not in the knowledge base, say you don't have info on them.

    Here is the knowledge base:
    ${JSON.stringify(knowledgeBase, null, 2)}

    Use the knowledge base to answer questions. Don't just repeat it - add context and make it conversational.
    `;

    const userResponse = await model.generateContent(
      `${SYSTEM_PROMPT}\n\n User message: ${message}`
    );
    const response = userResponse.response.text();

    return NextResponse.json({ response });
  } catch (error) {
    console.error('Chat API error:', error);
    return NextResponse.json(
      { error: 'Failed to generate response' },
      { status: 500 }
    );
  }
}

Key things happening here:

  • We extract the unique advisor names from the knowledge base
  • We include the ENTIRE knowledge base as JSON in the system prompt
  • We tell the AI how to use this information

Step 4: Test It!

Restart your dev server and try these prompts:

  • "What does Primeagen think about vim?"
  • "Tell me Theo's opinion on TypeScript"
  • "What's Brian's take on testing?"
  • "What do the advisors think about performance?"

The AI should now respond with content from your knowledge base!

Why This Works (And Its Limits)

We're doing the simplest possible RAG: dump everything into the prompt. This works because:

  • Our knowledge base is small (< 5KB)
  • Gemini has a large context window
  • We want the AI to have ALL the advisor info available

This approach has limits though:

  • Size: What if you have 1000 entries? You can't fit them all.
  • Cost: More tokens = more money on paid APIs.
  • Relevance: The AI has to sift through everything to find what's relevant.

Production systems use vector databases and embeddings to only retrieve the most relevant chunks. We'll talk about that on Day 5.

Key Takeaways

  • RAG = Retrieve content + Augment the prompt + Generate response
  • For small knowledge bases, just dump the whole thing into the prompt
  • Structure your data with clear fields (advisor, topic, content)
  • Tell the AI HOW to use the knowledge base in your system prompt

Challenge: Extend What You Built

RAG is the most common AI pattern in production. Push yourself:

  • Add more advisors: Pick 2-3 people whose opinions you care about. Add their content to knowledge-base.json.
  • Add more topics: What else do your advisors have opinions on? Career advice? Code review? Add entries for those.
  • Make it conversational: Tweak the system prompt so the AI speaks more like the advisor it's quoting.

This is the stuff companies actually pay for. Make it your own.

What's Next

Our AI can now access a knowledge base, but we wrote all that content ourselves. What if you want it to know what Theo actually said in his latest video? Or what Primeagen really thinks about Vim?

Tomorrow, we'll add real data from YouTube transcripts to make our advisory board even smarter.

See you on Day 4!