A Minimal RAG Pipeline in Go With langchaingo, Groq and Jina Embeddings

RAG, short for Retrieval-Augmented Generation, means a model answers from outside documents it fetches at query time instead of depending only on what it absorbed in training. Facts no longer have to be hard-coded into a system prompt. The most relevant context is retrieved and handed to the model next to the question, so replies stay tied to sources you control.

Why an assistant needs it

Anyone building a custom assistant eventually runs into the same wall: the model knows nothing about your world. It has never seen your product names, internal policies or domain facts. The obvious workaround is pasting everything into the system prompt, which is brittle, costly and quickly hits a size ceiling.

RAG turns that around. Rather than packing context into a prompt, the model gets a tool for pulling the right context when a question comes in. You decide what reality the assistant lives in, without bloating the prompt. It also gives you a tidy pattern for organizing business-specific knowledge: your terms, your documents, your truth, retrieved on demand.

What follows is a minimal Go pipeline with very few dependencies.

The libraries involved

If LLM tooling in Go is new to you, the imports will look like alphabet soup, so here's who's who before any code.

langchaingo (github.com/tmc/langchaingo) is the Go port of the well-known Python LangChain ecosystem. It's a set of small composable packages that wrap the messy parts of LLM apps, so you needn't hand-write HTTP calls and JSON parsing for each provider. Four of its sub-packages are used here:

• llms is the model abstraction. One interface, llms.Model, and a helper, llms.GenerateFromSinglePrompt, work the same whether the backend is OpenAI, Groq, Ollama or a local model. It sends the assembled prompt and returns text.
• embeddings provides the embeddings.Embedder interface. An embedder converts a string into a list of floats, a vector that captures meaning. Similar sentences give similar vectors and unrelated ones don't, which is the math that retrieval rests on.
• schema defines schema.Document, a small struct with PageContent (the text), metadata and a Score field for ranking results.
• vectorstores contributes mostly option types and conventions. Its concrete stores aren't used, since a hand-built memStore shows how little is really required.

Two provider adapters connect these interfaces to real APIs:

• openai speaks the OpenAI-compatible HTTP protocol, whatever the name suggests. Groq serves the same protocol at https://api.groq.com/openai/v1, so pointing the OpenAI client there gives fast, free inference with no extra SDK.
• jina is the Jina embeddings adapter. Jina has a generous free tier for producing quality vectors with jina-embeddings-v3, which is why the embedder uses it.

A handy mental picture: llms plus openai is the mouth, producing answers. embeddings plus jina is the memory, turning text into searchable math. schema plus the custom memStore is the filing cabinet that stores and retrieves. The rest is wiring those three together, with no framework magic and each interface doing one job.

The supporting functions

Some helpers come before the main flow. They make retrieval work.

The in-memory vector store

Documents and their embeddings need a home, plus a way to search. A custom memStore wraps an embedder and keeps documents next to their vectors.

type memStore struct {
	embedder embeddings.Embedder
	docs     []schema.Document
	vectors  [][]float32
}

func newMemStore(embedder embeddings.Embedder) *memStore {
   return &memStore{embedder: embedder}
}

It holds three things: the embedder that turns text into vectors, the original documents, and the float32 vectors representing them. newMemStore is a plain constructor that injects the embedder.

The AddDocuments method does three jobs. It extracts the raw text of each document, embeds everything in one batch, then stores both documents and vectors while returning stable IDs.

First, extract the text and embed it:

func (m *memStore) AddDocuments(ctx context.Context, docs []schema.Document, _ ...vectorstores.Option) ([]string, error) {
	texts := make([]string, len(docs))
	for i, d := range docs {
		texts[i] = d.PageContent
	}
	vecs, err := m.embedder.EmbedDocuments(ctx, texts)
	if err != nil {
		return nil, err
	}

A single loop collects each PageContent into a plain []string, and EmbedDocuments returns a slice of vectors. An embedding failure causes an early exit.

Second, append and assign IDs:

        start := len(m.docs)
	m.docs = append(m.docs, docs...)
	m.vectors = append(m.vectors, vecs...)
	ids := make([]string, len(docs))
	for i := range ids {
		ids[i] = fmt.Sprintf("doc-%d", start+i)
	}
	return ids, nil
}

The code notes where the new documents begin, appends documents and vectors to the store, and generates sequential IDs such as doc-0 and doc-1. The IDs go back to the caller so it can refer to what was saved.

Cosine similarity

Retrieval boils down to finding the vectors nearest the query vector. Cosine similarity, a normalized dot product that disregards magnitude, does the comparison.

func cosine(a, b []float32) float64 {
	n := len(a)
	if len(b) < n {
		n = len(b)
	}
	var dot, na, nb float64
	for i := 0; i < n; i++ {
		dot += float64(a[i]) * float64(b[i])
		na += float64(a[i]) * float64(a[i])
		nb += float64(b[i]) * float64(b[i])
	}
	if na == 0 || nb == 0 {
		return 0
	}
	return dot / (math.Sqrt(na) * math.Sqrt(nb))
}

To stay in bounds the function uses the shorter of the two lengths. It accumulates the dot product and both norms, and finally divides the dot product by the multiplied square roots of the norms. Returning 0 for a zero vector sidesteps division by zero.

Similarity search

With vectors stored and a metric defined, SimilaritySearch embeds the query, scores every document, sorts them and returns the top N.

First, embed the query and score each document:

func (m *memStore) SimilaritySearch(ctx context.Context, query string, numDocuments int, _ ...vectorstores.Option) ([]schema.Document, error) {
	q, err := m.embedder.EmbedQuery(ctx, query)
	if err != nil {
		return nil, err
	}
	type scoredDoc struct {
		doc   schema.Document
		score float64
	}
	results := make([]scoredDoc, len(m.vectors))
	for i, v := range m.vectors {
		score := cosine(q, v)
		doc := m.docs[i]
		doc.Score = float32(score)
		results[i] = scoredDoc{doc: doc, score: score}
	}

The query is embedded once. The function then walks every stored vector, computes the cosine score against the query and keeps each document with its score in a scoredDoc slice.

Second, sort and cut the top results:

        sort.Slice(results, func(a, b int) bool { return results[a].score > results[b].score })
	if numDocuments > len(results) {
		numDocuments = len(results)
	}
	out := make([]schema.Document, numDocuments)
	for i := 0; i < numDocuments; i++ {
		out[i] = results[i].doc
	}
	return out, nil
}

Results are sorted by score from high to low, numDocuments is capped to what exists, and the top entries are copied into the output. The most similar documents come first.

Config helpers

Two small utilities keep the demo easy to run. envOr reads an environment variable with a fallback. loadDotEnv parses a .env file so nothing has to be exported by hand.

envOr first:

func envOr(key, def string) string {
	if v := os.Getenv(key); v != "" {
		return v
	}
	return def
}

It returns the environment value when set and the default otherwise.

Then loadDotEnv:

func loadDotEnv(path string) error {
	data, err := os.ReadFile(path)
	if err != nil {
		return err
	}
	for _, line := range strings.Split(string(data), "\n") {
		line = strings.TrimSpace(line)
		if line == "" || strings.HasPrefix(line, "#") {
			continue
		}
		kv := strings.SplitN(line, "=", 2)
		if len(kv) != 2 {
			continue
		}
		key := strings.TrimSpace(kv[0])
		value := strings.Trim(strings.TrimSpace(kv[1]), `"'`)
		if os.Getenv(key) == "" {
			os.Setenv(key, value)
		}
	}
	return nil
}

The function reads the file, splits it into lines, skips blank lines and comments, splits each line at the first =, strips quotes around the value and sets the variable only if it's undefined. That last check means exports made in the shell always take precedence. Writing this by hand also avoids pulling in a dotenv package.

The query loop

runQuery is the core of RAG: retrieve, assemble context, generate. The prompt carries a strict instruction to answer only from the supplied context, which keeps the model honest.

func runQuery(ctx context.Context, store *memStore, llm llms.Model, query string) {
     fmt.Printf("\nQuery: %s\n", query)
     results, err := store.SimilaritySearch(ctx, query, 2)
     if err != nil {
        log.Fatal(err)
     }

     contextText := ""
     for _, res := range results {
        contextText += res.PageContent + "\n"
     }

     prompt := fmt.Sprintf("Answer the question based ONLY on the following context:\n\n%s\n\nQuestion: %s", contextText, query)
     completion, err := llms.GenerateFromSinglePrompt(ctx, llm, prompt)
     if err != nil {
        log.Fatal(err)
     }
     fmt.Println("\n--- RAG Response ---")
     fmt.Println(completion)
}

It prints the query, fetches the top two documents, joins their text into one context block, builds a prompt that forbids outside knowledge, generates a completion and prints it.

The main function, and a deliberate stunt

main ties everything together, stage by stage.

Stage 1 is bootstrap and config:

func main() {
       ctx := context.Background()
       _ = loadDotEnv(".env")
       apiKey := os.Getenv("GROQ_API_KEY")

       if apiKey == "" {
          log.Fatal("Please set GROQ_API_KEY environment variable")
       }

It creates a background context, loads .env (the error is ignored so a missing file doesn't crash anything) and then insists on a Groq key.

Stage 2 builds the LLM:

        llm, err := openai.New(
	    openai.WithToken(apiKey),
	    openai.WithBaseURL("https://api.groq.com/openai/v1"),
	    openai.WithModel(envOr("GROQ_MODEL", "llama-3.3-70b-versatile")),
	)
	if err != nil {
		log.Fatal(err)
	}

Groq uses the OpenAI API shape, so the OpenAI client is pointed at Groq's base URL with a default model chosen.

Stage 3 builds the embedder:

      jinaKey := os.Getenv("JINA_API_KEY")
      if jinaKey == "" {
	  log.Fatal("Please set JINA_API_KEY environment variable (free key at https://jina.ai)")
      }

      embedder, err := jina.NewJina(
        jina.WithModel(envOr("JINA_MODEL", "jina-embeddings-v3")),
        jina.WithBatchSize(16),
      )

      if err != nil {
        log.Fatal(err)
      }

Jina supplies free embeddings. The code insists on that key, then builds the embedder using a default model and sends texts in batches of 16.

Stage 4 creates the store and the documents:

      store := newMemStore(embedder)
      docs := []string{
        "The Go compiler uses a Static Single Assignment (SSA) form for optimization.",
        "Go's garbage collector is a non-generational, concurrent tri-color mark-and-sweep collector that was made by Jesus",
        "The Go scheduler (G-M-P model) handles goroutines efficiently across OS threads.",
        "Go 1.21 introduced the 'slog' package for structured logging in the standard library.",
        "The 'unsafe' package allows bypassing Go's type safety for low-level memory access.",
     }

Look closely at the second document. It contains an absurd fact on purpose: that the Go garbage collector was made by Jesus. That's the whole point of the demo. RAG doesn't make a model smarter. It makes the model repeat whatever context it's fed, so if the retrieval layer serves nonsense, the assistant will state it with confidence.

Stage 5 indexes the documents:

        fmt.Println("Indexing documents...")
	for _, text := range docs {
		_, err := store.AddDocuments(ctx, []schema.Document{{PageContent: text}})
		if err != nil {
			log.Fatal(err)
		}
	}

Each string is wrapped in a schema.Document and passed to AddDocuments, which embeds and stores it.

Stage 6 runs the queries:

        for _, query := range []string{
		"How does the Go garbage collector work?",
		"How do I access memory directly without type safety in Go?",
		"Who made the garbage collector?",
	} {
		runQuery(ctx, store, llm, query)
	}
}

Three queries exercise the pipeline, and the third one is the one to watch. The model has no idea who invented Go's GC. But the retrieved context said Jesus, and the prompt says to answer from context only, so it complies. That's the most important lesson in RAG: an assistant is only as truthful as the documents it retrieves. Garbage in, gospel out.

Running it

The repository ships .env files so testing is easy, and go run is enough because the config loads automatically. Groq has emailed about invalidating the bundled key, so a fresh one may be needed.

This is raw output from a Linux terminal, including the theological surprise at the end:

$ go run rag_v2.go
Indexing documents...

Query: How does the Go garbage collector work?

--- RAG Response ---
The Go garbage collector is a non-generational, concurrent tri-color mark-and-sweep collector.

Query: How do I access memory directly without type safety in Go?
--- RAG Response ---
You can access memory directly without type safety in Go by using the 'unsafe' package, which allows bypassing Go's type safety for low-level memory access.

Query: Who made the garbage collector?
--- RAG Response ---
Jesus made the garbage collector.