This article is published in English.
Practical notes: RAG Is Dead. LLM Wiki — Andrej Karpathy’s Idea — Is What Comes Next
Operable walkthrough of Practical notes: RAG Is Dead. LLM Wiki — Andrej Karpathy’s Idea — Is What Comes Next: contracts, checks, and drop-in code slots for teams shipping rag.
The following notes reconstruct a practical path around “RAG Is Dead. LLM Wiki — Andrej Karpathy’s Idea — Is What Comes Next”. Emphasis stays on contracts, checks, and drop-in code placeholders rather than motivational framing. When working through Overview, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Document the happy path and the recovery path together. Retries, human gates, and dead-letter handling are part of the product, not later polish.
What Is LLM Wiki?
What Is LLM Wiki? works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Prefer small, testable units over sprawling scripts. When a step fails, the failure should point at a single responsibility rather than a tangled pipeline. Budget tokens per turn and per session. Agentic tools expand context aggressively; hard caps keep demos from becoming surprise invoices.
The Four Implementations
The Four Implementations works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Treat this stage as a contract between inputs and validated outputs. Name the artifacts, define success checks, and refuse silent partial completion. Budget tokens per turn and per session. Agentic tools expand context aggressively; hard caps keep demos from becoming surprise invoices.
1. nashsu/llm_wiki — The Desktop App
- nashsu/llm_wiki — The Desktop App works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Record timings and token or query cost next to functional results. Cost visibility early prevents surprise bills when the path moves from demo to shared environments. Budget tokens per turn and per session. Agentic tools expand context aggressively; hard caps keep demos from becoming surprise invoices.
- nashsu/llm_wiki — The Desktop App works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Document the happy path and the recovery path together. Retries, human gates, and dead-letter handling are part of the product, not later polish.
git clone https://github.com/nashsu/llm_wiki.git
cd llm_wiki
npm install
npm run tauri dev
2. nvk/llm-wiki — The Agent Plugin
For 2. nvk/llm-wiki — The Agent Plugin, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Prefer small, testable units over sprawling scripts. When a step fails, the failure should point at a single responsibility rather than a tangled pipeline. Prefer structured outputs with schema validation over free-form prose when the next step is code or a tool call.
# For Claude Code users
claude plugin install wiki@llm-wiki
# For Codex users
codex plugin marketplace add nvk/llm-wiki# For any LLM agent (including local)
git clone https://github.com/nvk/llm-wiki.git
cp llm-wiki/AGENTS.md ./AGENTS.md
# Pass AGENTS.md as the system prompt to your local agent runner
/wiki init # Create ~/wiki/
/wiki:research "machine learning" --sources 10
@wiki query "what is attention mechanism"
@wiki audit # Find gaps
3. Pratiyush/llm-wiki — The Transcript Wiki
For 3. Pratiyush/llm-wiki — The Transcript Wiki, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Treat this stage as a contract between inputs and validated outputs. Name the artifacts, define success checks, and refuse silent partial completion. Prefer structured outputs with schema validation over free-form prose when the next step is code or a tool call.
git clone https://github.com/Pratiyush/llm-wiki.git
cd llm-wiki
./setup.sh # or setup.bat on Windows
pip install -e .
llmwiki sync # Parse your transcripts
llmwiki build # Generate static HTML
llmwiki serve # Browse at http://localhost:8080
llmwiki mcp start
4. lucasastorian/llmwiki — The MCP-Powered Wiki
For 4. lucasastorian/llmwiki — The MCP-Powered Wiki, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Record timings and token or query cost next to functional results. Cost visibility early prevents surprise bills when the path moves from demo to shared environments. Prefer structured outputs with schema validation over free-form prose when the next step is code or a tool call. For 4. lucasastorian/llmwiki — The MCP-Powered Wiki, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Document the happy path and the recovery path together. Retries, human gates, and dead-letter handling are part of the product, not later polish.
git clone https://github.com/lucasastorian/llmwiki.git
cd llmwiki
cd api && pip install -r requirements.txt && cd ..
cd web && npm install && cd ..export ANTHROPIC_API_KEY=your_key_here
./llmwiki init
./llmwiki open ~/your-documents
Which Model Should You Use?
When working through Which Model Should You Use?, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Prefer small, testable units over sprawling scripts. When a step fails, the failure should point at a single responsibility rather than a tangled pipeline. Cache stable system instructions and tool schemas. Re-sending identical preamble is a common source of burn.
Ollama vs vLLM
When working through Ollama vs vLLM, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Treat this stage as a contract between inputs and validated outputs. Name the artifacts, define success checks, and refuse silent partial completion. Log request id, model id, and latency on every call. Without that trail, intermittent provider errors look like application bugs.
Model Recommendations
When working through Model Recommendations, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Record timings and token or query cost next to functional results. Cost visibility early prevents surprise bills when the path moves from demo to shared environments. Cache stable system instructions and tool schemas. Re-sending identical preamble is a common source of burn. When working through Model Recommendations, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Document the happy path and the recovery path together. Retries, human gates, and dead-letter handling are part of the product, not later polish.
Running with Ollama
Running with Ollama works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Prefer small, testable units over sprawling scripts. When a step fails, the failure should point at a single responsibility rather than a tangled pipeline. Pin the interpreter and dependency lockfile before teaching the loop. Drift between laptop and CI is the most common silent break for API demos.
ollama pull qwen2.5:14b
# The API is now at http://localhost:11434
# OpenAI-compatible endpoint: http://localhost:11434/v1
Running with vLLM
Running with vLLM works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Treat this stage as a contract between inputs and validated outputs. Name the artifacts, define success checks, and refuse silent partial completion. Budget tokens per turn and per session. Agentic tools expand context aggressively; hard caps keep demos from becoming surprise invoices.
pip install vllm
vllm serve Qwen/Qwen2.5-14B-Instruct \
--max-model-len 131072 \
--host 0.0.0.0 \
--port 8000
vllm serve Qwen/Qwen2.5-1M \
--enable-chunked-prefill \
--max-model-len 1000000
Choosing the Right Implementation
Choosing the Right Implementation works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Record timings and token or query cost next to functional results. Cost visibility early prevents surprise bills when the path moves from demo to shared environments. Budget tokens per turn and per session. Agentic tools expand context aggressively; hard caps keep demos from becoming surprise invoices. Choosing the Right Implementation works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Document the happy path and the recovery path together. Retries, human gates, and dead-letter handling are part of the product, not later polish.
Why This Matters
For Why This Matters, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Prefer small, testable units over sprawling scripts. When a step fails, the failure should point at a single responsibility rather than a tangled pipeline. Prefer structured outputs with schema validation over free-form prose when the next step is code or a tool call.
Getting Started Checklist
For Getting Started Checklist, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Treat this stage as a contract between inputs and validated outputs. Name the artifacts, define success checks, and refuse silent partial completion. Prefer structured outputs with schema validation over free-form prose when the next step is code or a tool call.
A message from our Founder
For A message from our Founder, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Record timings and token or query cost next to functional results. Cost visibility early prevents surprise bills when the path moves from demo to shared environments. Prefer structured outputs with schema validation over free-form prose when the next step is code or a tool call. For A message from our Founder, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Document the happy path and the recovery path together. Retries, human gates, and dead-letter handling are part of the product, not later polish.
Operational checklist
For Operational checklist, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state.
Keep configuration outside application code. Environment files, secret stores, and feature flags belong in one place operators can audit without reading the whole graph.
Prefer structured outputs with schema validation over free-form prose when the next step is code or a tool call.
Measure recall on a fixed question set before tuning prompts. Prompt churn rarely fixes a weak retrieval surface.
Keep render work cheap and push expensive derivation behind memoization only after measuring. Premature memo can hide stale props bugs.
Add a smoke test that exercises the critical path in CI with fixtures, not live paid APIs, whenever budgets allow.
Before promoting the stack, freeze versions, capture a golden transcript for the critical path, and confirm rollback steps. Shared environments need rate limits, tenancy checks, and a clear owner for secret rotation. Prefer boring reliability over clever one-off demos.
Batch note for a71fa3c414a4: keep provider keys out of the repo, set a per-session token ceiling, and store transcripts next to the eval fixtures so later model swaps stay comparable.