This article is published in English.
Letting Gemini Pick the Source: FAISS, Tavily and Direct Answers in LangGraph
Build a small LangGraph workflow in which Gemini routes each question to a FAISS knowledge base, Tavily web search or a direct answer, with structured-output routing.
Most question-answering prototypes push every query through one fixed pipeline, even though questions differ in what they need: some depend on private company documents, some on facts that change daily, and some on nothing more than general knowledge. This walkthrough builds a compact Python workflow in which a language model looks at each question first and sends it to the right place: a FAISS vector store for internal knowledge, Tavily for live web results, or straight to Gemini. By the end you will understand how state, nodes and conditional edges fit together in LangGraph, why structured output makes an LLM router trustworthy, and where this teaching-sized design needs reinforcement before it faces real users. If you want a broader catalogue of graph shapes, the article on routing, fan-out, critique and approval patterns in LangGraph complements this hands-on build.
Three questions, three different sources
Picture three users. The first asks about the company's leave policy; only internal documents can answer that. The second wants today's weather in Uttarakhand; no internal document will ever contain it, so the system has to search the web. The third asks what RAG is; the model already knows, and fetching anything would only add latency.
The target architecture therefore places a Gemini-powered router in front of three branches that all converge on a single answer-generation step:
User Question
|
v
+----------------+
| AI Router |
| (Gemini) |
+----------------+
/ | \
/ | \
v v v
FAISS Tavily Gemini
Internal DB Web Direct
\ | /
\ | /
v v v
+----------------+
| Generate Answer|
| Gemini |
+----------------+
|
v
Answer
The router is the heart of the design. The tempting shortcut is keyword matching, like the pseudocode below, where specific words in the question select a branch:
if "weather" in question:
use_tavily()
elif "leave" in question:
use_faiss()
else:
use_gemini()
Here that decision is delegated to Gemini instead. Because the model interprets the question rather than scanning it for trigger words, the workflow earns the label agentic.
Generative AI, agents and agentic workflows
These terms are often used interchangeably, but they describe different levels of model involvement.
Plain generative AI
The simplest application passes a prompt to a model and returns whatever it produces:
User Question
|
v
LLM
|
v
Answer
It can explain RAG from training data, but it knows nothing about your leave policy.
Agents that call tools
An agent extends the model with tools such as a database, a search engine or an external API, and lets it decide whether to call one before responding:
User Question
|
v
LLM
|
+------> Database
|
+------> Web Search
|
+------> API
|
v
Answer
A weather question, for example, can be answered by querying a weather service rather than having the model invent a plausible forecast.
Agentic workflows
An agentic workflow gives the model influence over the path that execution takes at runtime. In this project the flow looks like this:
Question
|
v
Router
|
+----> FAISS
|
+----> Tavily
|
+----> Gemini
The developer defines the three possible routes; the model only selects among them for each incoming question. It is not handed open-ended control of the application, just a menu of permitted actions. "Agentic routing workflow" is the most accurate name for that arrangement.
Setting up the project
You need Python 3.10 or newer, an API key for Google Gemini and an API key for Tavily. Install the libraries in one go:
pip install -U langchain langchain-google-genai langchain-community langgraph faiss-cpu tavily-python python-dotenv pydantic
Keep both keys in a .env file rather than in source code:
GOOGLE_API_KEY=your_google_api_key
TAVILY_API_KEY=your_tavily_api_key
The imports cover typing helpers, Pydantic for the routing schema, LangChain's document and FAISS wrappers, the Gemini chat and embedding classes, LangGraph's graph primitives and the Tavily client:
import os
from typing import List, Literal
from typing_extensions import TypedDict
from dotenv import load_dotenv
from pydantic import BaseModel
from langchain_core.documents import Document
from langchain_community.vectorstores import FAISS
from langchain_google_genai import (
ChatGoogleGenerativeAI,
GoogleGenerativeAIEmbeddings,
)
from langgraph.graph import StateGraph, START, END
from tavily import TavilyClient
The next step, shown as a one-line snippet, is simply to load the environment variables:
Load the environment variables:
With override=True, values from .env take precedence over variables already set in your shell:
load_dotenv(override=True)
Configuring Gemini for routing, answering and embeddings
The chat model does double duty. It first chooses which source should handle a question, and later composes the final answer from whatever context was gathered. The configuration below enables two automatic retries and leaves token and timeout limits unset:
model = ChatGoogleGenerativeAI(
model="gemini-3.6-flash",
max_tokens=None,
timeout=None,
max_retries=2,
api_key=os.getenv("GOOGLE_API_KEY"),
)
Model identifiers change frequently, so confirm the name in the snippet against the current Gemini model list before running it.
Embeddings are a separate concern handled by a separate model. Gemini Embedding converts each document into a numeric vector:
embeddings = GoogleGenerativeAIEmbeddings(
model="models/gemini-embedding-001",
google_api_key=os.getenv("GOOGLE_API_KEY"),
)
Vectors are what make semantic search possible: texts with similar meaning land close together in vector space, so a question can retrieve relevant passages even when it shares few exact words with them.
A tiny internal knowledge base in FAISS
To keep attention on the workflow, the knowledge base holds just three short documents: a 30-day refund policy, an allowance of 20 paid vacation days with a seven-day notice requirement, and weekday support hours. In a real system these texts would come from handbooks, PDFs, Notion pages, support tickets, internal docs or database rows.
documents = [
Document(
page_content="""
Our company provides a 30-day refund policy.
Customers can request a refund within 30 days of purchase.
"""
),
Document(
page_content="""
Employees receive 20 paid vacation days per year.
Vacation requests must be submitted at least 7 days in advance.
"""
),
Document(
page_content="""
The company provides technical support from Monday to Friday,
9 AM to 6 PM IST.
"""
),
]
Building the store and wrapping it as a retriever takes two calls. Setting k to 3 asks for the three nearest documents, which with this toy corpus means every document comes back ranked by similarity:
vector_store = FAISS.from_documents(
documents,
embeddings,
)
retriever = vector_store.as_retriever(
search_kwargs={"k": 3}
)
FAISS has no understanding of language; it is an index for nearest-neighbour search. The embedding model turns the incoming question into a vector, and FAISS returns the stored documents whose vectors lie closest to it.
Adding a Tavily client for live information
Web search needs only a client instance built from the second API key:
tavily_client = TavilyClient(
api_key=os.getenv("TAVILY_API_KEY")
)
Designing the shared state
State is the central idea in LangGraph: a typed dictionary that travels through the graph, which each node reads from and contributes to. This workflow needs the question, any retrieved documents, the Tavily results, the chosen source and the final answer:
class AgentState(TypedDict):
question: str
documents: List[Document]
tavily_response: str
source: str
answer: str
Conceptually it behaves like a shared record that every step can see:
AgentState
|
+-----------+-----------+
| | |
question documents source
|
tavily_response
|
answer
A node receives the current state, uses the fields it cares about and returns updates, which LangGraph merges before handing the state to the next node.
The FAISS retrieval node
This node takes the question from state, runs it through the retriever and stores the matching documents:
def retrieve_from_faiss(state : AgentState) -> AgentState:
question = state['question']
""" Fetch the details from the FAISS vector database
"""
result = retriever.invoke(question)
return {**state, "documents": result}
It executes only when the router has chosen the internal knowledge base. A question such as the one below is a typical trigger:
"What is our company leave policy?"
For that input, the retriever should surface the vacation document:
Employees receive 20 paid vacation days per year.
Vacation requests must be submitted at least 7 days in advance.
The Tavily search node
The web-search node sends the question to Tavily with search_depth set to advanced, then pulls the content field out of each result:
def search_with_tavily(state: AgentState) -> AgentState:
question = state['question']
""" Using the Tavily to search the web and
fetch the latest information about user query
"""
response = tavilyClient.search(
query=question,
search_depth='advanced'
)
contents = [result["content"] for result in response["results"]]
return {**state, "tavilyResponse":contents}
Those snippets become the context Gemini will read when writing the answer. Note that the node returns a list of strings; if you prefer a single block of text, join the items before storing them so the field matches the str type declared in the state.
Making the router reliable with structured output
A naive router asks the model to reply with one of three bare words:
Return only:
faiss
tavily
gemini
Language models do not always comply with formatting instructions. You might get a full sentence back:
I would choose tavily.
Or a slightly different phrasing:
The best option is: tavily
Either reply breaks a graph that needs one exact string to choose an edge. Structured output closes that gap. First describe the allowed decisions as a Pydantic model whose single field is restricted to three literal values:
class RouteDecision(BaseModel):
source: Literal["faiss", "tavily", "gemini"]
Then derive a router model that must return an instance of that schema. The json_schema method asks the provider to constrain generation to the schema instead of relying on prompt wording alone:
router_model = model.with_structured_output(
RouteDecision,
method="json_schema",
)
The output is validated against the Literal type, so an invalid value surfaces as an error instead of steering the graph somewhere undefined.
Writing the decision node
The routing prompt describes each option: faiss for questions about company policy and other internal knowledge, tavily for anything that needs current or web-based information, and gemini for general knowledge that requires neither:
def decide_source(state:AgentState)-> AgentState:
question = state["question"]
prompt = f"""
Decide the best source for answering this question.
Choose exactly one:
faiss:
Use when the question can be answered using our internal
knowledge base.related to company policy and all
tavily:
Use when the question requires current, recent, or web-based
information.
gemini:
Use when the question is general knowledge and does not
require our internal documents or current web information.
Question:
{question}
Return only one word:
faiss, tavily, or gemini
"""
response = model.invoke(prompt)
# return response
return {**state,"source":response.text}
Look closely at the last lines. As written, the node still calls the plain model and stores response.text, which is exactly the fragile free-text approach described above. To get the benefit of the schema, call router_model.invoke(prompt) instead and store the source attribute of the result. With that change the router returns a typed object like this one rather than arbitrary prose:
RouteDecision(source="faiss")
Telling LangGraph where to go next
Conditional edges need a function that reports which branch to follow. This one makes no decision of its own; it reads the choice the router already saved in state and hands it back to LangGraph:
def route_source(
state: AgentState,
) -> Literal["faiss", "tavily", "gemini"]:
return state["source"]
One answer node for every route
All three branches finish in the same generation step. It inspects source and builds a prompt accordingly: retrieved documents joined into context for FAISS, search snippets as reference material for Tavily, or the bare question for direct answers:
# Generate the answer for the user
def generateAnswer(state:AgentState) -> AgentState:
source = state["source"]
question = state['question']
documents = state['documents']
tavilyResponse = state['tavilyResponse']
if source == "faiss":
context = "\n\n".join([doc.page_content for doc in documents])
prompt = f"""Based on the following context answer the question below
Context:
{context}
Question:
{question}
"""
elif source == "tavily":
prompt = f""" Based on the following search result , use this as an reference and provdie the
answer to the below question
Context:
{tavilyResponse}
Question:
{question}
"""
else:
prompt = f" Answer the following question : {question}"
response = model.invoke(prompt)
answer = response.content
return {**state, "answer":answer}
The same Gemini model writes every answer. The only thing that varies is the context placed in front of it, which is the essential insight of RAG: better inputs, not a different model, produce grounded responses.
Keeping names consistent when you assemble the snippets
The snippets mix two naming styles, and the mismatches will cause errors if you paste them together unchanged. The state declares tavily_response while the nodes read and write tavilyResponse; the client is created as tavily_client but called as tavilyClient; and the answer function is defined as generateAnswer but registered as generate_answer. Pick one convention and apply it everywhere before running the graph.
Assembling the graph
At this point the pieces are a decision node plus three possible continuations:
decide_source
|
+----> faiss
|
+----> tavily
|
+----> gemini
There is a subtlety here. The gemini route is not a retrieval step at all; it means "skip retrieval and answer directly." So instead of creating an empty node for it, that branch can point straight at the shared generation node.
Start by creating a graph over the state type:
workflow = StateGraph(AgentState)
Register the four nodes:
workflow.add_node("decide", decide_source)
workflow.add_node("faiss", retrieve_from_faiss)
workflow.add_node("tavily", search_with_tavily)
workflow.add_node("generate", generate_answer)
Make the decision node the entry point:
workflow.add_edge(START, "decide")
Wire the conditional edges. The mapping translates each value the routing function can return into a node name, which is where gemini is sent directly to generate:
workflow.add_conditional_edges(
"decide",
route_source,
{
"faiss": "faiss",
"tavily": "tavily",
"gemini": "generate",
},
)
The retrieval and search branches must then flow into answer generation:
workflow.add_edge("faiss", "generate")
workflow.add_edge("tavily", "generate")
Generation is the final step before the graph ends:
workflow.add_edge("generate", END)
Compiling turns the definition into a runnable application:
app = workflow.compile()
The finished graph
The complete flow, from start to finish:
START
|
v
+--------------+
| decide |
| source |
+--------------+
/ | \
/ | \
v v v
+------+ +--------+ +---------+
|FAISS | | Tavily | | Generate|
| | | | | directly|
+------+ +--------+ +---------+
\ | /
\ | /
v v v
+----------------+
| generate |
| answer |
+----------------+
|
v
END
Keep the key principle in view: the graph fixes the set of possible paths, and the model picks one of them at runtime. That combination is what makes the workflow agentic without making it unpredictable.
Trying the three routes
A small helper builds the initial state and invokes the compiled app:
def ask_question(question: str):
initial_state = {
"question":question,
"documents":[],
"tavilyResponse":"",
"source":""
}
result = app.invoke(initial_state)
return result
A question that needs the web
The weather question should be sent to Tavily:
result = ask_question(
"What is the current weather in Uttarakhand?"
)
Print both the chosen source and the generated answer:
print("Source:", result["source"])
print("Answer:", result["answer"])
The expected route:
Source: tavily
Current conditions exist only on the web.
A general-knowledge question
Next, ask about Retrieval Augmented Generation:
result = ask_question(
"What is Retrieval Augmented Generation?"
)
Print the result the same way:
print("Source:", result["source"])
print("Answer:", result["answer"])
The router should skip retrieval entirely:
Source: gemini
The model can explain the concept on its own.
A question about internal policy
Finally, the leave-policy question:
result = ask_question(
"What is our company leave policy?"
)
And the same print statements:
print("Source:", result["source"])
print("Answer:", result["answer"])
This time the internal knowledge base should win:
Source: faiss
LLM routing is probabilistic, so treat these as expected outcomes, not guarantees.
Why semantic routing beats keyword rules
Here is the hardcoded alternative again:
if "weather" in question:
use_tavily()
elif "leave" in question:
use_faiss()
else:
use_gemini()
Rules like these decay quickly. Consider a user asking whether the office is open on Saturday. Nothing in that sentence mentions policy, yet the answer may well live in the internal documentation. A model-based router can infer the intent; a keyword list cannot unless someone anticipates every phrasing.
The approach also scales better. When new back ends arrive, such as the ones below, you extend the schema and the prompt instead of growing a tangle of conditionals:
SQL Database
Internal API
CRM
Customer Support System
Documentation
Web Search
The trade-off is an extra model call, with its cost and latency, before any real work begins.
Is this really an AI agent?
It is more accurate to call it a small agentic workflow than an autonomous agent. The available actions are fixed in advance:
FAISS
Tavily
Direct Gemini
The model has no way to decide on its own to do something like the following, because those capabilities were never given to it:
delete a database
send an email
call an arbitrary API
The architecture reads as a chain of responsibility:
Developer defines possible actions
|
v
LLM chooses action
|
v
LangGraph executes
|
v
Result
That constraint is a feature: bounded choices keep behaviour predictable and auditable in production.
The role each component plays
LangGraph: the workflow engine
LangGraph owns the structure of the application: state, nodes, edges, conditional routing and the order of execution. In the abstract, every step follows the same pattern:
State
|
v
Node
|
v
Updated State
|
v
Conditional Edge
|
+----> Node A
|
+----> Node B
|
+----> Node C
A graph of small steps is easier to test and observe than one sprawling function.
FAISS: the retrieval layer
FAISS provides the RAG portion of the system. In simplified form:
Company Documents
|
v
Embeddings
|
v
FAISS
|
v
Similar Documents
|
v
Gemini
|
v
Answer
A real deployment adds an ingestion pipeline in front of it:
Documents
|
v
Load
|
v
Split into chunks
|
v
Generate embeddings
|
v
Store vectors
|
v
Retrieve relevant chunks
|
v
Generate answer
The example skips loading and chunking to stay focused on the workflow; real handbooks need both.
Tavily: live web search
Tavily handles information that changes over time:
User Question
|
v
Router
|
v
Tavily
|
v
Search Results
|
v
Gemini
|
v
Answer
Typical cases include current weather, breaking news, recent product releases, fresh documentation, recent events and live market data. In production, also decide how search results will be cited, filtered, validated and presented, since web content is neither guaranteed accurate nor safe to pass to a model unchecked.
The pattern worth remembering
The components are interchangeable. The durable idea is routing: matching each question to the capability best suited to answer it.
Internal Knowledge
|
+------ FAISS
Current Information
|
+------ Tavily
General Knowledge
|
+------ Gemini
That per-request choice of capability reappears in nearly every serious agentic application.
Where to take it next
Add more tools
The router could choose among many more back ends:
SQL Database
REST APIs
CRM
Email
Calendar
Internal Documentation
Strengthen retrieval
Three documents are a demonstration, not a knowledge base. Real RAG needs document loaders, chunking, metadata, better retrieval strategies, reranking, source citations and access control, so that users only retrieve what they are allowed to see.
Validate routing decisions
A validation step between the router and the tools can reject implausible choices and fall back safely:
Router
|
v
Validator
|
+---- valid ----> Tool
|
+---- invalid --> Fallback
This matters more as the tool count grows.
Handle failures
Structured output guarantees a valid route, not a successful tool call. A search API can time out or return nothing:
Router
|
v
Tavily
|
X
Search failed
|
v
Fallback
Production systems need retries and a fallback, such as answering directly with a clear caveat. The article on designing resilient agent graphs with retries and fallbacks goes deeper on this.
Make it observable
When an answer is wrong, you need to know which stage failed:
Wrong route?
|
v
Bad retrieval?
|
v
Bad search results?
|
v
Bad generation?
Logging the selected source and intermediate results makes that question answerable.
Introduce loops
The current graph makes exactly one decision:
Question
|
v
Router
|
v
Tool
|
v
Answer
A more capable agent evaluates what it found and decides whether to act again:
Question
|
v
Reason
|
v
Tool
|
v
Evaluate Result
|
+---- Need more information?
| |
| v
| Tool
| |
+------------+
|
v
Final Answer
This is where agentic systems gain real power: the model judges whether the information at hand is sufficient or whether another action is required. It is also where you need iteration limits, so a loop cannot run indefinitely.
Key takeaways
The whole system answers one question: how should an AI application decide where an answer comes from? The finished workflow looks like this:
User Question
|
v
Gemini
Router
|
+----------+----------+
| | |
v v v
FAISS Tavily Gemini
Internal DB Web Direct
| | |
+----------+----------+
|
v
Gemini
Final Answer
- Define capabilities and boundaries in the graph; let the model choose among them at runtime.
- Use structured output for routing decisions, and make sure the decision node actually calls the structured model.
- Keep one answer-generation node and vary only the context you feed it.
- Treat routing as a testable component: log decisions and check them against representative questions.
- Add validation, fallbacks and observability before adding more tools, because every new tool multiplies the ways a request can fail.
From here the same foundation extends naturally to SQL databases, APIs, memory, human approval, evaluation nodes, retries and multi-agent setups. The graph grows, but the principle does not change: give the model useful capabilities, define how they may be used, and let it decide which one fits the task.