Semantic Search Service with a Pinecone Vector Database Behind an MCP Server

A language model can only answer from what it saw in training. Engineering standards, policy manuals, product specs and internal wikis are invisible to it unless that material gets supplied when the question is asked. The pipeline described here handles that for PDF collections: documents are embedded once into Pinecone, and any AI agent can query them through an MCP server built with the FastMCP SDK.

Why keyword search falls short

Plain keyword search looks like the obvious fix, but people seldom phrase a question with the exact words the documentation uses, so relevant passages slip by. Synonym dictionaries and query expansion help, yet both depend on word relationships someone defined by hand. Neither one understands what the words mean.

Semantic search takes a different route. Documents and queries are both turned into vector embeddings that encode meaning, and relevance is judged by comparing those vectors with cosine similarity, a measure of how closely two vectors point in the same direction in embedding space. A higher score means a stronger semantic link, so related passages come back even if the vocabulary differs completely. A question such as "How do plants convert sunlight into energy?" can surface text about chloroplasts and photosynthesis even though neither term appears in it.

Two components: the vector database and the MCP server

The system splits into two parts that can change independently: an ingestion pipeline and an MCP server. Ingestion runs once per document. After that, any connected agent can query the same indexed knowledge through the server over and over.

The ingestion pipeline turns raw PDFs into searchable vectors. It extracts the text, slices it into overlapping chunks, creates embeddings with Pinecone Inference and writes the vectors to a Pinecone index. Embedding a large collection costs real compute, so a simple log prevents repeat work. For every indexed file it stores the time of the latest embedding. On the next run, the pipeline compares each file's current modification time with the stored one. A changed document gets fresh embeddings, and an unchanged one keeps its existing vectors. The database stays in step with document edits and no redundant embedding calls happen.

The MCP server, built on FastMCP, receives a natural-language question from an agent, embeds it, runs a similarity search against Pinecone and returns the chunks that score highest. Since the interface is MCP, several AI applications can share one retrieval service without knowing anything about the vector database or the embedding model behind it.

Separating the two parts was deliberate. Swapping embedding models, adjusting the chunking approach or rebuilding the vector index can all happen without touching the MCP interface or any agent that relies on it.

Building the embeddings and chunking pipeline

PDFs are made for human eyes, not machine retrieval, so they must first become a representation that captures meaning. The pipeline starts by pulling text from each file with Python's pypdf library, reading page by page and joining the result into one string for easier handling later. It accepts single files as well as entire directories, so new document sets can be indexed with no change to the retrieval side.

Choosing how to chunk

With the text extracted, the document gets cut into smaller pieces called chunks, and that choice turned out to be among the most consequential in the whole pipeline. Embedding models have fixed context windows, which rules out embedding a whole document as one vector. Even when a document fits, a single embedding of an entire file gives a broad, blurry representation, because several unrelated topics get squeezed together and retrieval precision drops.

So each document is split into fixed-size chunks that overlap, which lets every embedding stand for a smaller, tighter span of text. Picking the size is a balancing act between context and specificity. Small chunks usually retrieve more accurately because each embedding captures a narrower idea, though they generate more vectors and add indexing time and storage. Large chunks keep more surrounding context and reduce the vector count, but each embedding covers a wider range of information, which can blunt precision. Semantic or sentence-aware chunking exists as a more sophisticated option, yet plain fixed-size chunking proved a simple and effective baseline for general-purpose retrieval.

chunks = [
    text[i:i+500]
    for i in range(0, len(text), 450)
]

/*
Fixed-size overlapping chunk generation used during document ingestion.
A chunk length of 500 characters and a stride of 450 characters preserve
contextual continuity while maintaining semantic specificity.
*/

Embeddings with Pinecone

Next, chunks become embeddings. An embedding is a dense numeric representation of text that encodes its meaning. Rather than matching keywords, semantic search compares vectors, so related passages match despite different vocabulary. The vectors for "cat" and "dog" sit near each other in the embedding space, while "cat" and "airplane" are far apart, which lets similarity be computed mathematically and not by exact word match.

The embedding model here is Pinecone's hosted multilingual-e5-large, accessed through the Pinecone Inference API. A hosted service means there's no separate machine-learning model to deploy or maintain, so the ingestion code can stay focused on document handling and the architecture stays simpler.

This model was chosen because it's tuned for semantic retrieval and covers more than a hundred languages. The E5 family also treats queries and document passages as different input types, which suits retrieval well. At indexing time, chunks are embedded as passages, since they're the searchable content. At query time, the user's question is embedded as a query. Both end up in the same semantic space, which raises the odds of finding the right material even when the wording differs a lot.

For throughput, chunks go to the API in batches of 96, the largest batch Pinecone Inference currently allows for multilingual-e5-large. Sending many chunks per request cuts the number of API calls, makes better use of resources and shortens indexing compared with embedding chunk by chunk. The finished embeddings and their metadata are then stored in the Pinecone vector database, ready for retrieval.

embeddings = pc.inference.embed(
    model="multilingual-e5-large",
    inputs=batch,
    parameters={"input_type": "passage"}
)

The MCP server for semantic search

With the documents indexed, agents need a way in. Rather than exposing the vector database to applications directly, a Model Context Protocol server built with FastMCP fronts it. MCP gives AI applications one standard way to call tools, so any compatible client can use retrieval with no knowledge of the database or embedding model beneath. The extra layer lets the retrieval code be reused, stay easy to maintain and remain decoupled from whatever applications consume it.

Retrieval is the server's main job. When an agent submits a question, the server embeds it with the same multilingual-e5-large model used at indexing time, runs a similarity search on the Pinecone index and returns the top-ranked chunks. Using one model on both sides keeps chunks and queries in the same semantic space, which is what makes comparing them meaningful.

To raise result quality, the search tool only returns matches with a cosine similarity above 0.82. In experiments, scores over that line generally meant a strong semantic fit between query and chunk, while lower scores tended to be loosely related at best. Dropping the weaker matches gives downstream language models more relevant context. A question like "How do I submit an expense report?" can reach the right policy page even if the document words it differently. Beyond search, the server exposes tools for maintaining the vector database: inserting vectors, deleting them and listing available indexes.

Since the system is meant for enterprise deployment, security is part of the design. Credentials come from Azure Key Vault, and every request needs a valid API key before any retrieval runs. Sensitive infrastructure details stay away from client applications, and access to the indexed documents is protected.

results = index.query(
    vector=query_embedding,
    top_k=5,
    include_metadata=True
)

What the project shows about RAG for internal documents

An effective semantic search system depends as much on engineering decisions as on the language models underneath. The chunking strategy, the embedding model, the batching approach and the retrieval architecture all directly shape quality, scalability and maintainability. Putting ingestion and retrieval behind an MCP server yields a modular system that's reusable, simple to maintain and easy to extend as needs change.

The current version is a solid base with clear room to grow. Semantic chunking, reranking models, hybrid search and richer metadata could all improve retrieval further while keeping the modular structure intact. As retrieval-augmented generation becomes routine in enterprise AI, reliable semantic retrieval is turning into a basic building block of production systems, and pairing Pinecone Inference with an MCP retrieval layer is a practical, scalable way to build reusable semantic search.