A QuickBooks Finance Agent Running on a Local LLM With Ollama and LangChain

Here is a finance assistant that plugs into QuickBooks Online, pulls a business's live books and lets the owner chat with an AI about the actual figures: real invoices, real cash flow, real receivables aging, and none of it demo data. The model runs entirely on a laptop. An NVIDIA GeForce RTX 4070 with Qwen 3.5 9B served through Ollama does all the reasoning, so there's no OpenAI call, no Anthropic call, no cloud inference and no AI bill during development.

What follows covers the real code, the architecture choices and what the assistant produces, rather than theory.

The problem it addresses

A small business owner on QuickBooks has all the financial data in one place, but turning it into insight takes either accounting skills or paid professional help. Standard reports state what happened. They don't say what to do next.

The aim was to let someone type a plain English question about their own money and get a specific, data-backed reply: invoice numbers, customer names and the real cash position, not generic advice.

Architecture

Three layers make up the system:

• Frontend: Next.js 14 with TypeScript and shadcn/ui
• Backend: FastAPI, PostgreSQL and SQLAlchemy
• AI: a LangChain ReAct agent, Qwen 3.5 9B via Ollama, and a set of QuickBooks tools

The decision that matters most is building the AI layer on LangChain's agent framework. It lets the model choose for itself which data to fetch before it answers, and that is why replies come out specific instead of generic.

Running it on a laptop GPU

The full AI layer sits on a consumer laptop GPU, an RTX 4070 with 8 GB of VRAM. The measured numbers are below.

A response takes about a minute. For financial analysis that's tolerable, since people asking substantive questions expect thoroughness, not chatbot banter. In production, pointing the LangChain abstraction at a cloud inference provider brings that under five seconds, and the application code stays untouched.

Three reasons made local the right call for development:

• It costs nothing. Every prompt tweak, debugging session and edge-case test ran with no spend and no rate limits.
• It keeps data private. QuickBooks figures never left the machine, so no invoice amount, customer name or cash flow number reached a third-party AI provider.
• It proves a point. If a 9B-parameter model on a consumer GPU can give the quality of output shown later, the hardware bar for building local AI apps is lower than many developers assume.

Because LangChain sits in the middle, development and production share one codebase. Swapping the inference backend takes two lines:

Tools, memory, agent logic and prompts all stay the same. The model is an implementation detail.

The agent

The AI layer uses LangChain's ReAct agent framework. The agent holds a set of tools, which are plain Python functions that fetch live QuickBooks data, and picks which ones to call according to the user's question. This is the full setup from agent.py:

from langchain_ollama import ChatOllama
from langchain_core.tools import tool
from langchain.agents import create_agent
from langchain_core.runnables import RunnableConfig
from quickbooks_client import fetch_invoices, fetch_profit_loss, \
                             fetch_cash_flow_summary, fetch_ar_aging

llm = ChatOllama(
    model='qwen3.5:9b',
    base_url='http://127.0.0.1:11434'
)

@tool
def get_invoices(config: RunnableConfig) -> str:
    """Fetch all invoices from QuickBooks including amounts and balances."""
    user_id = config.get('configurable', {}).get('user_id')
    return json.dumps(fetch_invoices(user_id=user_id), default=str)

@tool
def get_ar_aging(config: RunnableConfig) -> str:
    """Get AR aging report with buckets and DSO calculation."""
    user_id = config.get('configurable', {}).get('user_id')
    return json.dumps(fetch_ar_aging(user_id=user_id), default=str)

@tool
def draft_collection_notice(customer_name: str, amount: float,
                            days_overdue: int) -> str:
    """Drafts a reminder for an overdue invoice based on severity."""
    if days_overdue > 60:
        tone = 'Firm & Urgent'
        msg = f'Final reminder: your balance of ${amount} is significantly overdue.'
    elif days_overdue > 30:
        tone = 'Standard Follow-up'
        msg = f'Checking on your ${amount} payment, now 30+ days past due.'
    else:
        tone = 'Friendly Reminder'
        msg = f'Quick note: your payment of ${amount} was due recently.'
    return json.dumps({'tone': tone, 'draft': msg})

tools = [get_invoices, get_profit_loss, get_cash_flow_summary,
         get_ar_aging, draft_collection_email, draft_collection_notice]

agent = create_agent(
    model=llm,
    tools=tools,
    system_prompt=(
        'You are an AI CFO assistant for small businesses. '
        'Analyze financial data and give clear, actionable advice. '
        'Always highlight cash flow risks and overdue invoices. '
        'Be concise and specific with numbers.'
        '\n\n--- COLLECTIONS SPECIALIZATION ---'
        '\nWhen asked about unpaid invoices or specific customers:'
        '\n1. Use get_ar_aging to identify debt seniority.'
        '\n2. Use draft_collection_notice to generate professional follow-ups.'
        '\n3. Explain the impact on liquidity, not just the invoice.'
    )
)

Two points deserve attention. First, user_id travels through every tool call via LangChain's RunnableConfig, which is what makes the system multi-tenant: each user's agent touches only that user's QuickBooks data. Second, the agent actually reads tool docstrings when deciding what to invoke, so they should be written as instructions to the model, not as descriptions for humans.

Conversation history

Each session keeps its own history so follow-up questions can lean on earlier turns:

conversation_history: dict[str, list] = {}

def ask(question: str, session_id: str = 'default',
        user_id: str = None) -> str:
    if session_id not in conversation_history:
        conversation_history[session_id] = []

    conversation_history[session_id].append(
        {'role': 'user', 'content': question}
    )

    config = {'configurable': {
        'user_id': user_id,
        'session_id': session_id
    }}

    response = agent.invoke(
        {'messages': conversation_history[session_id]},
        config=config
    )

    answer = response['messages'][-1].content
    conversation_history[session_id].append(
        {'role': 'assistant', 'content': answer}
    )
    return answer

A user can ask "What is my DSO?" and then "Which customers are driving that?" and the agent understands what "that" means. History is keyed by session_id, so concurrent users keep separate conversations.

The chat endpoint

The FastAPI endpoint behind the chat is simple. The key detail is that user_id comes from the verified JWT and never from the request body, which stops anyone from reaching another user's data by editing a parameter:

@app.post('/ask')
async def ask_agent(
    request: AskRequest,
    current_user: User = Depends(get_current_user)
):
    # Always use the verified user from JWT, never trust the request body
    user_id = str(current_user.id)

    answer = ask(
        question=request.question,
        session_id=request.session_id,
        user_id=user_id
    )

    return {
        'question': request.question,
        'answer': answer,
        'session_id': request.session_id
    }

Token refresh

The QuickBooks client refreshes OAuth tokens automatically. They expire after 60 minutes, so the client checks the token's age before each request and renews it when needed:

def get_qb_client(user_id: str = None):
    db = SessionLocal()
    qb_token = db.query(QBToken).filter(
        QBToken.user_id == uuid.UUID(user_id)
    ).first()

    if not qb_token:
        raise Exception(f'No QB token found for user {user_id}')

    auth_client = AuthClient(
        client_id=os.getenv('QB_CLIENT_ID'),
        client_secret=os.getenv('QB_CLIENT_SECRET'),
        redirect_uri=os.getenv('QB_REDIRECT_URI'),
        environment=os.getenv('QB_ENVIRONMENT'),
        access_token=qb_token.access_token,
        refresh_token=qb_token.refresh_token,
    )

    # Only refresh if token is older than 55 minutes
    token_age = datetime.utcnow() - qb_token.updated_at
    if token_age > timedelta(minutes=55):
        auth_client.refresh()
        qb_token.access_token  = auth_client.access_token
        qb_token.refresh_token = auth_client.refresh_token
        qb_token.updated_at    = datetime.utcnow()
        db.commit()

    return QuickBooks(
        auth_client=auth_client,
        refresh_token=auth_client.refresh_token,
        company_id=qb_token.realm_id,
        minorversion=75
    )

Refreshing at 55 minutes instead of 60 leaves a cushion against clock skew. Refresh happens server-side and the token lives in PostgreSQL, never in the frontend.

What the agent produces

This is the part that counts. Here's the actual output for the question "What is my DSO and is it healthy?", run against QuickBooks sandbox data:

Nothing here is canned. The agent invoked get_ar_aging, got live bucket data back and worked through it. The reply points at named customers, estimates what the delay does to cash flow and suggests drafting collection notices. It can offer that because a draft_collection_notice tool exists and the agent recognizes when it fits.

Why this isn't a simple RAG bot

What separates it from a retrieval chatbot is that the agent chooses its tools. Asked "What is my biggest cash flow risk?", it doesn't run everything. It reasons about the question and picks the relevant ones. The ReAct loop looks like this:

For that question it skips get_invoices and get_profit_loss because they're irrelevant. That selectivity keeps answers focused and stops the context window from filling with unneeded data.

The dashboard

The dashboard draws on the same QuickBooks tools the agent uses. Every figure is live, with no caching layer between the user and QuickBooks, so the aging chart and the list of top overdue invoices refresh on every page load.

The AR aging page goes past the stock QuickBooks report. It computes DSO on the fly, assigns a risk level to each bucket and lists every invoice with its exact days overdue. The numbers come from the same fetch_ar_aging function the agent calls, which gives one source of truth.

Design choices worth calling out

• Read-only OAuth. The scope requested has no write permissions, so the app structurally can't change QuickBooks data.
• user_id from the JWT. The /ask endpoint trusts the verified token and nothing the client sends.
• Nothing financial is stored. QuickBooks numbers are requested when needed and discarded afterward. Only users, tokens and schedules live in the database.
• A local LLM while building. No financial data went to any outside AI provider during build and test.

The practical benefits of staying local compared with a cloud-first approach:

The cost is latency, around 60 seconds on the 4070 versus under 5 with a cloud API. For a tool whose users want substantive answers, that's fine in development and testing, and switching to a cloud model for production is trivial.

How long it took

Work that would normally take several weeks shrank to a few days. A rough timeline for a solo developer using AI coding assistance throughout:

The core system was running after three to four days, plus another day for refinement and testing. Three things ate the most hours: oddities in the QuickBooks API (the aging report layout is far from intuitive), tuning the LangChain agent (docstrings on tools count for more than expected), and carrying user_id correctly through each tool call for multi-tenancy. Speed came mostly from FastAPI's built-in validation, LangChain's provider abstraction, and AI coding assistance for boilerplate, error debugging and architecture questions.

What should change

Several things would be done differently on a second pass:

• Wire tools and agent together from day one. The data layer was finished first and connected to the agent afterward, so interface mismatches showed up late.
• Prototype prompts on a 3B model. The 9B Qwen gives better quality but crawls on this GPU, so a tiny model for quick prompt experiments and the 9B for final checks would have saved hours.
• Stream from the start. Waiting a minute without a single token on screen is miserable, and token streaming belonged in the first day's requirements.
• Test the QuickBooks tools. Edge cases only appear on real data, and a small suite against sandbox data would have prevented several long debugging sessions.

Known limitations

Any honest write-up should say what's unfinished. Here's what holds the current build back from scale, and how each gap would be closed.

In-memory history

Conversation history sits in a plain Python dictionary in RAM:

That's fine for one development server and has three production problems. History disappears on every restart. It can't work across several server instances, which rules out horizontal scaling. And nothing survives between sessions, so closing the browser ends the conversation.

The production fix is moving history to Redis, for fast reads and writes with TTL expiry, or to PostgreSQL for full persistence. LangChain's message history abstractions support both, so the swap is easy.

No RAG, on purpose

The system uses no retrieval: no embeddings, no vector database, no chunking pipeline. That's deliberate. Financial data moves constantly, with invoices getting paid, new ones appearing and balances shifting, so embeddings built yesterday may be stale today. Fetching live data from QuickBooks on every request guarantees that answers reflect the business as it stands.

RAG would fit static or slow-moving material such as contracts, policies and historical reports. For live transactional data, direct tool calls beat retrieval over embeddings in accuracy.

The price is latency and token use, since fresh fetches are slower and fill more of the context than a compact summary from a vector store. At current volumes of tens to low hundreds of invoices per business, that's acceptable. At larger scale a hybrid makes sense, with RAG for history and live tools for current data.

Large invoice volumes

Right now all invoices go into the agent's context in full. With 30 to 50 invoices that works well. With thousands it hits token limits and quality drops. The fix is filtering before the model sees anything: fetch only invoices relevant to the question, summarize big datasets first, or use two stages where a quick retrieval step narrows the pool before reasoning begins.

No streaming yet

The server returns the whole answer at once, so on the local GPU users stare at a blank screen for about a minute. FastAPI offers StreamingResponse, Ollama can stream tokens through its API and LangChain agents support streaming callbacks. Connecting them is easy, but it was postponed to get core functionality working, and it's a priority before launch.

Sessions that don't last

The session_id is created in the frontend when the chat page loads. A new tab, a refresh or a return after closing the browser creates a new one and history starts fresh, even if the server never restarted. A financial assistant would gain a lot from persistent memory, such as remembering that a customer is on a payment plan or that an invoice was discussed last week. Doing that properly means storing sessions in the database and letting users name, resume and delete past conversations.

Takeaway

LangChain's abstraction, Ollama's local inference and QuickBooks' data access add up to something more useful than any one part. The agent doesn't merely retrieve data. It reasons over it, selects what matters and answers in terms of each user's own finances.

Going local isn't a compromise. It's a legitimate development strategy with real privacy, cost and iteration advantages, and a consumer GPU like the RTX 4070 runs capable 9B models. For anyone planning an AI-powered app, the better question isn't which cloud provider to pick. It's whether capable models can run locally during development, so the product gets validated at zero inference cost and the provider decision waits until real usage data exists. On an RTX 4070, the answer is yes.