How to structure a LangGraph agent with more than 80 tools

"How do you structure a single LangGraph LangGraph is a developer framework for building AI agents that follow multi-step workflows with branching logic, rather than answering in a single pass. Full definition → agent that has more than 80 tools?" The LangGraph docs show the mechanics — StateGraph, ToolNode, bindTools — but not what to do once the tool catalogue no longer fits in your head. Our answer from production is short: keep the graph as simple as it gets, two nodes in a loop, and move all the structure into the tool catalogue. That is how the eKsięgowy AI agent works: 58 base tools and up to 82 with the HR and KSeF KSeF is Poland's mandatory national system for issuing and receiving e-invoices directly with the tax authority. Full definition → modules, with no router and no sub-agents. Below we cover how we group the tools, how we name and describe them, what goes into the system prompt A prompt is the instruction you give an AI model, and how precisely it is written decides whether the answer comes back useful or vague. Full definition →, how much context all of it takes, and how we will know it is time to change the structure. The tour of the whole product is in Inside Accounting AI Agent; this piece is a how-to you can carry over to another project.

The short answer: one graph, two nodes, 82 tools

The graph is a StateGraph over MessagesAnnotation state. The agent node calls the model with the tools bound to it (model.bindTools), and the tools node is the stock ToolNode with the same list. A conditional edge checks one thing: does the model's last message contain tool_calls? If it does, we go to the tools and back to the agent; if not, we are done. The whole run has a recursionLimit of 25, so the agent cannot spin in endless calls.

flowchart LR
  subgraph CAT["Tool catalogue (code)"]
    D1["16 domains<br/>58 base tools"]
    D2["HR: 15 tools<br/>when the HR service is wired"]
    D3["KSeF: 9 tools<br/>when the KSeF service is wired"]
  end
  CAT --> F["createAllTools<br/>one flat list, up to 82"]
  F --> B["bindTools<br/>names, descriptions, schemas"]
  P["System prompt<br/>cross-tool rules + memory"] --> A
  B --> A["agent node<br/>model call"]
  A -->|tool_calls| T["tools node<br/>ToolNode runs the code"]
  T -->|markdown results| A
  A -->|no tool_calls| E["answer"]

The tool list is built on every request in a single function, createAllTools, because the tools are closures: each one receives the user's id, their language and their wFirma connection. That matters for security — no tool takes a userId argument, so the model cannot "ask" for another customer's data. And it matters for structure: since the catalogue is assembled in code anyway, code is where it can be organised.

Why not a router or a supervisor

In January 2026 we sketched the variant intuition suggests: a router and four specialists — contractors, finance, invoices and tax — each with its own subset of tools and its own prompt. The router was to return JSON with the specialist's name and a confidence level. The production chat, however, ran on one agent with the full tool list from the start; the router code was never wired into it, and in April we deleted it as dead code.

The reason is visible in the router's own prompt. The instruction read: if the query concerns two or three areas, choose the dominant one. An example from that prompt: "Show invoices and calculate VAT due" → the tax specialist. In accounting, questions like that are the norm, and the tax specialist had no invoice tools. A router turns a cross-cutting question into a single-domain one and adds a separate model call before every answer. A flat list lets the model do what an accountant would: fetch the period's invoices with get_invoices, then do the maths.

To be fair: we did not A/B test the two variants. This is a design decision based on the mechanism, not on a benchmark. A router makes sense when the domains genuinely do not mix, or when the specialists need different models or different permissions. In our case neither held.

Group in code, not in the graph

Since the graph does not divide the tools, the code does. Each domain has its own file of factories: invoice.tools.ts exports ten createXxxTool functions, contractor.tools.ts four, and so on. A factory takes its dependencies (the wFirma service, the cache A cache reuses the repeated, unchanging part of a prompt, which model providers charge less for — so a well-ordered prompt can cost several times less. Full definition →, the subscription service), the user's id and language, and returns a ready LangChain tool with a Zod schema. createAllTools puts everything into one array, with a comment above each group.

When a file grows, we split it further without changing what the model sees. HR is 15 tools in four files — employees (5), contracts (3), payroll (4), absences and summary (3) — behind a single barrel file, hr.tools.ts. To the model it is still 15 entries on a flat list; to a developer it is four files of 170–330 lines instead of one large one.

Optional groups are added conditionally: the 15 HR tools join the list only when the HR service is wired, and the 9 KSeF tools only when the KSeF service is. It is the cheapest way to a smaller catalogue: a tool that cannot run should not take up room in the context. Two tools — the tax-deadline calendar and the White List bank-account check — do not depend on wFirma and are always available.

Names: verb_noun and list–details pairs

All 82 names are unique and follow a verb_noun shape in snake_case: get_invoices, get_invoice_details, create_invoice, update_invoice, delete_invoice. Lists and details always come in pairs — get_payments and get_payment_details, get_vehicles and get_vehicle_details — so a model that has learned one domain predicts the names in the next.

Our catalogue is not perfect here, and it is worth showing. Contractors and invoices are created with create_, while payments, vehicles and terms use add_; that is a trace of how the modules arrived one after another. There are also names close to each other: send_invoice emails an invoice, while send_invoice_to_ksef submits it to the state e-invoicing system. On a flat list, close pairs like that have to be defused in the description, because the name alone will not separate them. If you are starting from scratch, fix your verb vocabulary before the first tool, not after the fiftieth.

Tool descriptions do the routing

In an agent without a router, the descriptions decide which tool the model picks. The median description in our catalogue is 130 characters long, the shortest is 33 (add_invoice_note) and the longest 543 (verify_bank_account_white_list). The spread is deliberate: we spend length where tools collide, not evenly across all of them.

The invoices-and-expenses pair shows it best. Until April 2026 the get_invoices description read: "Get list of invoices from wFirma. Can filter by year, month, and payment status". The model mixed up sales invoices with costs and could skip the date filter even when the user named a month. The fix changed only the descriptions and the schema. get_invoices now says outright that it returns OUTGOING invoices, issued to clients, and that bills from vendors come from get_expenses; the get_expenses description says the same in the other direction. Both add a rule: if the question mentions a period, passing dateFrom and dateTo is mandatory, and an unfiltered result is a bug.

A description can also point to a tool that has to be called alongside. get_terms returns the deadlines the user entered in wFirma themselves; its description adds that when someone asks about deadlines in general (in Russian too, "сроки"), get_tax_deadlines should be called as well, for the statutory VAT, CIT, ZUS and PIT deadlines. Without that one sentence the agent would answer a question about deadlines with the user's own notes alone.

The system prompt: only rules that span more than one tool

A tool description talks about that tool. Rules that tie several tools together, or depend on how the conversation unfolds, live in the "Tool Usage Guidelines" section of the system prompt. There are three kinds.

The first is the boundary between tools and knowledge: tools when the user asks about their own data (invoices, contractors, payroll), the model's knowledge when they ask about the law in general. The second is multi-step flows. calculate_payroll only computes a preview, while save_payroll_record computes and saves; the prompt says the words "save", "zapisz", "сохранить", "записать" always mean save_payroll_record, and that a record is replaced with delete_payroll_record plus a new save — never with the payment tools. The third is legal duties: before recording a payment of 15,000 PLN or more to a Polish contractor, the agent calls verify_bank_account_white_list, because Art. 117ba of the Polish Tax Ordinance requires it.

Note that the trigger rules list words in several languages. The agent answers in Polish, English and Russian and detects the language from the message itself, so a rule written only in English would not fire on "zapisz wypłatę". Beyond these rules, the prompt does not enumerate tools. bindTools does that, and a tool list in the prompt would be a second copy that sooner or later drifts away from the code.

Parameters: one shape across sibling tools

The same April fix unified the schemas. Previously get_invoices took a year and a month and get_expenses something else; now both take dateFrom and dateTo as YYYY-MM-DD, and the parameter description carries an example: for "April 2026" pass 2026-04-01 and 2026-04-30. A model that has learned to filter one list filters all of them.

Optional parameters are written in Zod as .nullable().optional() — the catalogue has more than 200 such fields. That is an adaptation to OpenAI's strict mode: according to the function calling guide, every field must be marked required, and optionality is expressed by allowing null as a type. Where the set of values is closed we use an enum — the invoice status, for instance: all, paid, unpaid, overdue, draft, issued, sent — so the model does not guess the spelling. There are 304 parameter descriptions in the catalogue, and they are what carries the format examples.

The prompt budget: how much 82 tools take

This is the question the docs do not answer, and with a large catalogue it is the one that matters most. We measured, statically, the text that goes to the model on every call.

The takeaway is simple: with 82 tools, the largest fixed part of the context is the catalogue, not the system prompt. The tool and parameter descriptions alone come to nearly 24 thousand characters, while the whole system prompt, tax data included, is about 9.5 thousand — two and a half times less. By the rough four-characters-per-token A token is a fragment of text, usually a piece of a word, and it is the unit AI models measure length and charge by — bills are counted in tokens, not in questions. Full definition → rule our code uses for its memory budget, the catalogue's prose alone is on the order of 6 thousand tokens; the serialised JSON schemas add field names, types and enum values on top, which we did not measure separately. OpenAI states plainly that function definitions count against the context limit and are billed as input tokens.

That cost is paid on every call of the agent node, not once per question: an answer that needs two tools in sequence is three model calls with the full catalogue. So we cut where it is cheap. Optional groups drop out when they cannot be used. Long-term memory has a hard cap of 20 entries and 6,000 characters. And the conversation history stored in the database holds only the user's messages and the agent's answers — tool results live for one turn and do not pile up with every exchange. How to turn tokens into money is covered in how much an AI agent costs per month.

Tool results: ready markdown, caps, and errors as text

The other side of the budget is what tools return. Our tools do not hand back raw JSON from the wFirma API An API is the interface two systems use to exchange data automatically, with no Excel exports and no copying by hand. Full definition →. Each has a formatter that returns finished markdown — a table, a heading, a few bold fields — already in the user's language and with Polish number formatting. The model gets less text, does not have to rewrite it, and the prompt tells it to keep that formatting as is.

Lists are capped: invoices, expenses and contractors at 100 items, documents at 50, HR records at 20 by default. Without a cap, a single "show me all invoices" from a client with several years of history could take more context than the entire tool catalogue.

Errors do not break the graph. A tool catches the exception and returns a short message in the user's language — that the invoices could not be fetched, or that the wFirma request limit has been reached — and the prompt tells the model to explain what went wrong and suggest a next step instead of inventing data. Exception text from external systems goes through a sanitizeForPrompt function: it is trimmed to 500 characters and stripped of markdown headings and bold, so outside data does not look like instructions to the model.

When we would change this structure

The OpenAI guide advises having fewer than 20 functions at the start of a turn — as a soft suggestion, not a limit. We have more than four times that, and the setup works in production, but we have no benchmark that says how much better a smaller catalogue would do. What we do know is how we will tell that a flat list is no longer enough: two tools keep getting confused despite cross-references in both descriptions, the catalogue grows by domains that connect to nothing else, or the catalogue's cost at every step starts to dominate the bill.

There are three ways forward then. The first is tool selection by search: the langgraph-bigtool library indexes tool descriptions and gives the agent a tool for finding the rest, and OpenAI offers tool search, which defers rarely used tools — but only on models from gpt-5.4 onwards. The second is subgraphs for domains that genuinely do not mix. The third, often the cheapest, is merging one domain's CRUD tools into fewer tools with an operation parameter — for us the candidates would be terms and term groups, ten tools of a very similar shape.

One constraint is specific to us and may apply to you too. Customers connect their own keys: by default that is gpt-4o on OpenAI or gemini-2.5-flash on Google, and the user can pick a different model. The agent's structure therefore has to work the same with both providers, which is why mechanisms that only one of them offers, such as OpenAI's tool search, cannot be its foundation.

Checklist

Graph. A StateGraph with two nodes, a ToolNode, a conditional edge on tool_calls, and a recursionLimit. Nothing more until there is a concrete reason.

Catalogue. One file of factories per domain, one function that assembles the list, optional groups added only when they can run. User context in the closure, not in the arguments.

Names and descriptions. One verb vocabulary, list–details pairs, two-way cross-references between tools that get confused, and more words where the mistakes happen.

Prompt. Only rules that span several tools, with trigger words in every language your users write in. No tool list.

Budget. Measure the catalogue, cap lists in results, return finished markdown, keep tool results out of the history, and remember that the catalogue is paid for at every step of the loop.

What the whole agent looks like — with KSeF, receipt OCR OCR (Optical Character Recognition) turns a scan or photo of a document into text a computer can read — the first step to an emailed invoice entering your system by itself. Full definition → and long-term memory — is covered in Inside Accounting AI Agent, and the product itself is at eKsięgowy AI.