How Legalka KB uses AI: RAG, voice, and zero hallucinations

Moving to another country is almost always stressful, and Poland is no exception. A residence card, a visa, ZUS, a bank account, taxes, diploma recognition, family reunification — every step raises dozens of questions, and the answers are scattered across forums, chats, outdated articles, and government-office windows where the desk next door often tells you something different. It is easy to get lost, easy to trust bad advice, and easy to lose weeks, money, and nerves along the way. That mess is exactly what our project grew out of.

Legalka KB is a free Telegram bot that helps you make sense of relocating to Poland. Unlike a generic chatbot — which answers such questions with confident fiction, inventing account numbers, deadlines, and office addresses that do not exist — Legalka KB answers strictly from a verified knowledge base and honestly says “I don’t know” when the base has no answer. Here is exactly how the AI that makes that possible works inside.

At its core is RAG (Retrieval-Augmented Generation). Every section of every knowledge-base page is turned into a vector by the text-embedding-3-small model and stored in an index. When a user asks a question, the question is embedded too, and the system finds the closest sections by cosine similarity. If the user has picked a voivodeship, that region’s pages are boosted over general ones — so specifics like an account number or an office address win over generic pages.

Conversations rarely consist of complete questions. “And which documents?”, “for this”, “what about there?” — such replies are meaningless for search once detached from context. So the compact gpt-4o-mini model rewrites short and referential questions into standalone queries, taking the dialogue history and chosen voivodeship into account: “and which documents?” after a question about a business residence permit becomes “documents for zezwolenie na pobyt czasowy i pracę, mazowieckie”. That way the retriever never loses the thread.

The answer is composed by the same gpt-4o-mini — but only from the retrieved sections, with no knowledge off the top of its head. Each fact is tagged as “norm” (what the law requires) or “practice” — an observation from applicants in diaspora chats, always with a date of validity. If the context holds concrete data — an account number, an amount, an address — the bot gives it directly instead of deferring to “check with the office”, and adds official links for self-verification. It answers in the language of the question — Russian, Ukrainian, Polish, or English — translating facts from the Russian-language base. And if the relevance of what it found is below threshold, it honestly says “I don’t know” rather than guessing.

An important caveat: Legalka KB is not legal advice — it is collected and structured data. Even when the bot answers confidently and with references, that does not replace checking for yourself: follow the official links it provides and verify, one more time. Rules change, details depend on your specific case, and the final word always rests with the office.

You do not have to type your question — you can dictate it. A voice message from Telegram is transcribed by the whisper-1 model with a language hint for accuracy; the bot echoes the recognised text and runs it through the same RAG. The reverse works too: under every answer there is a “Read aloud” button — the text is stripped of markup, links, and the disclaimer, and voiced by the tts-1 model.

The practice layer of the base is filled semi-automatically. A separate LangGraph pipeline processes exports of diaspora Telegram channels and chats in batches: so far around 2.4 million messages have passed through it, of which 28,091 were deemed relevant, and from those 17,284 practical facts were extracted. The extraction is done by gpt-4o-mini — anonymised facts only, with no names, handles, or quotes. Next comes a fail-closed PII filter: regex over contacts, document numbers, and long quotes, optionally a second pass through an LLM; any “dirty” fact is dropped without being saved. Finally a classifier sorts the facts into 16 thematic verticals (pobyt, praca, banki, podatki, ZUS, and others) — batching here cut the share of the “junk” category from 51% to 23%.

The base curator edits pages through a dashboard with an AI agent — also built on LangGraph. The agent first works out the intent (a question about the page, or a request to edit it), draws up a minimal editing plan informed by the dialogue history, and rewrites only the page body, leaving the metadata untouched. It is forbidden to invent facts: instead it inserts a note that a source is needed. After editing comes validation — did any sections disappear, was the body truncated; if more than 15% is lost, up to two retries, then a rollback with no file written.

Answer quality is measured, not assumed. An LLM judge runs a golden-QA set and checks two things: did the answer convey the substance of the expected facts (by meaning, not word for word), and does it assert anything forbidden as fact. Current metrics — citation ~99%, fact coverage ~97%, abstention 100%.

None of this is tied to a single vendor. Every model — embeddings, generation, speech recognition, and speech synthesis — can be swapped for a local one (Ollama, LM Studio) via a single OPENAI_BASE_URL variable, while the code stays the same. For sensitive relocation data, that means the whole stack can run on your own server. Need a similar RAG assistant that answers from your data and does not make things up? Write to us: [email protected].

You can try the bot itself right in Telegram, for free — @legalka_pl_bot. And below is a summary: which model handles which task.