Moving to another country is almost always stressful, and Poland is no exception. A residence card, a visa, ZUS, a bank account, taxes, diploma recognition, family reunification — every step raises dozens of questions, and the answers are scattered across forums, chats, outdated articles, and government-office windows where the next desk over often tells you something different. It is easy to get lost, easy to trust bad advice, and easy to lose weeks, money, and nerves along the way. That mess is exactly what our project grew out of.
Legalka KB is a free Telegram bot that helps you make sense of relocating to Poland. Unlike a generic chatbot — which answers such questions with confident fiction, inventing account numbers, deadlines, and office addresses that do not exist — Legalka KB answers strictly from a verified knowledge base and honestly says «I don't know» when the base has no answer. In this article we walk through, in plain terms, how the system works inside: where the knowledge comes from, how an answer is born, what keeps the bot from making things up, and how we keep the base up to date. Along the way — every diagram from our internal documentation.
The core principle that everything else rests on is simple: the knowledge base is the product. The bot is not «smart on its own» — it merely carefully retrieves and retells what already lives in the verified base. Not in the base — no answer. The whole architecture, which we now turn to, follows from this principle.
How the whole system is built
Before diving into details, it helps to see the whole map. The system has four large parts: the knowledge base, the practice collector, the assistant with its Telegram bot, and a control panel for the curator.
On the left are the sources: open diaspora chats and channels on Telegram. From them the collector draws «practice» — how things actually go. On the right are the users themselves, asking the bot questions in five languages. In the centre is the knowledge base, split into two layers, and the assistant that turns a question into an answer. The curator runs all of this through a separate panel. Next we go through each block in turn.
Technically the project is assembled as a single whole: each large part is responsible for its own area of work and can evolve independently of the others.
The knowledge base: «norm» and «practice»
The heart of the system is the knowledge base itself — a set of structured pages. Its knowledge is deliberately split into two layers that are never mixed.
The «norm» layer is what the law requires. Every claim comes with a link to a source (where possible, to a specific article), is marked verified only after a lawyer's review, and is re-checked at least once every 60 days.
The «practice» layer is how it works in reality: timelines, inspectors' habits, the quirks of specific offices. These are applicants' observations, always framed «based on applicants' reports» and always dated, with a date after which the fact is considered stale (no later than +90 days).
No page reaches users right away. Everything is born as a draft and must pass review — this guards against raw or mistaken data going «to production».
While a page is still a draft, the bot can still show it, but honestly marks such an answer as unverified — so as not to lose fresh data, yet not to pass it off as finally checked.
How an answer is born
This is the most important part — the whole point of the exercise. When a user types a question, the answer runs through a chain: rephrasing → retrieval → relevance scoring → generation with a check → recording for feedback. At every step the system has the option to honestly stop and say «I don't know».
Let's go step by step.
Rephrasing. A conversation rarely consists of complete questions. «And which documents?», «for this», «what about there?» — such replies are meaningless for search once detached from context. So a compact gpt-4o-mini model rewrites short and referential questions into a standalone query, taking the dialogue history and chosen voivodeship into account: «and which documents?» after a question about a business residence permit becomes «documents for zezwolenie na pobyt czasowy i pracę, mazowieckie». That way the search never loses the thread.
Retrieval. Every section of every page in the base is turned in advance into a vector by the text-embedding-3-small model and kept in a search index. The question is turned into a vector too, and the system finds the closest sections by meaning using cosine similarity. If the user has chosen a voivodeship, that region's pages are boosted over general ones — so specifics like an account number or an office address win over general pages.
Relevance scoring. Before generating anything, the system looks only at the search scores (a separate, strictly tested step). If the best result is below a set threshold, or there are no hits, the bot abstains right away. If the match is obviously good and the question was not rephrased, it answers immediately, without an extra model check. In the in-between cases an additional check kicks in.
The model check. It compares the ORIGINAL user question (not the rephrased query) against the retrieved fragments — because rephrasing sometimes «drifts» onto the previous topic and scores the wrong page highly. If the check cannot be interpreted unambiguously, the system leans towards answering. The bot abstains only if the context is judged irrelevant.
Generation. The answer is composed by the same gpt-4o-mini model — but only from the retrieved sections, with no knowledge «off the top of its head». Each fact is labelled as «norm» (what the law requires) or «practice» (an applicants' observation, always dated). If the context holds specifics — an account number, an amount, an address — the bot gives them directly instead of deferring to «check with the office», and adds official links for self-verification. It answers in the language of the question, translating facts from the Russian-language base. At the end — a disclaimer.
Recording and feedback. Every answer served is recorded (with personal data always removed) so that a later 👍 or 👎 attaches to exactly what grounded it, and survives a restart of the bot. A 👎 vote is automatically split into a type — bad retrieval, bad generation, or a gap in the base — and lands on the panel as a signal for what to fix first.
What the bot can do for the user
An answer strictly from the base is the core, but around it we've built a whole set of conveniences to make the bot comfortable to use.
Voice both ways. You don't have to type your question — you can dictate it: a voice message is transcribed by the whisper-1 model with a language hint, the bot echoes back the recognised text and handles it like an ordinary question. The reverse works too: under every answer there is a «Read aloud» button — the text is stripped of the extra bits and voiced by the tts-1 model.
Buttons under the answer. 👍/👎 — feedback; 🔊 — read aloud; ✍️ — suggest an edit (goes into a suggestions queue); 📍 — show on a map, if the answer contains a Polish address; 🔔 — join a waitlist if your voivodeship has no local practice yet; 📋 — create a checklist, if the answer lists points.
Commands. /topics — a browse of the base's sections with sample questions; /checklist and /checklists — personal, editable lists (e.g. «Me» and «Wife»); /region — pick a voivodeship; /lang — the interface and answer language; /new, /chats, /reset and others — memory across several parallel chats.
A word on languages: the interface and answers are available in five — Russian, Ukrainian, Belarusian, Polish, and English. The language choice is «sticky»: once you set it, a stray message in another language won't knock it off. The base itself stays in Russian, and the model translates each answer into the chosen language at answer time.
Personal checklists are more of a notebook than part of the base: editing a list never touches the knowledge base itself or the search. Deleting an item that came from a bot answer removes it from your list only.
Where the practice comes from
The normative layer is written by the curator from official sources. The practice layer, on the other hand, is filled semi-automatically — by a separate collector built on LangGraph that turns diaspora chats into anonymised, deduplicated facts.
The numbers so far: around 2.4 million messages have passed through the collector, of which 28,091 were deemed relevant, and from those 17,284 practical facts were extracted. The extraction is done by the gpt-4o-mini model — anonymised facts only, with no names, handles, or quotes.
A key point — the personal-data filter works on a «safety first» principle: first an automatic search for contacts, document numbers, and long quotes, and if needed an additional model check; any «dirty» fact is dropped without being saved. Moving a fact into the base is always a manual step by the curator, never automatic.
Another detail we had to fight for is the topic classifier. It used to send the whole set of facts in a single request to the model, which would hit a timeout and silently fall back to a crude keyword heuristic, dumping useful facts into a «junk» catch-all category. We split the work into small batches — and the share of «junk» dropped from 51% to 23%, with useful-but-misfiled facts down to about 3%.
The historical load is built so it can be interrupted and resumed: each channel has a checkpoint, and a restart doesn't begin from scratch but continues from where it stopped.
How we keep the base up to date
Collecting facts is half the job. Next they have to be checked, edited, and approved. For that the curator has a local control panel — it runs only on their own computer and is not exposed anywhere outside.
The panel has three tabs: «Collection» — control over fact collection, «Review» — viewing, editing, and approving pages, and «Suggestions» — read-only signals from users (edit suggestions, the 👍/👎 rollup, the region waitlist).
The «Collection» tab shows the status of the historical load and the state of each Telegram channel with simple badges — what is already done, what is running, what was interrupted and will resume on the next run.
The «Review» tab is the reviewer's workbench: a page list with filters, a detailed view with sources and review questions, and the approval actions.
The approval process itself looks like this: the reviewer opens a page, optionally runs its review questions through the live bot, may fix the text with the help of AI, and then — in the role of curator or lawyer — approves. Approval marks the page as verified, records who checked it, and adds an entry to the change history.
The most interesting thing here is editing through a dialogue with the AI. Instead of editing the text by hand, the curator simply tells the agent what is needed. The agent works out for itself whether it's a question or an instruction to edit.
The agent touches only the page text — the service fields, sources, and draft status stay untouched. It is forbidden to invent facts: if there is no source, it leaves a note that a source is needed, rather than making something up. After the edit comes a check: did any sections disappear, was the text truncated; if too much is lost — up to two retries, then a rollback with nothing saved.
From the curator's point of view it looks like an ordinary chat with memory — you can add «no, shorter», «also mention this», and undo each edit with a single button.
How we measure quality
Answer quality here is measured, not assumed. There is a set of 109 «golden» questions, and each new page adds at least one. A separate judge model runs this set and checks two things: did the answer convey the substance of the expected facts (by meaning, not word for word), and does it assert anything forbidden as fact. The check is tuned to be strict on purpose, so that «forbidden» fires only on an explicit assertion, not on a mere mention of the topic.
Current metrics: citation accuracy about 99%, fact coverage about 97%, abstention on out-of-base questions — 100%.
Your own server, your own models
None of this is tied to a single provider. Every model — meaning recognition, generation, speech recognition, and speech synthesis — can be swapped for a local one (for example, Ollama or LM Studio) with a single setting, while the code itself stays the same. For sensitive relocation data, that means the whole stack can run on your own server, without sending users' questions outside.
The whole cycle at a glance
Putting it all together, the life of a single fact looks like this: it is collected from chats, the curator promotes practice or writes a norm (a draft), the reviewer checks and approves it, the changes are committed, and then the search is rebuilt and the review questions are run before a release.
Conclusion
Legalka KB shows that a «smart assistant» and an «honest assistant» are not the same thing. The secret is not the size of the model but the discipline around it: a strict tie to a verified base, an honest «I don't know» instead of guessing, mandatory sources and dates, anonymisation on the way in, and measurable quality on the way out. That is what separates a useful relocation helper from an eloquent generator of fiction.
You can try the bot itself right in Telegram, for free — @legalka_pl_bot. And if you need a similar assistant that answers strictly from your data and doesn't make things up, write to us: development@mi-code.pl.
Which model does what
Finally — a summary of which model handles which task.
| Task | Model |
|---|---|
| Knowledge-base indexing | text-embedding-3-small |
| Retrieval (search) | text-embedding-3-small |
| Follow-up condensation | gpt-4o-mini |
| Answering the user | gpt-4o-mini |
| Voice input (STT) | whisper-1 |
| Voicing answers (TTS) | tts-1 |
| Practice fact extraction | gpt-4o-mini |
| PII check (optional) | gpt-4o-mini |
| Topic classification | gpt-4o-mini |
| Intent classification | gpt-4o-mini |
| KB page revision | gpt-4o-mini |
| Eval judge | gpt-4o-mini |
Frequently asked questions
- What is Legalka KB and is it free to use?
- Legalka KB is a free Telegram bot (@legalka_pl_bot) that helps you make sense of relocating to Poland — residence cards, visas, ZUS, bank accounts, taxes, diploma recognition, family reunification and more. You can try it right in Telegram at no cost.
- How does Legalka KB avoid inventing account numbers, deadlines, or office addresses?
- Unlike a generic chatbot, Legalka KB answers strictly from a verified knowledge base and never draws on knowledge «off the top of its head». When the base has no answer, it honestly says «I don't know» instead of making something up.
- Which languages can I use with Legalka KB?
- The interface and answers are available in five languages — Russian, Ukrainian, Belarusian, Polish, and English. Your language choice is «sticky», so a stray message in another language won't change it; the knowledge base itself stays in Russian and the model translates each answer into your chosen language.
- What is the difference between «norm» and «practice» in the answers?
- «Norm» is what the law requires: every claim carries a link to a source, is marked verified only after a lawyer's review, and is re-checked at least every 60 days. «Practice» is how things work in reality — applicants' observations, always dated and framed as «based on applicants' reports», with the fact considered stale no later than 90 days on.
- Is Legalka KB a substitute for legal advice?
- No — Legalka KB is not legal advice but collected and structured data. Even when it answers confidently and with references, you should still follow the official links it provides and verify for yourself, since rules change and the final word always rests with the office.