Intelligent Knowledge Agent

A grounded question-answering assistant for international truck drivers relocating to Germany. It answers only from the EuroJobsCenter Visa & Immigration Handbook, cites the exact PDF page for every claim, and says “not found in the document” instead of guessing.

THE PROBLEM

Drivers recruited from abroad get a 41-page handbook (v3.2, August 2026) covering recruitment fraud, escrow payments, the visa journey, arrival, Code 95, driver cards and changing employers. The questions they ask are short and anxious: “I got my contract yesterday. What should I do now?” A wrong answer about a fee or a job guarantee causes real harm, so a generic chatbot is not acceptable.

WHAT I BUILT

Refactored a research notebook (RAG_V1.ipynb) into a modular, typed FastAPI service with durable ingestion jobs, versioned corpus snapshots and API-key auth.
A chat UI with page-level citations, an inline PDF page viewer, follow-up suggestions, voice input and read-aloud.
A 34-case gold evaluation suite verified against physical PDF pages, plus a judge harness and Grafana dashboards.

Measured Results

98.3%

Required-fact accuracy
judge, 29 answerable cases

99.6%

Faithfulness
claims supported by packed evidence

100%

Citation label validity
30 answers, 0 unknown labels

98.9%

Citation support precision
189 claim–source pairs

0

Forbidden claims
incl. 4 adversarial / injection cases

4.5 s

p50 latency (p95 7.3 s) 34
warm queries, CPU models

How it works: The Hiring Workflow

1 · Analyze resumes

Upload CVs. Text is extracted with PyPDF/docx2txt and an LLM returns a structured profile.

2 · Analyze job

Paste or upload a JD. Requirements are extracted per section.

3 · Match

Pick a job and candidates. The LLM scores 6 sections and the weighted average gives the final score.

4 · Review

Ranked results, a comparison chart, and an AI assessment for each candidate.

Full 34-case run eval_f5ed6c2d…, corpus v2, generator and judge gpt-5-mini (same-model judge, disclosed). Total cost for the run: ≈ $0.04.

Architecture

Ingestion is separate from the request path. Queries only ever see a complete, published corpus snapshot.

INGESTION ( DURABLE WORKER)

Upload / CLI

PDF signature, size and page limits; SHA-256 dedup

Extract

PyMuPDF with physical page provenance

Clean & structure

header/footer removal by position, handbook & FAQ headings

Semantic chunk

BGE similarity breaks, hard token caps, no cross-section overlap

Publish

Chroma + BM25 + SQLite registry, versioned snapshot

QUERY PATH

Rewrite

Follow-ups (“Yes.”) become standalone queries

Hybrid retrieve

Dense k=20 + BM25 k=20, RRF (k=60)

Rerank

MiniLM cross-encoder, tuned score filter

Pack

Whole blocks into a 3,000-token budget; tracks used IDs

Generate or abstain

Source-only prompt; citations validated against the source map

Engineering Decisions That Mattered

Evaluation caught a bad default: The notebook’s reranker cut-off of 0.0 removed all gold evidence for 9 of 29 answerable questions, including “Is my job guaranteed?”. A sweep on the tuning split chose −6.0; packed coverage rose from 67.8% to 98.9%.
Cite only what the model saw: The source map is built from evidence that actually survived token packing. Unknown labels are repaired or rejected, never shown.
Abstain without an LLM call: If no passage passes the filter, the API returns insufficient_evidence with a reason. Provider failures return real errors instead.
Physical pages, not labels: The handbook’s “Page 7” section starts on PDF page 23. Citations use 1-based physical pages and keep section labels separately.
Consistent lifecycle: Content hashes plus an ingestion fingerprint stop duplicate chunks. Restart re-queues interrupted jobs and rebuilds BM25 from the published snapshot.
Secrets out of the code: Hard-coded keys in the notebook were flagged for rotation. The service reads keys from the environment only.

Product Experience

Suggested starter questions for the main topics: next steps, job security, fees, career, Code 95, fraud safety.
A document sidebar with ingestion status, page and chunk counts, and handbook version. Ticking a document limits answers to it.
Conversational follow-ups (“Yes, please”) that continue the visa journey step by step.
Microphone input and read-aloud for drivers who prefer voice.
An admin retrieval-debug toggle and an Evaluations tab.

Walkthrough