AI ENGINEERING — Independent project

AI-Powered Peacebuilding Knowledge Assistant (RAG)

Context

In the Central African Republic, information relevant to peacebuilding actors is distributed across NGO reports, United Nations and MINUSCA publications, and documents produced through dialogue processes, including the 2015 Bangui National Forum. The project aims to make this corpus easier to query while preserving the traceability of the information used to generate each answer.

Summary

Multilingual documentary assistant (FR/EN) designed to retrieve information from a complex knowledge corpus and produce source-grounded answers with verifiable citations.

Contributions

RAG & AI Engineering: Designed the RAG pipeline: document ingestion, embeddings, PostgreSQL/pgvector retrieval with HNSW indexing, source-grounded generation with Gemini/OpenAI, uncertainty handling, and structured citations.
Backend & Data: Built an asynchronous FastAPI/Pydantic API with SSE streaming, conversation and model-usage persistence, async SQLAlchemy, PostgreSQL, and Alembic migrations.
Evaluation & Robustness: Implemented retrieval benchmarks (Hit@K, Recall@K, MRR), source faithfulness, citation, completeness/relevance, FR/EN consistency, uncertainty, and adversarial prompt-injection evaluations.
Production Engineering: Built a modular LLM-provider-independent architecture with secure configuration, logging, health/readiness checks, Docker, Ruff, and 611 unit and integration tests validated from a clean clone.

Results

Retrieval: Hit@20: 100% · Recall@20: 100% · MRR@20: 0.798 across 9 answerable benchmark cases.
Source faithfulness: 85.7% claim-support accuracy across 7 cases, with 100% exact match for the identified supporting source.
Completeness & relevance: 94.4% mean expected-concept coverage and 100% relevant answers across 9 evaluation cases.
Citations: 100% citation presence and 100% structurally valid citations on the dedicated benchmark.
FR/EN: Expected-source recall reached 100% in French and 100% in English across the evaluated multilingual cases.
Software quality: 611 automated tests passed, with Ruff, migrations, FastAPI startup, PostgreSQL connectivity, and health/readiness also validated from a clean clone.

Value

Verifiable information: Answers maintain an explicit connection to retrieved documentary passages so users can verify the supporting sources.
Measurable quality: Evaluation separates retrieval, generation, citations, multilingual behavior, uncertainty, and robustness to identify regressions and improvement opportunities precisely.
Evolvable architecture: Decoupling ingestion, embeddings, retrieval, generation, and LLM providers allows individual components to evolve without rewriting the entire system.

Technical stack

Python · FastAPI · Pydantic · PostgreSQL · pgvector · Gemini/OpenAI · Docker · Alembic · pytest · Ruff

System evaluation

Evaluation separates retrieval quality from generation quality. Hit@K, Recall@K, and MRR measure source retrieval, while dedicated evaluations cover faithfulness to retrieved passages, citation validity, completeness and relevance, uncertainty handling, and FR/EN consistency. These benchmarks use small, targeted evaluation sets to detect regressions and validate the architecture; they are not a guarantee of general system performance.

Production engineering

Environment-based configuration, Alembic migrations, failure logging at system boundaries, separate liveness and readiness endpoints, Docker containerization, and clean-clone release validation. The API image was reduced from approximately 1.16 GB to 559 MB by excluding development dependencies and artifacts from the runtime image.

Open full-size image

Multilingual RAG assistant in action: an answer generated from the Central African Republic knowledge base, with the supporting sources displayed to enable information verification.

Open full-size image

Answer completeness and relevance evaluation across 9 cases; a separate benchmark from the 7 cases used to evaluate whether generated claims are supported by retrieved sources.

Open full-size image

End-to-end RAG architecture

The pipeline separates offline ingestion from query-time processing: document acquisition and loading, chunking, embedding generation, and storage in PostgreSQL/pgvector. At runtime, the question is embedded, relevant passages are retrieved, and the resulting context is passed to the model to produce a grounded answer with structured citations. FastAPI exposes the API and SSE streaming, while conversations and usage metrics are persisted in PostgreSQL.

Robustness and controlled responses

A 12-case adversarial evaluation set covers direct and indirect prompt injection, source fabrication, and citation manipulation. Retrieved documentary content is treated as untrusted context, with deterministic tests validating its propagation through the pipeline. When retrieved evidence is insufficient, the system favors an explicit insufficient-information response over unsupported generation.