Anonymised under NDA
On-premise compliance AI on open-source LLMs
Financial Services · A financial institution in the GCC
Fully air-gapped retrieval and reasoning system over a financial institution's policy and case corpus. No data leaves the estate; every decision is auditable.
The problem
- Compliance reviewers were buried in document review, with case backlogs growing month over month.
- Hosted LLMs were not viable: data-residency and client-confidentiality rules required workloads to remain in-country and on-premise.
- Earlier proofs-of-concept produced fluent but unverifiable answers, eroding trust.
The architecture
- Air-gapped deployment of quantised open-source models on local GPUs, served via vLLM behind an internal gateway.
- Hybrid retrieval with a domain-tuned re-ranker, returning citations from the canonical policy corpus.
- Human-in-the-loop workflow: every recommendation is reviewed and signed off by a compliance officer before action.
LlamavLLMQdrantPostgreSQLKubernetesOpenTelemetry
The outcome
- Reviewers retained authority over every decision, but spent their time on judgement rather than search.
- The audit log gives the firm a defensible, regulator-ready record of every prompt, retrieval, model, and reviewer.
Services drawn on
A similar problem on your roadmap?
A Discovery Sprint is the fastest path to a scoped, costed proposal.