Sovereign AI is the most over-marketed and under-engineered category of AI infrastructure in 2026. Every vendor with an on-premise SKU is calling itself sovereign. Few of those architectures actually satisfy the constraints that the CBUAE Sovereign Financial Cloud (SFCSI) imposes when you trace the data flows end-to-end.
This post walks through a reference architecture for CBUAE-aligned RAG that we've shipped on production engagements (and use on Private AI, our own product). It's written for the bank architect, fintech CTO, or compliance lead trying to separate marketing claims from functional sovereignty.
What "sovereign" actually means under SFCSI
The Sovereign Financial Cloud framework, as launched in February 2026, doesn't say "your AI must run on UAE-resident hardware." It says — and this is the framing the supervisory team uses — that regulated data must remain under UAE jurisdiction throughout its processing lifecycle, with controls preventing extraterritorial access.
That last clause is the hard part. "Extraterritorial access" includes:
- A foreign-cloud provider's ability to legally compel disclosure under their home jurisdiction's laws (US CLOUD Act being the canonical example)
- Network paths that traverse foreign infrastructure even briefly
- Logging, telemetry, or observability pipelines that egress to foreign-cloud vendors
- Inference APIs that send the prompt or retrieval context to a foreign endpoint
A typical "sovereign RAG" demo — open-weight model, AWS me-south-1 deployment, OpenAI embedding API — fails on point four. The embedding generation sends the source text to OpenAI's foreign endpoint. The text leaves UAE jurisdiction every time you index a document.
The four data flows that have to stay sovereign
For a RAG system processing CBUAE-regulated data, four data flows must remain within sovereign infrastructure:
- Document ingestion — the corpus that becomes RAG ground truth
- Embedding generation — converting documents and queries to vectors
- Inference (foundation model call) — the LLM that generates the response
- Logging and telemetry — what gets stored about the conversation
Most architectures handle (1) and (3). Many miss (2). Almost all miss (4).
Flow 1: Document ingestion
The simple part. Document storage in UAE-resident infrastructure (AWS me-south-1, on-premise, or Etihad/du datacentre) with at-rest encryption keyed to a sovereign KMS. Standard pattern, well-trodden by banks already operating in the region.
Flow 2: Embedding generation
The most common leak. Embedding models (sentence-transformers, OpenAI ada-002, Cohere embed-v3) convert text to dense vectors. If you call a foreign-cloud embedding API, the text leaves jurisdiction.
The fix: run embedding models on sovereign infrastructure. Open-weight embedding models — intfloat/e5-mistral-7b-instruct, BAAI/bge-multilingual, dunzhang/stella_en_400M_v5 — run on consumer-tier GPUs and produce embeddings competitive with proprietary cloud embeddings on most benchmarks. For Arabic-heavy corpora, Falcon-H1 Arabic paired with a fine-tuned embedding head is the current strong choice.
This is the architectural move that separates real sovereign RAG from marketing sovereign RAG.
Flow 3: Inference
The headline flow. The foundation model that generates the user-facing response sees the user's query and the retrieved RAG context. If that model lives in OpenAI's data centre or Anthropic's, the regulated data has just travelled to California.
The pattern that works: open-weight models (Llama 3.3 70B, Qwen 2.5 72B, Falcon-H1 Arabic) running on sovereign GPU infrastructure. Performance gap with frontier closed models has narrowed dramatically since 2024. Llama 3.3 70B benchmarked on financial reasoning tasks (Hisabi.ai's production telemetry) runs within 8–12% of Claude 3.5 Sonnet on most metrics. For 90% of internal banking workloads, that gap doesn't matter.
For workloads where the gap does matter — typically high-stakes customer-facing or analyst-augmenting tasks — the architecture has to make a routing decision: is this query operating on regulated data, or not?
Flow 4: Logging and telemetry
The flow nobody puts in the architecture diagram and the one auditors increasingly ask about. Conversation logs, eval traces, cost telemetry, observability — all of these typically route through foreign-cloud observability vendors (Datadog, Grafana Cloud, Honeycomb, Braintrust).
The fix: sovereign telemetry pipeline. Self-hosted Grafana / Loki / Tempo on UAE infrastructure, or a managed sovereign observability provider with the right contractual commitments. This is where the "we use Datadog" admission becomes a finding in the audit.
The reference architecture
For a CBUAE-aligned RAG system processing regulated banking data:
[Document corpus, encrypted at rest, UAE jurisdiction]
↓
[Open-weight embedding model on sovereign GPU pool]
↓
[Vector index — pgvector, Qdrant self-hosted, or LanceDB on UAE infra]
↓
[Query → routing layer: regulated data? yes/no]
↓ ↓
[Sovereign inference: Llama/Qwen/Falcon] [Foreign-cloud frontier model]
↓ ↓
[Response → sovereign telemetry pipeline] [Standard cloud telemetry]
↓
[Audit trail in CBUAE-compliant immutable store]
For our Sovereign tier deployments, this typically maps to:
- 2–4× NVIDIA H100 or A100 GPUs (capacity sized to expected concurrency)
- vLLM or Ollama for inference, depending on tenancy model
- pgvector or Qdrant for retrieval
- Self-hosted Grafana stack for telemetry
- Existing CBUAE-compliant key management for encryption
Total hardware cost for a mid-sized bank deployment: AED 280,000 to AED 480,000 capex, plus AED 12,000 to AED 25,000 monthly opex (electricity, cooling, support). Cloud equivalent at the same workload would run AED 35,000 to AED 80,000 per month operationally — cheaper at low scale, more expensive past about 5M tokens per day, and not compliant for in-scope workloads at any volume.
What to insist on in vendor evaluations
If you're evaluating sovereign AI vendors, ask these five questions. Most will fail on at least two.
-
"Where does the embedding model run?" If the answer references any cloud API — OpenAI, Cohere, AWS Bedrock — the architecture is not sovereign at the embedding step.
-
"Where is your telemetry stored?" If the answer is Datadog, Honeycomb, or Grafana Cloud, the regulated data has been leaking through telemetry.
-
"What happens when the H100 cluster is over capacity?" If the answer is "we burst to AWS Bedrock," the sovereign claim is conditional. Acceptable if the routing layer correctly excludes regulated workloads from the burst path; not acceptable otherwise.
-
"Show me the audit trail for a query end-to-end." Vendors who can't produce an audit trace from query receipt through embedding through retrieval through inference through response don't have the observability to satisfy SFCSI evidentiary requirements.
-
"Which model do you use for Arabic?" If the answer is GPT-4o or Claude with translation, the Arabic-language quality is suspect AND the data is leaving jurisdiction during translation. Falcon-H1 Arabic or Jais 2 are the current sovereign-capable options.
Where Codenovai fits
We design and deploy CBUAE-aligned RAG architectures as our Sovereign AI + RAG offer, with three productized tiers (Foundations, Sovereign, Government) at fixed starting prices. We run the same architecture pattern on our own Private AI product and on Hisabi.ai — meaning the cost models, hardware specs, and audit traces we ship to clients are battle-tested, not theoretical.
Book a scoping call or read the full Sovereign AI + RAG offer.