Waqar Uddin

Shared capacity, separate data: can banks share an inference pool?

October 24, 2026 (1w ago)7 views

Disclosure: I work at Jazz as Principal Evangelist Cloud & AI. This article is written as personal industry analysis. Views are my own.

Two competing banks can share one inference pool without either bank's data touching the other. The model is shared. The data is not.

That is an engineering claim before it is a legal one, and it turns on a short list of controls that either exist or do not. It is also the question that decides whether local inference has a market at all, because regulated institutions are the only customers who will pay a premium for capacity inside the country.

I got here from the previous piece, where I worked through what the published DeepSeek inference numbers imply for anyone selling tokens: the 155 trillion tokens a day headline is a cached fleet average, the smallest efficient deployment is 128 GPUs, and a token business breaks even at 37.5 percent utilisation. That piece ended on the question I could not answer with arithmetic. This is the attempt.

Three numbers you can actually plan with

Strip out the headline and the study gives you three usable inputs.

The smallest efficient unit is 128 GPUs, split evenly between prefill and decode. A token business breaks even at 37.5 percent sustained utilisation, and that number does not move when you change the fleet size, because cost and revenue scale together. And one GPU-hour at 5,000 tokens per second is about 18 million tokens, which is the only conversion you need to price anything.

Apply those to real capacity, with the rates treated as inputs rather than measurements. Offshore B200 rents at roughly $1.89 per GPU-hour, which puts a floor near $0.26 per million tokens at 40 percent utilisation. In-country capacity runs an estimated three to five times that hourly rate, so the same tokens land at $0.80 to $1.30.

That is the number to hold on to. It is roughly three to ten times cheaper than the list price of frontier models, which is the commercial case for local inference. It also means nobody in Pakistan wins by reselling tokens at offshore prices from local metal. Either the difference is subsidised or nobody ran the numbers.

The capex problem is a sequencing problem

A 128 GPU unit is two and a half to three million dollars of hardware, or around $210,000 a month rented. No board approves that against a forecast, and token demand from Pakistani enterprises is genuinely unknown because almost nobody is running production workloads yet.

So do not build capacity and then look for demand. Sell contracts first, then build.

The structure that works is borrowed from hosting. Sell committed capacity with a minimum monthly commitment, take or pay, to two or three anchor tenants. That contract covers the capex and the compliance obligation. Then sell the headroom above those commitments as metered tokens.

You are selling the same GPU twice. The anchor pays for the metal. The token product monetises the part of the day the anchor is not using, and above 37.5 percent utilisation that headroom is margin rather than subsidy.

Without the anchor, every token you sell below breakeven is a loss you chose. With it, the loss is bounded and the upside is real.

Can competing banks share the same pool?

Yes, and the sentence that resolves the argument is this: the model is shared, the data is not.

A bank's obligation attaches to its data, not to the silicon. Two banks can use the same weights, the same scheduler and the same rack, and never see each other's prompts, provided the data plane is genuinely isolated. That is a build decision, not a legal one, and it has a specific checklist.

The controls above cover data at rest and in transit. They do not answer the question a security team asks next: the GPU has to decrypt a prompt to compute on it, so what stops the operator, or a neighbouring process, from reading it in memory? On most current hardware the honest answer is a dedicated pod. Where shared capacity is mandatory, the answer is confidential computing, which encrypts GPU memory and gives the tenant a hardware attestation that the provider itself cannot inspect. It exists, it costs performance, and it is limited to specific configurations. That limitation is why the dedicated pod is a product line rather than a consolation prize.

The rules that govern this are risk based, not prohibition based. The State Bank's Framework on Outsourcing to Cloud Service Providers, issued as BPRD Circular No. 01 of 2023, allows any workload to go to a domestic provider and requires prior approval before a bank sends material workloads offshore. It also asks for board approval, control over where data sits, logical separation from other tenants, audit access, concentration limits and an exit you have rehearsed. None of that requires a dedicated building. All of it requires documentation and engineering you can show.

Shared regulated infrastructure already runs Pakistani finance. The payment rails every bank depends on are shared, and no bank argues that sharing a switch means sharing customer data. Inference differs in one way that matters: a switch moves a message and forgets it, while an inference server holds the prompt in memory while it computes. That difference is why the controls above exist. It is an argument about architecture, not a reassurance.

Some banks will still say no, and that is a product

Compliance teams vary. Some will accept a documented multi-tenant platform with exportable audit logs. Others will require dedicated capacity for credit, fraud and anything touching customer identity.

Do not treat that as a lost deal. Sell them a dedicated pod: 8 to 16 GPUs, same platform, same control plane, priced as reserved capacity with a minimum term. Reserve pricing is what makes a small footprint profitable, because the customer pays for the capacity whether they use it or not. You are selling capacity, not tokens, so the utilisation risk sits with the buyer instead of with you.

The multi-tenant pool and the dedicated pod are the same platform at two price points, and the difference between them is who carries that risk.

Where to start

Not with 128 GPUs.

Start with a rented 8 to 16 GPU slice inside the country. That is enough for two replicas of a 70B model at FP8 with headroom, enough to serve two or three design partners, and enough to measure your real cache hit rate. Rent, because the first six months are a measurement exercise and you should not be paying depreciation while you measure.

Commit capital only when signed minimum commitments cover about 60 percent of the capacity you intend to buy. That threshold is my rule of thumb, not a measured constant, but it is the number that turns 37.5 percent from a hope into arithmetic: if most of the fleet is paid for before it is switched on, the utilisation you need to survive is the utilisation you have already sold.

The 8 GPU question deserves a direct answer, because it is the most common mistake. Eight GPUs sold as metered tokens is a loss with extra steps. Eight GPUs sold as a reserved pod to one customer is a product, for the reason above: they carry the utilisation risk. The efficient unit for a token business is 128, and everything below it is a laboratory.

What to sell, in order

Document intelligence, priced per document. Know-your-customer packs, bank statements, salary slips, credit memos. The bank cannot tell you its token volume, but it can tell you how many statements it processes in a month. Price what they can already count. This also removes your demand uncertainty, because document volume is a number that exists in their operations today.

Reserved pods for the workloads that fail the sharing test. Compliance, fraud, credit. Priced per GPU-month with a minimum term.

Metered tokens for everything else, including the bank's own engineering teams. This grows without a sales cycle and fills the troughs in your utilisation curve.

The Urdu and voice layer. Speech to text, text to speech, summarisation on code-switched speech. It is the capability that makes you hard to replace, and the one global providers are least likely to invest in for a market this size. Every bank has a call centre.

The first ninety days

Sign two or three design partners with letters of intent that carry a minimum volume, not a pilot that ends in a slide. Build tenant isolation, key management, audit export and metering before you build model features, because those four are the product. Run fifty real documents through an evaluation set so quality is proven rather than claimed. Measure cache hit rate and cost per document every week.

Then decide on capex, with numbers instead of conviction.

Anybody can rent GPUs. Almost nobody can hand a bank an audit trail, a key it controls, and a deletion certificate. That gap is the business.

If you want to test the utilisation arithmetic against your own numbers, the AI factory calculator has sliders for GPU count, utilisation, throughput, cost and price. The source document behind the DeepSeek figures is public if you want to check my reading of it.