Home/Engineering/How we score risk in real time

Engineering

How we score risk in real time

Every card authorisation and every transfer that leaves a Yusr account passes through the risk engine before it clears. We have a hundred milliseconds to decide what to do about it. Here is what happens in that window.

30 January 2026 Maha Al-Ghamdi · Staff Engineer, Risk Platform 11 min read

When we launched the app in the first quarter of 2025, fraud review at Yusr was a batch job. Payments cleared, a scoring run went out overnight, and a queue of suspicious activity landed on an analyst's screen the next morning. That is a perfectly respectable way to run anti-money-laundering surveillance. It is a terrible way to stop card fraud, because by the time anyone looks at the case the money is gone and the work has turned into recovery.

So over the second half of 2025 we moved the whole thing onto the payment path. The risk engine is now a synchronous call inside authorisation. The authorisation service will not return a decision until the risk engine has answered or the deadline expires, which means the engine has become the sort of service that wakes people up at night — and had to be built accordingly.

This post covers the feature pipeline that feeds it, the model itself, how we hold the latency budget, what the five score bands do, and how we watch the model for the slow rot that eventually gets every fraud model.

1.9m

Payment events scored on an average day

74 ms

p99 end to end, against a 100 ms budget

340

Features in the production model

Monthly

Retrain cadence, first Tuesday

Stage one

The feature pipeline

Nothing interesting gets computed on the payment path. By the time a payment arrives, almost everything the model needs is already sitting in memory, waiting to be looked up by key.

Ingest

Card authorisations, SARIE and instant transfers, logins, device registrations, profile edits and app events all land on one partitioned event log, keyed by customer reference. One ordered stream per customer, so a consumer never has to reason about two events arriving out of sequence.

Aggregate

A stream consumer maintains rolling windows off that log — counts, sums, distinct-value cardinalities and ratios over one hour, twenty-four hours, seven days, thirty days and ninety days. Windows are updated incrementally, not recomputed, so the cost of an event is constant regardless of how much history sits behind it.

Serve

Aggregates are written into an in-memory feature store in the primary Riyadh region, sitting on the same network as the authorisation service. A scoring call fetches one row per entity — customer, device, counterparty, merchant — and assembles the vector locally. Median fetch is under nine milliseconds.

Same pipeline, two consumers

Fraud detection and anti-money-laundering monitoring read from the same feature store. They ask different questions of it — fraud cares about whether this payment looks like this customer, AML cares about structuring, layering and unexplained changes in counterparty mix — but duplicating the aggregation layer for each would have meant two sets of windows drifting apart and two on-call rotations. The AML models run off the payment path, on a schedule, and write into the same case queue. Sanctions and watchlist screening is a separate control and is not part of this engine.

What the model actually looks at

The production model carries 340 features. They fall into six families, and in terms of contribution to the score, the first and the last do most of the work.

Family	Examples	Freshness
Transaction velocity	Count and value of payments in the last hour, day and week against the customer's own thirty-day baseline. Time since the previous accepted payment. Ratio of this amount to the customer's median and 95th-percentile amount. Repeated near-identical amounts in a short window.	Streaming, sub-second
Merchant category	Merchant category code, whether this customer has used that code before, how common the code is in the customer's own history, and the population-level fraud rate for the code over the last ninety days. Card-present versus card-not-present. Cross-border acceptance.	Streaming, plus nightly rates
Geolocation	Country and city of acceptance, distance from the last accepted event divided by the elapsed time between them, and whether the coarse location reported by the app agrees with where the card was presented.	Per event
Device fingerprint	A stable device reference derived from hardware and operating-system attributes, how long that device has been associated with the account, how many accounts the same device reference has appeared on, whether the OS reports the device as rooted or jailbroken, and whether an emulator is suspected.	Per event, cached per device
Behavioural signals	Keystroke cadence on the amount and passcode screens — the inter-key intervals, not the keys. The accelerometer and gyroscope tilt profile while a payment is being confirmed, which is a decent proxy for how the customer holds the phone. Scroll and tap rhythm through the payment flow. Each is compared against a per-customer profile built over the previous ninety days.	Per session, profile nightly
Counterparty history	Whether this beneficiary or merchant has been paid before from this account, the age of the relationship, and how many other Yusr accounts have paid the same counterparty in the last seven days.	Streaming, sub-second

The behavioural family was the last one we added, in October 2025, and it was the one we were most sceptical about. It earned its place: on its own it is weak, but it is close to uncorrelated with the rest of the vector, which makes it unusually valuable to a tree ensemble. Adding the five behavioural features moved recall on confirmed account-takeover cases by just over eleven points without moving the hold rate at all.

None of these features are hand-tuned rules. We still run a thin layer of deterministic rules in front of the model — a handful of hard blocks that exist for regulatory reasons and do not need a probability attached to them — but the rules layer has shrunk from seventy-one rules at launch to nine.

Stage two

A boosted tree ensemble, retrained every month

The scorer is a gradient-boosted decision tree ensemble: roughly six hundred trees, maximum depth six, trained on around fourteen months of labelled payment events. We tried a neural model twice. Both times it was marginally better on the offline metric and materially worse to live with — slower to train, harder to explain to an analyst holding a case, and much harder to debug when a feature went stale. Boosted trees handle the mix of categorical and heavily skewed numeric features we have without much persuasion, and they give us per-event feature attributions cheaply enough to attach to every case.

Labels come from three places: confirmed fraud reported by customers, chargebacks that come back through the scheme, and analyst dispositions on closed cases. Chargebacks are the awkward one because they arrive late, so a training window only matures after forty-five days. Anything more recent than that goes into evaluation, never into training, which is a rule we have broken exactly once and paid for.

Retraining runs on the first Tuesday of the month. The candidate is evaluated against the current champion on a held-out month, then deployed in shadow — scored on live traffic, decisions discarded — for seven days. We compare the shadow score distribution against the champion's before we promote anything. A candidate that beats the champion on offline recall but shifts more than two percent of traffic between bands in shadow does not get promoted until somebody understands why.

Class imbalance

Confirmed fraud is roughly one event in eleven thousand. We train on a downsampled negative set and then recalibrate the output back onto the true prior with isotonic regression, so the number the engine emits is an actual probability rather than a rank. This matters more than it sounds: the band thresholds are set on that calibrated probability, so if calibration drifts, every band boundary moves at once without anyone changing a threshold. We monitor calibration error as a first-class metric for that reason.

Stage three

Holding the hundred milliseconds

The budget is a hundred milliseconds from the authorisation service issuing the call to a decision coming back. That number is not arbitrary. A mada authorisation has its own end-to-end ceiling at the switch, and the risk engine is one hop inside it. Blow the budget and we are not slow, we are declining people's groceries.

Where the time goes on a p99 event: 31 ms fetching features, 9 ms in inference, 14 ms in the rules layer and decision routing, and the rest in serialisation and network. Inference is nowhere near the expensive part, which surprises people. The model is compiled to native code at deploy time and runs in-process; the feature fetch is the thing we spend our engineering effort on.

The call itself is deliberately boring. The authorisation service sends the event and a set of entity references, and the engine returns a score, a band, an action and a case reference if one was raised.

POST /internal/risk/v1/score
Content-Type: application/json
X-Deadline-Ms: 100

{
  "event_id":    "evt_01HN7QK4P2X9",
  "event_type":  "card_auth",
  "customer_ref":"cus_8f21ab",
  "device_ref":  "dev_5b1e9c",
  "amount_sar":  4750.00,
  "currency":    "SAR",
  "mcc":         "5944",
  "acceptance":  { "country": "TR", "city": "Istanbul", "cnp": false },
  "counterparty_ref": "mer_2c70d1",
  "captured_at": "2026-01-27T19:04:11.882+03:00"
}

200 OK

{
  "event_id":      "evt_01HN7QK4P2X9",
  "score":         93,
  "band":          5,
  "action":        "HOLD",
  "case_id":       "case_20260127_00418",
  "top_features":  [
    "geo.impossible_travel_kmh",
    "device.accounts_seen_7d",
    "amount.ratio_to_p95_30d",
    "mcc.first_use_for_customer"
  ],
  "model_version": "risk-txn-2026.01",
  "scored_in_ms":  38
}

If the engine misses the deadline, the authorisation service does not wait. It applies a fail-open default for low-value domestic card-present payments and a fail-closed default above a value threshold, and every timeout is scored asynchronously afterwards so the case still gets raised. Timeouts run at about four in every hundred thousand calls, almost all of them during a feature-store failover.

The output

Five bands, and only one of them touches the payment

The calibrated probability is mapped onto a score from 0 to 100 and then into one of five bands. The band, not the score, is what the rest of the bank consumes. Everything downstream — the case queue, the analyst tooling, the daily reporting — keys off the band, so changing a threshold changes behaviour in one place.

Band	Score	The payment	What happens next	Typical day
1	0–24	Proceeds	Scored, logged, no case. The score is retained so the customer's baseline keeps moving.	~1.79m (94.4%)
2	25–49	Proceeds	No case. Feeds the overnight pattern review, where clusters of band-2 events across accounts are looked at in aggregate.	~104,000 (5.50%)
3	50–74	Proceeds	A case is raised on the standard queue. An analyst reviews it, usually within the working day. The customer is not contacted unless the review finds something.	~1,400 (0.075%)
4	75–89	Proceeds	A case is raised on the priority queue with a two-hour target. Same as band 3 in every respect except urgency — the money still moves.	~570 (0.030%)
5	90–100	Held automatically	The authorisation is declined at the switch and the payment is placed on hold. A case is raised on the priority queue and an analyst reviews it before the hold is released or the payment is cancelled.	~190 (0.010%)

Volumes are a December 2025 daily average across card authorisations and outbound transfers. Percentages are rounded.

Why only band 5 blocks

Raising a case and stopping a payment are two different actions, and for most of 2025 we had them welded together. That was the wrong design. A case is cheap — it costs an analyst a few minutes and the customer never knows it happened. A hold is expensive, because it happens to somebody standing at a till. Splitting them let us push bands 3 and 4 much wider than we could ever have pushed a blocking threshold, which is why the case queue grew fivefold while the hold rate fell.

Band 5 is set where it is because above a score of 90 the model has been right often enough, for long enough, that letting the payment through has a worse expected outcome than stopping it. That threshold is reviewed by the risk team every quarter and moved by them, not by us.

The routing, in code

switch band {
case 1, 2:
    // nothing to do; the score is
    // written to the event log only
    return Proceed(ev)

case 3, 4:
    // case only. the payment is
    // not touched at any point.
    cases.Raise(ev, band, attribution,
        queueFor(band))
    return Proceed(ev)

case 5:
    // the payment stops here, and a
    // person decides what happens
    // to it next.
    hold := ledger.Hold(ev)
    cases.Raise(ev, band, attribution,
        QueuePriority, hold.Ref)
    notify.Customer(ev, TmplHoldPlaced)
    return Decline(ev, "risk_hold")
}

What a hold looks like from both ends

From the customer's side: the payment is declined, and a push notification arrives within a second or two saying we have stopped something and are looking at it. Confirming it was them takes one tap in the app, and that confirmation goes straight onto the case. If they do not answer the notification, the contact centre calls.

From the analyst's side: the case lands on the priority queue with the event, the score, the four highest-contributing features, the customer's recent history, the device history and the customer's own response if one has come in. The analyst releases the hold, cancels the payment, or escalates. Median time to a decision on a band-5 case was 6 minutes 40 seconds in December, and the 90th percentile was 24 minutes. The overnight shift is thinner, and it shows in the tail.

Whatever the analyst decides is written back as a disposition, and dispositions are one of the three label sources feeding the next month's retrain. A released hold is not a neutral outcome in the training data. It is a negative label with a fairly loud voice, because it is a case where the model was confident and wrong.

Stage four

Watching the model, not just the service

Service monitoring tells you the engine answered. It tells you nothing about whether the answer was any good. Those need separate dashboards and separate alerts, and we learned that the expensive way.

Feature drift

Population stability index computed weekly per feature against the training distribution. Anything over 0.20 opens a ticket automatically. Most of what it catches is not the world changing — it is an upstream field quietly turning null after somebody else's deploy.

Score distribution

Hourly band mix against a fourteen-day trailing baseline, with alerts on band 5 in either direction. A sudden drop in holds is as alarming as a spike; it usually means a feature is stale and the model has gone quietly blind.

Calibration

Expected calibration error on the matured label window, tracked per band. Because thresholds sit on a calibrated probability, calibration drift silently moves every boundary at once.

False positives

The metric that matters most to us is the band-5 release rate: of the payments we held, how many did an analyst release as legitimate. At launch it was 41%. Four out of ten people we interrupted had done nothing wrong. In December it was 22.6%, and every point of that came from three places: the behavioural family, splitting cases from holds so we stopped using the hold as a blunt instrument, and per-merchant-category thresholds for a small number of categories where the population fraud rate is so high that a global threshold made no sense.

Recall over the same period went the other way and stayed there: 78.4% of confirmed fraud by value was scored at band 4 or above. We report both numbers together internally, every week, because either one on its own can be made to look excellent by ruining the other.

22.6%

Band-5 holds released as legitimate, down from 41% at launch

78.4%

Confirmed fraud by value caught at band 4 or above

6m 40s

Median analyst decision time on a band-5 case

0.004%

Scoring calls that miss the deadline

Asked internally, often

Four things other teams ask us

Because the complaints and the losses are not the same population. Pushing the threshold from 90 to 95 would cut holds by roughly a third and cut caught fraud by value by about nineteen percent, and the nineteen percent is concentrated in exactly the high-value account-takeover cases the engine exists for. The threshold is a business decision with a real cost on both sides, which is why the risk team owns it and reviews it quarterly rather than an engineer moving it in a config file.

They could, and early on they did. There is now a suppression window: once a hold is placed on an account, further band-5 events on the same account and counterparty within thirty minutes attach to the open case instead of creating a new hold. The payments still stop, but the customer gets one notification and the analyst gets one case with several events on it.

The feature store is the constraint, not the model. The model artefact is small and deployed everywhere; the rolling aggregates are large and replicated asynchronously, so immediately after a failover some windows can be a few seconds behind. The engine detects staleness per feature row and, if a row is beyond its freshness tolerance, scores without that family and marks the response degraded. Degraded scores are capped at band 4, so a stale feature store cannot cause a hold.

It sees a stable counterparty reference and the category code. The merchant descriptor string is in the event but is not a model input — descriptors are inconsistent enough between acquirers that the model was learning acquirer formatting rather than anything about the merchant. It is still shown to the analyst on the case, because a human reads it correctly and the model did not.

What we are working on in 2026

Three things, in the order we expect to ship them.

Counterparty graph features

Mule networks are visible in the shape of the payment graph long before they are visible in any single account's behaviour. Computing them inside the latency budget is the hard part.

Weekly retraining

Monthly is fine for drift and slow for adversaries. Moving to weekly means the shadow window has to shrink, which means the promotion checks have to get much better than they are.

Attribution in the app

Telling a customer why a payment was stopped, in a sentence a person would actually write, rather than the generic hold message we send today.