Six things financial firms actually run on their data: transaction fraud detection, credit decisioning, anti-money-laundering triage, collections prioritisation, churn, and reconciliation. Every one has a regulator or an auditor attached, which changes what "working" means. For each this sets out the data it needs, how you would prove it worked, and where it fails.
Scroll
The use cases
01
Transaction fraud detection
Score a transaction in flight, and decide whether to let it through.
Transaction attributes, device and behavioural signals and the customer’s own history produce a score inside the time available before authorisation. The decision is never only the model’s: a threshold turns a score into an action, and where that threshold sits is a business judgement about the relative cost of a missed fraud and a blocked customer.
What it needs
Confirmed fraud outcomes joined back to the original transaction. Chargebacks arrive months later, which means today’s labels describe last quarter’s fraud.
Features you can actually compute inside the latency budget. A signal that takes 400ms is not available at authorisation.
A record of what you blocked. Without it the training data contains only fraud that got through, so the model learns from the fraud you were already worst at catching.
A named owner for the threshold, and a process for moving it.
How you would know it worked
Never accuracy. Precision and recall separately, precision-recall AUC, and the two costs stated in money: fraud missed, and good customers declined. A model that improves one while quietly worsening the other has not improved anything.
Where it fails
On the feedback loop. Blocked transactions never produce an outcome, so the model stops learning about the fraud it already catches and drifts toward what it misses. It also fails quietly when a pattern changes: performance degrades for weeks before anyone notices, unless somebody watches the score distribution rather than the headline metric.
02
Credit decisioning
Estimate the probability that this borrower will not repay.
Application data, bureau data and, where permitted, transaction history estimate default risk. The model is one input to a decision that must be explainable to the applicant and defensible to a regulator, which rules out techniques that would otherwise score better.
What it needs
Outcomes observed over a full credit cycle. A model trained only on a benign period has never seen the conditions it exists to price.
Rejected applications, and some way of reasoning about them. You observe repayment only for people you accepted.
A feature set you can defend one attribute at a time as not being a proxy for something you must not price on.
Reason codes. An applicant is entitled to know why, and "the model said so" is not an answer.
How you would know it worked
Against the existing scorecard, on the same applicants, over a window long enough for outcomes to mature. Approval rate and loss rate together — improving either alone is trivial and means nothing.
Where it fails
On reject inference and on proxies. A postcode is not an ethnicity field and can behave like one; a model that has never seen the people you declined cannot say what declining them cost. Both are invisible in a validation score and obvious in a regulatory review.
03
AML alert triage
Rank the alert queue so investigators start where the risk is.
Existing rule-based alerts are scored for likely materiality so a team can work the queue in a sensible order. Note what this is not: it does not replace the rules, and in most jurisdictions it cannot decide that something goes unreviewed.
What it needs
Investigation outcomes recorded consistently — what was escalated, what was closed, and why. Free-text closure notes are the usual blocker.
The full alert history including everything closed as a false positive, which is where most of the signal is.
Compliance in the room from the start, because the binding constraint is regulatory before it is technical.
How you would know it worked
Time to first escalation, and the share of escalations found in the top decile of the ranking. Not "alerts reduced" — that is not the objective and it reads badly in an inspection.
Where it fails
When triage quietly becomes suppression. Ranking an alert lower is defensible; a model that closes one usually is not, and the difference matters enormously to a regulator even when the mathematics is identical.
04
Collections prioritisation
Work the accounts where contact will change the outcome.
Arrears history, payment behaviour and contact history estimate which accounts will self-cure and which need intervention, so limited collections capacity goes where it changes something.
What it needs
Contact history with outcomes, not merely attempts.
A clean definition of cure, and consistency about when the clock starts.
Visibility of which customers are in hardship, because prioritising on recoverability alone produces conduct problems.
How you would know it worked
Roll rates and cure rates against the current strategy on comparable arrears buckets, plus amount recovered per collector hour.
Where it fails
When it optimises purely for recovery. A model that learns to chase hardest the people least able to pay is doing exactly what it was asked and creating a conduct issue, and no validation metric will flag it.
05
Customer churn and retention
Spot the accounts about to leave, early enough to act.
Balance, transaction frequency, product holdings and service contacts predict attrition, so retention effort is spent before the decision is made rather than after it.
What it needs
A definition of churn that fits the product. A dormant current account and a closed one are different things.
Enough lead time for the prediction to be actionable. Knowing on the day of closure is knowing nothing.
Service history, which is frequently in a separate system nobody has joined to the account.
How you would know it worked
A holdout that receives no intervention. Without one, every customer who stays looks like a save, including everybody who was never leaving.
Where it fails
When the retention offer goes to whoever scores highest, including customers who would have stayed anyway. That is not retention spend, it is a discount programme with a model attached.
06
Reconciliation and exception handling
Match what cleared against what you recorded, and surface only the breaks.
Ledger entries, settlement files and bank statements are matched, leaving a much smaller exceptions queue. Mostly matching logic rather than machine learning, and usually the fastest payback on this list.
What it needs
Consistent references across systems, or a documented rule for deriving them.
Timing conventions written down: cut-offs, value dates, and what a weekend does.
A route for an exception to be resolved rather than merely flagged.
How you would know it worked
Auto-match rate, exceptions per thousand transactions, and hours spent reconciling. All three: the first without the third can hide work that simply moved somewhere else.
Where it fails
When tolerances are loosened to raise the auto-match rate. The number improves and real breaks pass through, which is precisely the wrong trade in a control process.
Before any of it works
Every use case above assumes these. Where one is missing, it is the project — not a prerequisite to be waved through in a kick-off meeting.
Fraud, default, cure, churn — each is observed long after the decision it judges. Where the two cannot be joined there are no labels, and none of the models above can be built or evaluated however much transaction data exists.
Blocked transactions and rejected applications produce no outcome, so every model trained on what you accepted is trained on a sample you selected. It cannot be repaired retrospectively, and it is the most consequential data decision on this list.
In a regulated decision the question is not only whether the model performs but whether each input is permissible and each outcome explainable. A feature set nobody can walk through attribute by attribute is a model that will not survive its first review.
A score does nothing until a threshold turns it into an action, and where that sits is a commercial and conduct judgement rather than a technical one. Projects that never name the owner of that number end up with it set by whoever built the model.
Questions people ask
Because fraud is rare. In the public credit-card dataset used across this literature, 0.172% of transactions are fraudulent — about one in 577 — so a model that labels everything legitimate is over 99.8% accurate and catches nothing. The published work uses precision, recall, F1 and precision-recall AUC instead, the last specifically because it describes minority-class performance where ROC-AUC flatters it. If a vendor quotes an accuracy figure for a fraud model, that is the review finished.
Reject inference. You observe repayment only for applicants you accepted, so a model trained on the accepted book has never seen the people you declined and cannot tell you whether declining them was right. It is not a modelling detail but a structural bias in the only data you have, and every honest answer to it is expensive.
Ranking an alert queue is defensible. Closing alerts usually is not, and that distinction matters far more to a regulator than the mathematics behind it. Treat triage as ordering the work rather than reducing it, and measure the share of true escalations found at the top of the ranking rather than the number of alerts avoided.
By keeping a holdout that receives no intervention. Without one, every customer who stays looks like a save, including everybody who was never leaving, and the programme reports a return it did not earn. It is the most common measurement failure in retention work and entirely avoidable.
With reconciliation. It is mostly matching logic rather than machine learning, it pays back in weeks, it forces exactly the data hygiene every other use case here depends on — consistent references, agreed timing conventions — and nobody will ask you to explain it to a regulator.
A Data Health Check runs these questions against your systems rather than your sector: what you have, what it would support today, and what the first useful thing would take.