Your model reports 94% accuracy. Somebody is about to make a decision worth real money on the strength of that number, and nobody in the building can currently tell you whether it is true. That is rarely because anyone was careless. The ways a model can look better than it is are specialised and unintuitive, and they are almost never checked.
Usually the other half of the question goes unchecked as well: what it takes to run the model reliably, who notices when it degrades, and what the monthly bill looks like.
We check both, independently, on a model you already have, including one built by somebody else. You get a ranked list of everything inflating your confidence, with the evidence for each, a plan for putting the model into production, and the harness to keep watching after we leave.
Two ways in
Validation & Deployment Scan
€4,950 excl. VAT. Fixed price, typically two to four weeks. The scan covers one model and one evaluation dataset, or just its predictions. We run the standard checks for evaluation leakage and calibration, compare against a simple baseline, and look at up to three slices. You also get a production-readiness checklist covering packaging, hosting, monitoring gaps and the obvious cost and security problems. The report ends with a clear verdict: go, fix first, or rethink.
Validation & Deployment Review
€24,500 excl. VAT. Fixed price, typically eight to twelve weeks. The review covers one model and its pipeline, with the full battery below and a deployment plan written for your setup. The calibration and monitoring harness is handed over to run in your CI, and one retest is included within three months.
Timelines start once the model, the data and the access are in place. If the evaluation data turns out not to support a verdict, that is itself the finding, and we write it up along with what is missing so the project does not stall.
What we look for
Calibration
When the model says 90%, is it right nine times out of ten? We measure reliability curves and calibration error on held-out data. Most deployed models are miscalibrated and have never been tested for it, which makes every confidence threshold downstream arbitrary.
Evaluation leakage
Typical cases are duplicate specimens split across train and test, images from one sample landing in both folds, and a time-ordered process evaluated with a random split. Leakage is endemic in scientific ML, and it is the single most common reason a number does not survive contact with production.
Shortcut learning
Is the model reading the physics, or is it reading the scanner, the background, or the technician who prepared the batch? We check attributions against what a domain expert says should matter, and we have the domain experts to ask.
Slice performance
Aggregate accuracy hides the failures that cost money, such as the rare defect class, one instrument, one site or one operator. We break performance down where it matters to you.
Distribution shift
Do today's inputs still resemble the training set, and does the system notice when they stop? Undetected degradation is the most common way a good model stops being a good model.
Honest baselines
Was a simple model ever tried? Sometimes logistic regression on four features matches the network, and that is worth knowing before you maintain the network for five years.
The deployment half
A model that survives the checks still has to be run by somebody, usually a team that is already busy. The review tier ends with a written plan for that, specific to your situation instead of a generic architecture diagram:
- Where it runs. Cloud, on-premise or at the edge. The choice follows from where the data is allowed to be and what latency the decision needs, not from what we happen to like deploying.
- How it is served. Batch, API or embedded in an existing system, the throughput it has to reach, and what happens when it is unavailable.
- Monitoring and drift. What is measured in production, what the thresholds are, and who gets told. Drift that nobody watches for only turns into a surprise later.
- Retraining cadence. How often, on what data, and what has to be true before a new version replaces the old one.
- What it costs. A monthly running-cost estimate, so that the operating bill is visible when the decision to deploy is made and is not first discovered on an invoice.
- Security and rollout. The obvious exposure checks, followed by a staged rollout with the risks named and something to fall back to.
The infrastructure behind all of this is our cloud and deployment practice. We write the plan as the people who would otherwise have to implement it.
How it runs
- 1
Scoping call
We agree on one model and the one decision it feeds, what it predicts, what happens when it is wrong, and what you already measure. The call takes half an hour, and it fixes the price.
- 2
Handover
You provide the model or its predictions, the evaluation data, and a description of how the splits were made. Everything can stay on your infrastructure if it has to.
- 3
Analysis
Calibration, leakage, slices, shift, attributions and baselines are run as a standard battery, plus whatever your domain specifically demands.
- 4
Report and readout
Findings are ranked by how much each one inflates trust in the model, each with its evidence and a concrete remedy, and on the review tier the deployment plan is included. We present it to your team instead of emailing a PDF.
- 5
Harness handover
The calibration and monitoring checks are yours to keep and to run in CI, so the next model gets the same scrutiny without us.
What you do with the findings
A review that only tells you what is wrong is half a service. Most findings have a concrete, bounded fix, which we can quote separately or hand to your team with the code:
- Miscalibration → conformal calibration wrapped around the existing model, giving intervals with a guaranteed error rate. No retraining is required.
- Weak slices → targeted data collection and retraining where it pays off, instead of more data everywhere.
- Shortcut learning → a frank conversation about whether the model can be salvaged, and what a defensible version would need.
- Drift blindness → monitoring wired into the deployment, so the next degradation announces itself.
None of these are long programmes either. Each is a bounded, separately quoted piece of work, and you are free to hand the findings to your own team instead.
Not included in the review itself: implementing the fixes, retraining, and carrying out the deployment. Those are projects, priced separately once the review has shown what they need to be, which is also why the review can be fixed price. How your data is handled during the work is your choice at intake; the three levels are set out with the other assessments, and the analysis can run entirely on your infrastructure.
Who this is for
This is for R&D groups with a model heading toward a real decision, and for quality and process teams running inspection or prediction on a line. It also suits anyone whose model is about to be pointed at something regulated, audited or expensive, and anyone who has inherited a model whose original authors have left.
The review is deliberately independent. We are happy to review a model we did not build, including one built by another supplier, and that is usually when it is worth the most.
Where to go next
How we build
The method these checks come out of: foundation models, domain adaptation, fast models and conformal calibration.
AI Trust & Security
The same discipline applied to LLM applications and agents, where the failure mode is adversarial rather than statistical.
All assessments
The full set of fixed-scope reviews, before you build and after you ship.