The situation
Where they started.
Clinicians wanted to ask for cohorts in plain English: patients within a range of adherence, a number of days, a device condition. The existing parser used a hosted frontier model, which meant every question left the provider's environment and cost money per call. The team needed to know whether a private model could match it before committing.
What FlatClaw does
What was built.
- A constrained-intent pipeline: the model emits a fixed schema that is checked against an allowlist of fields before anything runs; never free-form database queries.
- A small open-weight model served on an inexpensive GPU inside the same cloud project as the application, so inference never crosses the boundary.
- A provider seam that lets the application switch between the hosted model and the private one with an environment variable.
- The provider's own benchmark questions replayed against both to settle the question with evidence.
Results
What changed.
- The open model reproduced the hosted model's answers on nine of ten benchmark questions; the tenth was unverifiable in the original logs.
- Every query valid against the schema; no invented fields.
- Inference at a fraction of the hosted model's cost, and the GPU can be paused when idle.
- A dress rehearsal for larger private deployments on the same pattern.