Skip to main content

Service

Data engineering & MLOps

The substrate everything else stands on: pipelines that fail loudly, retrieval that stays fresh, registries that record provenance, and releases that can be undone.

Model work sits on top of data movement, and most disappointing AI systems are disappointing for data reasons: a source that changed shape, a corpus that stopped refreshing, an entitlement model nobody encoded, a retraining trigger nobody defined.

We build the unglamorous layer (contracts, lineage, freshness, registries, pipelines) and we build it to be operated by your team rather than by us.

What you get

Named artefacts, not a slide pack. Each one is a thing your team can open, run, or hand to an auditor.

ArtefactWhat it contains
Data contractsSchema and semantic expectations enforced as tests, so an upstream change fails in staging rather than degrading a model silently.
Ingestion & lineagePipelines with recorded provenance from source to index, including what was excluded and why.
Retrieval pipelineChunking, embedding, hybrid search, reranking and freshness tracking, with entitlement metadata carried through.
Model & prompt registryVersioned artefacts with digests, tied to deployment records so any output can be traced to what produced it.
CI/CD for modelsPipelines that run the evaluation suite, gate on regression, and support canary and rollback.
Drift monitoringInput, output and behaviour monitoring with defined retraining or review triggers rather than dashboards nobody reads.

How it runs

  1. 01 · 1–2 weeks

    Assess

    Trace the existing flow end to end and find where it silently degrades.

  2. 02 · 3–5 weeks

    Stabilise

    Contracts, tests and alerting on the paths that matter most.

  3. 03 · 6–12 weeks

    Build

    Pipelines, registry, CI/CD and monitoring, in your tooling.

  4. 04 · 2 weeks

    Hand over

    Your team runs a change through the full pipeline including a rollback.

What we use

Chosen per engagement against your constraints. We have no reseller relationships and no incentive to recommend one of these over another.

Pipelines

AirflowDagsterdbtKafkaSpark

Storage & retrieval

PostgrespgvectorOpenSearchQdrantObject storage

Delivery

GitHub ActionsGitLab CIArgo CDTerraform

Monitoring

PrometheusGrafanaOpenTelemetryGreat Expectations

What we do not do

Knowing where our usefulness stops saves everyone a procurement cycle.

  • We do not replace a working warehouse or platform because it is not what we would have chosen. Migration has to earn itself.
  • We do not build dashboards nobody has agreed to act on. A monitor without a defined response is decoration.
  • We do not take ownership of your data platform. We build within it and hand it back.

Questions we are asked

Can you work with our existing stack?

Almost always, and we prefer to. The tools your team already operates have a large advantage over the ones we would pick, because somebody already knows what they do at 03:00.

How do you handle document-level permissions in retrieval?

By pushing the entitlement predicate into the query rather than filtering results afterwards. Post-filtering fails badly when permissions are sparse, and it leaks document existence through citation lists. We measure both paths before committing to one.

What triggers a retraining or review?

Defined thresholds agreed during the build, input distribution shift, output distribution shift, escalation-rate change, and human override rate. The last is usually the earliest signal and the one most often left uninstrumented.

Related

Start with the constraint.

Most of these projects are shaped by what you cannot do rather than what you want. Data that cannot leave the estate, a model you cannot host with a third party, a decision somebody has to justify to a regulator. Tell us yours and we will say honestly whether we can work inside it.