MLOps interviews test the Monday after the notebook

The notebook reports 0.91 AUC and someone asks how you would ship it. “Docker” gets you through one sentence. The next questions are which data snapshot trained the model, how Tuesday’s distribution is measured and how you roll back without serving features from the wrong point in time.

The MLOps track covers that lifecycle in focused sessions. It starts with an overview, then moves into experiment tracking, data versioning, feature stores, model registries, orchestration, testing and CI for ML. Two deployment sessions cover serving shapes and release patterns, followed by monitoring for drift, data quality and alerts. A batch-scoring team will care about a different subset than a low-latency inference platform.

Reproducibility begins before training

A reproducible run points back to its code commit, data snapshot, configuration and random seed. If those details live in filenames and somebody’s memory, the team has a folder of artifacts with poor repeatability. Name the tracker you used, then explain how it connects runs to data and code. That chain of provenance is the subject of the Experiment Tracking interview.

Data versioning deserves equal attention. A failed pipeline can leave a half-written partition or mix late-arriving labels into an earlier snapshot. Talk through atomic writes, table versions, validation and lineage. DVC, lakehouse time travel and object-store manifests are possible implementations. The interview cares about the guarantee you need and the failure you prevent.

Reproducibility has an annoying edge. Some GPU operations are nondeterministic, upstream packages move and external data can be corrected after the fact. Admit where exact replay ends, then describe how you record enough context to investigate a difference. Absolute certainty sounds suspicious in a system with this many moving parts.

Feature stores can leak the future

Feature-store questions turn on point-in-time correctness. Training data must contain the feature values available when each prediction would have been made. A join that uses tomorrow’s customer status for yesterday’s example can make offline performance look wonderful and production look cursed. That leakage is the central failure in the Feature Stores interview.

Explain the difference between offline features used for training and online features fetched during inference. The definitions, transformations and freshness expectations need to agree across both paths. If the online store lags or returns a default, the model receives a distribution it may never have seen during training.

I think feature leakage kills more projects than exotic architecture mistakes. It is unglamorous and often hides behind a clean metric. That makes it good interview material, since a candidate has to trace data through time before reaching for a larger model.

Pipelines fail in partial states

An orchestration answer should cover DAG structure, scheduling, retries and idempotency. A task can exit successfully while producing an empty file. A retry can duplicate records. Two backfills can race, or a downstream step can read a partition while it is still being written.

Describe the contracts between tasks and the checks that decide whether output is safe to publish. Then cover recovery. Which steps rerun, which artifacts are immutable and how do you prevent a partial run from being promoted? A diagram filled with vendor boxes becomes useful once each arrow has a failure condition.

Testing ML code needs several speeds. Transformation functions and schemas can run in ordinary CI. Small-data pipeline tests can verify wiring and invariants. Full training may belong in a scheduled or gated workflow because it is slow and expensive. The point is to catch code and data-contract failures before a long run creates a model that should stay out of production.

Deployment starts with the serving shape

Batch scoring overnight has different constraints from an endpoint with a 50ms budget. Streaming adds ordering, state and freshness concerns. Choose the serving shape from the product requirement, then discuss capacity, feature access and failure behavior. Kubernetes may be part of the implementation, but pod flags do not answer how users receive predictions.

Release patterns control model risk. Shadow deployment lets a candidate model observe traffic without affecting decisions. A canary exposes a limited slice, while an A/B test measures product outcomes under assigned traffic. Explain how you compare versions, which signal stops the release and how routing returns to the prior model.

The registry holds more than a binary. It ties an artifact to lineage, evaluation, approval and deployment status. Define what “production” means when several models sit behind one endpoint or different customer groups receive different versions. Then state who can promote and how that action is audited.

Monitoring needs an owner

Monitoring covers service health, input quality, feature drift, prediction behavior and delayed outcome metrics. Data drift means the input distribution changed. Concept drift means the relationship between inputs and outcomes changed, and the Model Monitoring interview carries both signals into the response decision. Neither label tells you what action to take until you identify the affected segment and the product cost.

Alerts should route to someone who can respond. Set thresholds around an investigation or release guard, and keep diagnostic charts for quieter signals. If every wobble pages the team, the alerts will be muted by Thursday. The first response may be checking a broken upstream feed or rolling back a feature change; immediate retraining is rarely a sensible reflex.

Prepare one path from training to production and annotate every version boundary. Add the commit, data snapshot, registry entry, deployment decision, monitoring signal and rollback target. Then introduce a failure at each boundary and explain what the system does next.

A model that survives the week after the notebook closes has a history and an escape hatch. Practice the relevant pieces in the MLOps track, then bring a system diagram that tells you exactly which model is live, which data built it and which button returns users to the previous version.