The Boundary With Agentic Engineering
ML Engineering owns the model: training, fine-tuning, evaluation, embeddings, inference, serving, drift, MLOps. Agentic Engineering owns what is built on top: prompting, RAG, tools, memory, agent architecture, agent evals. The line is where the depth lives, not where the LLM is.
The problem, the obvious approach, and why it breaks
Every lesson starts where the work starts: someone has a problem, and the first model that comes to mind looks fine offline.
Where does Machine Learning Engineering end and Agentic Engineering begin, and which of the boundary cases — fine-tuning, prompt-tuning, embedding drift in a RAG system, retrieval evals — falls on which side?
We now have an ML platform team and an AI applications team, both working on "the LLM stuff". Fine-tuning tickets land on the applications team, retrieval-quality incidents land on the platform team, and an embedding drift went unnoticed for a month because each thought the other was watching it.
Anything involving an LLM is "AI" and belongs to whichever team is called AI. The model team does the classical models; the agent team does everything with a prompt in it, including fine-tuning, since it is the same model.
Fine-tuning lands on the applications team, who treat it as a config change; there is no held-out evaluation, no artifact version, no check for catastrophic forgetting, and the fine-tuned model quietly regresses on tasks nobody measured (Fine-Tuning).
- Fine-tuning lands on the applications team, who treat it as a config change; there is no held-out evaluation, no artifact version, no check for catastrophic forgetting, and the fine-tuned model quietly regresses on tasks nobody measured (Fine-Tuning).
- The encoder behind the RAG index is upgraded by the platform team; the query side is re-pointed and the documents are not re-embedded, so queries and documents live in different spaces. Retrieval quality collapses and the agent team's evals blame the prompt (Embedding Drift).
- A retrieval-quality incident is routed to the platform team as "the embedding model is bad". The embeddings are fine; the chunking strategy split every table across two chunks. That is an agentic concern with an agentic fix (Ingestion: Parsing & Chunking).
- Both teams build an eval harness. One measures model behaviour on a benchmark, the other measures agent task completion, and neither can say whether a regression came from the model version or the prompt because the two are versioned separately and deployed together.
What is being predicted, and from what data
This domain leads with these two. A target nobody defined precisely is a label nobody can trust, and a dataset nobody can describe is a model nobody can debug.
- There is no model target here; the target of the boundary is that every concern has exactly one owner and one place where its depth is taught, so that neither domain re-teaches the other and no concern falls between them.
- The decision this lesson supports is routing: given a concern — a degraded RAG answer, a request to adapt a model, a drifting embedding space — which domain's lessons, tools and team does it belong to?
- The shared object is a model — usually a foundation model (Foundation Models) — and its embeddings (Embeddings). ML Engineering produces and serves them; Agentic Engineering consumes them through prompts, retrieval and tool loops (The Agent Loop).
- The data crossing the boundary is an inference API (a model endpoint with a latency, a cost and a version), an embedding space (an encoder version and an index built with it), and an evaluation contract (what the model was validated on, so the application team knows what it was not).
How it actually works
Precisely enough to predict its behaviour — not a framework API.
- The line is drawn by what is being changed and what is being measured. When the *weights* change — pretraining, fine-tuning, parameter-efficient adaptation, distillation, quantization — that is ML Engineering, and it carries the whole loop: a dataset version, a split, a held-out evaluation, an artifact, a registry entry, drift monitoring (Parameter-Efficient Fine-Tuning, Quantization). When the *inputs* change — prompt, retrieved context, tool results, memory — the weights are fixed and the discipline is Agentic Engineering (Context Engineering, RAG Overview).
- Embeddings sit on both sides in a specific way. How they are learned, why the geometry is what it is, and whether the space has moved are ML (Embedding Training, Embedding Drift). How they are used to retrieve, chunk, filter and rerank, and whether the retrieved context helped the answer, are agentic (Vector Storage, Reranking, RAG Evaluation). The interface between the two is an encoder version and an index built with it, and the failure mode is one side changing without the other.
- Evaluation splits the same way. Model evals — held-out metrics, calibration, robustness, regression against the previous artifact — are ML (Model Regression Tests). Agent evals — task completion, trajectory correctness, tool-use accuracy, LLM-as-judge with its own validation — are agentic (Evaluating Agents: Testing Probabilistic Systems, LLM-as-Judge). A degraded agent metric with a stable model eval points at the inputs; a degraded model eval points at the artifact.
- Serving is ML up to the inference API — batching, quantization, GPU cost, latency, fallbacks (Model Serving Architecture, Inference Cost) — and agentic above it: the tool loop, retries, budgets and termination (Budgets, Limits and Termination, Tool Errors, Retries and Timeouts). Security follows the same cut: poisoning and supply chain are ML (Data Poisoning, The Model Supply Chain); prompt injection and tool misuse are agentic (Prompt Injection, Tool Misuse and Data Exfiltration).
Concern → owner → where the depth lives
The matrix is the deliverable. Each row is a concern that has landed on the wrong team somewhere, the domain that owns it, and the lesson in that domain where it is taught in depth. The rule that generates the rows: if the weights change, ML; if the inputs change, agentic; if it is the interface between them, both, and the row says which check each side runs.
The right-hand column is a set of links, deliberately: this lesson cross-links rather than duplicating, because the failure it is about is two teams each re-teaching half of the other's domain badly.
The four boundary cases
Four cases generate most of the mis-routing, and each is decided by the same rule. Fine-tuning an LLM changes weights: ML, with the full loop. Prompt-tuning — few-shot selection, instruction rewriting, soft prompts held outside the model — changes inputs: agentic, with agent evals. Embedding drift in a RAG system is the space moving under a fixed consumer: ML owns the monitor and the re-embed, agentic owns the retrieval eval that notices the symptom. Retrieval-quality evaluation is a measure of what the context did for the answer: agentic, with the encoder as one suspect the ML side can rule in or out.
The decision below is the rule as a question. The reason it works is that each answer names not just a team but a loop: the ML loop with its dataset version, split and artifact, or the agentic loop with its prompt version, trace and task eval.
What is being changed, and what would you measure to know it worked?
when The artifact that answers requests is different afterwards.
cost The full ML loop: dataset version, held-out evaluation with regression against the base, artifact in the registry, drift monitoring after deploy. Slower than a config change, because it is not one.
when The same artifact receives different context.
cost The agentic loop: prompt and config versioning, traces, task-level evals, injection and permission review. No new artifact, but every eval run must record which artifact it ran against.
when Both sides are affected — an index must be rebuilt, or every prompt now hits a different model.
cost A joint deploy with both loops' checks: encoder parity and drift on the ML side, retrieval and task evals on the agentic side. The one case where "whose is it" has the answer "both, at once".
The assumption at the seam
The interface between the two domains is narrow — an endpoint version and an encoder version — and the failure that crosses it is silent on both sides. The ML side sees a healthy model with stable drift metrics; the agentic side sees task evals falling and blames the prompt. Neither dashboard shows that queries and documents are being embedded by different encoders.
So the seam gets its own assumption, its own check, and an owner on each side. Parity is cheap to assert at request time and expensive to discover a month later from a retrieval-quality complaint.
The encoder embedding queries at serving time is the one that built the index they search, and every agent evaluation result names the model version it ran against.
holds when Encoder and index carry a version, the retrieval path asserts they match, and an encoder upgrade is a joint deploy that re-embeds before re-pointing; eval runs tag model and prompt versions.
breaks when A hosted model is upgraded behind a stable API name; the query encoder is bumped for latency without re-indexing; an eval harness records the prompt version only.
respond Roll the query encoder back to the index's version or re-embed, then re-run both loops' evals before either team declares the incident closed.
1type IndexMeta = { encoder: string; builtAt: string }2 3export async function retrieve(query: string, index: IndexMeta, encoder: { version: string; embed: (q: string) => Promise<number[]> }) {4 // ML owns the encoder version; agentic owns this call. The seam is one string.5 if (encoder.version !== index.encoder) {6 // Do not degrade quietly into a different space - that is the month-long incident.7 throw new Error(`encoder ${encoder.version} does not match index built with ${index.encoder}`)8 }9 const q = await encoder.embed(query)10 return searchIndex(q, index)11}The interesting decision is fail-closed. A lenient version would log a warning and search anyway, returning plausible-looking results from the wrong space, and nobody would read the warning until the eval numbers had been wrong for weeks.
How to build it
Most important first.
- Assign every concern to the domain that changes the thing the concern is about — weights to ML, inputs to agentic — and put the matrix below somewhere both teams can point at it.
- Version the interface, not just the sides: the inference API carries a model version; the index carries an encoder version; every agent eval run records both, so a regression can be attributed to the artifact or to the prompt.
- Monitor the embedding space as an ML concern with an agentic consumer: the platform team owns the drift monitor and re-embedding, the agent team owns retrieval-quality evals, and an encoder change is a joint deploy with both checks.
- Route fine-tuning through the ML loop regardless of who asked for it: dataset version, held-out evaluation including regression on the base model's tasks, artifact, registry, rollout. Prompt-tuning and few-shot selection stay in the agentic loop with agent evals.
What to measure
Which number actually maps to the decision — and which numbers look relevant and are not.
- For the boundary itself: the fraction of incidents that were routed to the wrong team first, and the time an incident spends between teams. Both should fall once the matrix exists.
- For the interface: encoder-version parity between the query path and the index; model-version presence on every agent eval run; the correlation between model-eval regressions and agent-eval regressions, which should be high when the model is the cause and near zero when the prompt is.
- Neither domain's metric measures the other: a benchmark score says nothing about task completion, and task completion says nothing about whether the artifact regressed.
What must stay true after deployment
The field this whole domain exists for. A model is a set of assumptions with weights attached; these are the ones a monitor or a test should be checking.
- Every concern involving a model has one owning domain, recorded, and the boundary cases are decided by whether the weights or the inputs are what changes.
- The encoder version used to embed queries at serving time is the one that built the index they are searched against.
- Every agent evaluation run records the model version and the prompt version, so a regression can be attributed to one side.
- A fine-tuned model has passed the ML loop — held-out evaluation with regression against the base — before the agentic team receives it as an endpoint.
- Offline: for each concern in the matrix, name the team, the lesson and the monitor; any row with a blank is an incident waiting to be mis-routed.
- Online: an encoder-parity check in the retrieval path that fails closed when query and index versions differ; a model-version tag on every agent eval result.
- Over time: review the matrix on every reorganisation and every new model integration; a hosted model swap or a new index is a boundary event and gets both teams' checks.
What can go wrong
- The matrix is written and the teams reorganise; the concern that was on the line — embedding drift — is now owned by a team that no longer exists, and the monitor goes stale.
- Fine-tuning is correctly routed to ML, and the ML team evaluates it on the base model's benchmarks, which do not contain the application's task; the agent team's evals catch the regression after deploy.
- The agent team switches to a hosted model version behind the same API name; the ML team's drift monitors were on the self-hosted artifact and see nothing.
- A clean boundary means two teams for one feature and a joint deploy for every encoder change; the alternative is one team that has to hold the whole loop from pretraining data to tool permissions, which does not scale past a few people.
- Routing fine-tuning through the full ML loop is slower than the applications team treating it as a config change; the speed was the reason it kept regressing.
- "It uses an LLM, so it is agentic." Fine-tuning an LLM changes weights and is ML with every ML obligation. Prompt-tuning changes inputs and is agentic. The model is the same; the discipline is decided by what moves.
- "Retrieval is bad, so the embedding model is bad." Retrieval quality is chunking, filtering, reranking and the query formulation as much as the encoder; it is evaluated with agentic evals, and the encoder is one suspect among several.
- "Our agent evals cover the model." They cover the system's behaviour on tasks. A regression in the artifact that happens not to affect those tasks today will affect the next task, and only a model-level regression test would have seen it.
Where this applies
ML advice is stated as universal far more often than it is. These labels say what each claim is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALThe weights-versus-inputs cut applies to any organisation building on foundation models, whether the models are self-hosted or consumed through a hosted API; what changes is how much of the ML side is visible to the team.
- SCALE-SPECIFICA small team holds both domains in the same people and the boundary is a mental one — which lesson to read — rather than an organisational one; the routing failures in this lesson appear once the two sides are separate teams with separate pagers.
- CONTESTEDA serious position holds that the boundary is a historical accident and the two domains should merge: the loop of dataset, evaluation, artifact and monitoring applies just as well to prompts and retrieval configurations as to weights, and separating them creates exactly the hand-off failures this lesson describes. The counter-argument is that the depth on each side — GPU memory and all-reduce on one, tool permissions and injection on the other — is already more than one curriculum holds.
Where the depth lives
This domain teaches the model and hands the rest off by name.