← PracticeAdvancedStructured

They Pressed Stop and the Bill Kept Rising

Pull up the evidence one item at a time, commit to a diagnosis, and only then see the schedule that actually ran.

What was reported

Our report generator runs a twelve-step pipeline, several steps of which call an LLM provider. Users press Stop on long reports fairly often — the UI clears immediately and the request shows as cancelled. Our provider bill this month is 4.1x what our usage dashboard says users consumed. We also have a support queue full of people saying cancelled reports appeared in their export folder up to twenty minutes later, sometimes with the wrong data.
1app.post('/reports', async (req, res) => {
2  const job = createJob(req.body)
3  res.status(202).json({ jobId: job.id })    // returns immediately
4  runPipeline(job)                            // 12 steps, not awaited, no signal
5})
6 
7app.post('/reports/:id/cancel', async (req, res) => {
8  jobs.markCancelled(req.params.id)           // sets a flag nothing reads
9  res.status(200).end()
10})
11 
12async function runPipeline(job: Job) {
13  for (const step of job.steps) {
14    const out = await step.run(job.ctx)        // ctx carries no cancellation token
15    job.results.push(out)
16  }
17  await writeExport(job)
18}

Evidence

Nothing here is labelled as relevant. Some of it is not.

What is actually happening?