Agent Ops playbook
LLM Cost Regression Watch
Weekly comparison of execution cost against the trailing baseline, flagging agents whose cost-per-run rose and attributing the cause.
Once a week, reads this workspace's agent execution analytics (tokens in, tokens out, and cost per agent, per playbook, and per workspace), and compares the current week against the trailing four-week baseline. Flags any agent whose cost-per-run rose materially, attributes the rise to prompt length, model changes, tool iterations, retrieved context, or retry rate, and flags runs that consumed budget without producing output. Filing the weekly report is handled by a downstream ticket node.
- Identifier
llm-cost-regression-watch- Version
- 1.0.0
- Steps
- 3
- Triggers
- 1
- llm
- cost
- regression
- analytics
- weekly
- budget
When it runs
schedule.weekly- Schedule
- Every Monday at 07:00 UTC
0 7 * * 1 - Timezone
- UTC
The pipeline
The graph below is the one the workflow opens with in the builder, same steps, same layout, drawn on the same canvas. The run playing through it is a simulation; the branches and conditions are real.
- 01Weekly Schedule (Monday 07:00 UTC)
triggerThe event that starts the run.
- 02Analyse LLM Cost Regression
agent.runOne agent works the task with its toolkit.
- 03File Cost Regression Ticket
ticket.createOpens a ticket on the connected tracker.
The agent
LLM Cost Watch Agent
- Base type
- Senior Developer
- Temperature
- 0.2
- Max iterations
- 40
- Tools
- 7
Memory · 2
memory_writeWrite Memory · writememory_readRead Memory · read
Terminal · 1
run_terminal_cmdRun Terminal Command · write
Filesystem · 1
read_fileRead File · read
Code search · 1
code_searchCode Search · read
Tickets · 1
create_ticketCreate Ticket · write
Status · 1
update_statusUpdate Status · write