e-volv

Agent Ops playbook

LLMCostRegressionWatch

Weekly comparison of execution cost against the trailing baseline, flagging agents whose cost-per-run rose and attributing the cause.

Once a week, reads this workspace's agent execution analytics — tokens in, tokens out, and cost per agent, per playbook, and per workspace — and compares the current week against the trailing four-week baseline. Flags any agent whose cost-per-run rose materially, attributes the rise to prompt length, model changes, tool iterations, retrieved context, or retry rate, and flags runs that consumed budget without producing output. Filing the weekly report is handled by a downstream ticket node.

Identifier
llm-cost-regression-watch
Version
1.0.0
Steps
3
Triggers
1
  • llm
  • cost
  • regression
  • analytics
  • weekly
  • budget

When it runs

Scheduleschedule.weekly
Schedule
Every Monday at 07:00 UTC 0 7 * * 1
Timezone
UTC

The pipeline

The graph below is the one the workflow opens with in the builder — same steps, same layout, drawn on the same canvas. The run playing through it is a simulation; the branches and conditions are real.

React Flow
  1. 01
    Weekly Schedule (Monday 07:00 UTC)trigger

    The event that starts the run.

  2. 02
    Analyse LLM Cost Regressionagent.run

    One agent works the task with its toolkit.

  3. 03
    File Cost Regression Ticketticket.create

    Opens a ticket on the connected tracker.

The agent

LLM Cost Watch Agent

Base type
Senior Developer
Temperature
0.2
Max iterations
40
Tools
7

Memory · 2

  • memory_writeWrite Memory · write
  • memory_readRead Memory · read

Terminal · 1

  • run_terminal_cmdRun Terminal Command · write

Filesystem · 1

  • read_fileRead File · read

Code search · 1

  • code_searchCode Search · read

Tickets · 1

  • create_ticketCreate Ticket · write

Status · 1

  • update_statusUpdate Status · write