Skip to content
DOCS

How a prompt is scored

Scoring is deterministic. The same prompt always produces the same number, with no model call, so a score can be explained to the person who received it and reproduced months later.

Four dimensions

CONTEXTWEIGHT 30%

Does the agent know where to look?

  • File paths, and a bonus for more than one
  • Line anchors such as src/auth/session.ts:88
  • Symbol names — functions, classes, types
  • Branch or commit reference
  • A test that reproduces the problem
  • Pasted error output
SPECIFICWEIGHT 28%

Is the request unambiguous?

  • An imperative verb rather than a description of a feeling
  • Numeric constraints — rates, budgets, limits, timeouts
  • A done-condition: what must be true when it is finished
  • Length that is neither one line nor a wall of text
SCOPEWEIGHT 22%

Is the blast radius bounded?

  • One task rather than several stapled together
  • A stated file or directory budget
  • An explicit list of what not to touch
ITERATIONWEIGHT 20%

Is this a fresh attempt or a blind retry?

  • Penalty for retries that add no new information
  • Penalty for changing the target mid-task
  • Penalty for reverting work already accepted
  • Credit for corrections that add a constraint

Bands

The four numbers combine on their weights into a 0–100 score, which is shown as a band so nobody argues about a single point:

  • Pass — 80 and above. The agent had what it needed.
  • Watch — 60 to 79. Workable, but something was missing.
  • Risk — below 60. This prompt is likely to cost a loop.

Task scores

A task is a run of consecutive turns in one session on one repository and branch. A gap of more than thirty minutes, or a switch of repository, starts a new one. The task score is the mean of its prompts, excluding bare approvals like “ship it” — those are not attempts, and averaging them in would flatter a session. More than 4 asks on one task marks it a rework loop.

EXAMPLE
turn 1  "fix the rate limiter"                       score 22   CONTEXT 0
turn 2  "no, the other one"                           score 18   ITERATION 30
turn 3  "src/mw/rate_limit.go, token bucket,
         100 rps burst 20, tests must pass"           score 91   CONTEXT 92

task    3 asks · rework loop: no · score 44

First-try wins

A first-try win is a task finished in a single ask. It is the headline number on Pulse because it is the one a better prompt moves immediately.