Replay evaluation cases on a candidate

With feedbackCaseId, prepare conditional guidance from that kept correction and test it with the Agent’s newest kept cases, always including the source. Do not combine feedbackCaseId with candidate or caseIds. This does not publish. Requires conversation read access. Voice Agents use a spoken-content rehearsal with an independent model (evaluationMode voice_text), not the live voice model, audio, interruptions or telephony. A person must review its applicability to calls before approval. Otherwise measure a candidate version on its kept evaluation cases before publishing it: the WhatsApp Agent rewrites each case's reply with the candidate's instructions, procedures, integrationProcedures, model and reasoningEffort (omitted ones stay as the live Agent has them), without running any integration tool (each answers with what it answered in the original conversation) and without sending anything; then a judge model compares each new reply with what the team expects. Replays every kept case, or caseIds, at most 50. It calls paid models, so it needs confirmCost: true. Follow it with getEvalRun; with the Agent's settings.evalGate, updateAgent publishes exactly this content with evalRunId. EVAL_CASES_UNAVAILABLE: no kept case to replay. Not an AI tool.

Scope: raiz:agents:write; some inputs also need raiz:conversations:read, raiz:conversations:write
Credentials: an API key, a client-credentials token or a person's access token
Effect: writes
Limits: request body up to 512 KB, response up to 2 MB
CLI: arbol raiz eval-runs create · not available on MCP

POST

https://api.getarbol.com/v1/raiz/eval-runs

Authorization

Authorization
string
required
Bearer <token>, where the token is an API key, a client-credentials token or a person's access token with the scope raiz:agents:write. Get an API key.

Request body JSON required

agentId
string
required
Agent id.

pattern: ^agt_(?:[0-7][0-9a-hjkmnp-tv-z]{25}|(?:[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}|[a-f0-9]{32}))$
confirmCost
boolean
required
A run calls paid models: the candidate's model on every step of each case's reply, then a small judge model per case. Send true to accept that cost.
candidate
optional
object
The candidate version: what changes in what the model reads. Omitted fields keep the live Agent's; integrationProcedures and procedures are complete sets, as in updateAgent.
- instructions
  optional
  string | null
  What the Agent does and how: its role, what it may and may not do, the facts it relies on.
  
  maxLength: 100000 · minLength: 1
- integrationProcedures
  optional
  array of
  object
  Text Agents. Procedures a connection publishes (listConnections, checked live in Account), each saved with its current text and run as a free-form procedure beside procedures. In an update, the complete new set.
  
  connectionId
  string
  required
  A connection's id in Account (con_…).
  
  maxLength: 255 · minLength: 1 · pattern: ^[A-Za-z0-9_-]+$
  procedure
  string
  required
  The procedure's name, as its connection lists it.
  
  maxLength: 64 · minLength: 1 · pattern: ^[a-z0-9]+(-[a-z0-9]+)*$
- model
  optional
  arbolito/agile
  or
  arbolito/balanced
  or
  arbolito/advanced
  or
  arbolito/top
  or
  null
  Text Agents. The tier that writes replies. arbolito/agile: quickest and least expensive, for direct conversations. arbolito/balanced: more deliberate answers at moderate cost. arbolito/advanced: strong reasoning for complex instructions and many tools, with an adjustable reasoningEffort. arbolito/top: the most capable, for the hardest cases, with an adjustable reasoningEffort up to xhigh. Every tier reads images and PDF documents. Null: the default, as arbolito/agile.
- procedures
  optional
  array of
  object
  The Agent's procedures, as a complete set: a procedure left out is removed. Keep each existing procedure's id; a new one may omit it.
  
  name
  string
  required
  A short label for people. The model never reads it.
  
  maxLength: 120 · minLength: 1
  type
  free_form
  or
  structured
  required
  free_form: instructions the Agent adapts. structured: steps that run the same way every time. Fixed once created.
  
  content
  optional
  string
  Free-form procedures: what to do, in Markdown, with inline references such as [tool id="buscar_citas"], [kb id="kdc_…"], [procedure id="prc_…"], [system_tool id="end_conversation"] and {{contact_name}}. Empty for structured procedures.
  
  maxLength: 50000
  id
  optional
  string
  The procedure's id (prc_…), stable for the Agent's life.
  
  pattern: ^prc_[0-7][0-9a-hjkmnp-tv-z]{25}$
  status
  optional
  active
  or
  paused
  A paused procedure runs on no channel.
  
  steps
  optional
  array of
  Structured procedures: the steps, in order. Empty for free-form procedures.
  
  One of the following:
  object
  object
  object
  object
  object
  object
  object
  trigger
  optional
  string
  When it applies, from the contact's side: When the contact asks to book or move an appointment. Empty makes a sub-procedure, which only runs when another procedure references it.
  
  maxLength: 1000
- reasoningEffort
  optional
  none
  or
  low
  or
  medium
  or
  high
  or
  xhigh
  or
  null
  Text Agents whose tier reasons (arbolito/advanced, arbolito/top, or an Agent already running a model that reasons): none answers fastest, high deliberates most, and xhigh, on arbolito/advanced and arbolito/top, deliberates past it. Null: the tier's default. A tier that does not reason refuses it (AGENT_SETTINGS_INVALID).
}
feedbackCaseId
optional
string
Evaluation case id.

pattern: ^evc_(?:[0-7][0-9a-hjkmnp-tv-z]{25}|(?:[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}|[a-f0-9]{32}))$
caseIds
optional
array of
string
The Agent's kept cases to replay. Omitted: its 50 newest kept cases.

pattern: ^evc_(?:[0-7][0-9a-hjkmnp-tv-z]{25}|(?:[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}|[a-f0-9]{32}))$

Returns

200

Successful operation.

JSON

success
boolean
required
data
object
required
- agentId
  string
  required
  Agent id.
  
  pattern: ^agt_(?:[0-7][0-9a-hjkmnp-tv-z]{25}|(?:[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}|[a-f0-9]{32}))$
- evaluationMode
  text
  or
  voice_text
  required
  voice_text tests spoken response content with an independent model; it does not test the live voice model, audio, interruptions or telephony. Human review is required before approval.
- baseVersion
  integer | null
  required
  minimum: 1
- feedbackCaseVersion
  integer | null
  required
  minimum: 1
- baseVersionId
  string | null
  required
  The Agent version that was live when the run was created (agv_…).
- caseIds
  array of
  string
  required
  pattern: ^evc_(?:[0-7][0-9a-hjkmnp-tv-z]{25}|(?:[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}|[a-f0-9]{32}))$
- feedbackCaseId
  string | null
  required
  Evaluation case id.
  
  pattern: ^evc_(?:[0-7][0-9a-hjkmnp-tv-z]{25}|(?:[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}|[a-f0-9]{32}))$
- proposal
  object | null
  required
  appliesWhen
  string
  required
  maxLength: 600 · minLength: 1
  guidance
  string
  required
  maxLength: 1200 · minLength: 1
  exceptions
  string
  required
  maxLength: 600 · minLength: 1
  replaceText
  optional
  string | null
  maxLength: 2000 · minLength: 1
- proposedInstructions
  string | null
  required
  For a feedback proposal, the exact candidate instructions to review and send to updateAgent with evalRunId and expectedVersion. No live Agent has changed.
- contentSha256
  string
  required
  SHA-256 of the candidate's content (instructions, procedures, connection procedures, model, reasoning effort): what settings.evalGate compares with an update's.
  
  pattern: ^[a-f0-9]{64}$
- createdAt
  string
  required
  An instant in RFC 3339 format, in UTC.
  
  format: date-time
- endedAt
  string | null
  required
  An instant in RFC 3339 format, in UTC.
  
  format: date-time
- errorCode
  string | null
  required
  Why a failed run stopped: agent_not_found, or integrations_unavailable (Account could not describe the Agent's tools for 15 minutes).
- errors
  integer
  required
  Cases that could not be replayed or judged.
  
  minimum: 0
- failed
  integer
  required
  minimum: 0
- id
  string
  required
  Evaluation run id.
  
  pattern: ^evr_(?:[0-7][0-9a-hjkmnp-tv-z]{25}|(?:[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}|[a-f0-9]{32}))$
- passed
  integer
  required
  minimum: 0
- results
  array of
  object
  required
  caseId
  string
  required
  Evaluation case id.
  
  pattern: ^evc_(?:[0-7][0-9a-hjkmnp-tv-z]{25}|(?:[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}|[a-f0-9]{32}))$
  createdAt
  string
  required
  An instant in RFC 3339 format, in UTC.
  
  format: date-time
  errorCode
  string | null
  required
  empty_reply: the candidate wrote nothing, so the contact would have read the fallback. case_turn_missing: the case's turn or conversation is gone. model_error, timeout, replay_failed: the replay or the judge failed.
  
  escalated
  boolean
  required
  The candidate escalated the conversation to the team.
  
  handedOff
  boolean
  required
  The candidate handed the conversation to a person.
  
  reasons
  string | null
  required
  The judge's reasons, without personal data.
  
  reply
  string | null
  required
  The candidate's reply as the contact would have read it, with the options it offered in brackets. Personal data like any message: it is erased with its case.
  
  toolCalls
  array of
  string
  required
  The tools the candidate called, in order, by name. None ran: each answered with what it answered in the original conversation.
  
  usage
  object | null
  required
  Tokens and model calls the replay and the judge used.
  
  verdict
  passed
  or
  failed
  or
  error
  required
  passed: the judge found the new reply does what the team expects. failed: it does not, or the candidate wrote no reply. error: the case could not be replayed or judged (errorCode).
- status
  queued
  or
  running
  or
  completed
  or
  failed
  required
  queued until the channel evaluator takes it (within a minute), running while cases are replayed (each result appears as it is judged), completed once every case has a result, failed when it could not continue.
- updatedAt
  string
  required
  An instant in RFC 3339 format, in UTC.
  
  format: date-time
}
error
null
required
metadata
object
required
- requestId
  string
  required
  Request correlation identifier, also sent as X-Request-Id.
}

400

INVALID_INPUT: The request does not match the published schema.

JSON

error
required
One of the following:
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
}

401

UNAUTHORIZED: A valid access token is required.

JSON

error
required
One of the following:
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
}

403

FORBIDDEN: The token does not authorize this operation.

JSON

error
required
One of the following:
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
}

404

NOT_FOUND: The requested operation does not exist.
RESOURCE_NOT_FOUND: The requested resource does not exist in this organization.

JSON

error
required
One of the following:
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
}

405

HTTP_METHOD_NOT_ALLOWED: The HTTP method is not supported for this resource.

JSON

error
required
One of the following:
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
}

409

EVAL_CASES_UNAVAILABLE: The Agent has no kept evaluation case to replay, or a named case is not one of its kept cases.
INVALID_STATE: The resource is not in a state that allows this operation.

JSON

error
required
One of the following:
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
}

413

REQUEST_TOO_LARGE: The request exceeds the allowed size.

JSON

error
required
One of the following:
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
}

415

UNSUPPORTED_MEDIA_TYPE: The request content type is not supported.

JSON

error
required
One of the following:
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
}

422

AGENT_PROCEDURES_INVALID: A procedure is incomplete, references a tool, document or procedure the Agent does not have, or breaks the structured step rules. issues says where.
AGENT_SETTINGS_INVALID: The settings do not apply to this Agent's modality, or two settings contradict each other.
INTEGRATION_PROCEDURE_UNAVAILABLE: A procedure does not exist in this organization's connections, Raíz cannot run its connection, or the Agent is not a text Agent.

JSON

error
required
One of the following:
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
}

429

RATE_LIMITED: The request rate limit was exceeded. Try again later.

JSON

error
required
One of the following:
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
}

500

INTERNAL_ERROR: The service could not complete the operation.

JSON

error
required
One of the following:
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
}

502

OUTCOME_UNKNOWN: The operation may have been applied; check its result before repeating it.

JSON

error
required
One of the following:
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
}

503

UPSTREAM_UNAVAILABLE: The upstream service could not complete the request.
FEEDBACK_PROPOSAL_UNAVAILABLE: A usable improvement could not be prepared. The Agent has not changed. Try again later.
INTEGRATIONS_UNAVAILABLE: The organization's integrations cannot be read right now. Try again shortly.

JSON

error
required
One of the following:
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
}

504

UPSTREAM_TIMEOUT: The upstream service did not respond before its deadline.

JSON

error
required
One of the following:
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
object
}