Agent
ModelsExpand Collapse
AutomateEvent = object { data, event } or object { data, event } or object { data, event } or 30 moreA Server-Sent Event from /v1/automate. Typed discriminated union keyed on event.
A Server-Sent Event from /v1/automate. Typed discriminated union keyed on event.
AIGeneration object { data, event } Envelope for the “ai:generation” event from /v1/automate.
Envelope for the “ai:generation” event from /v1/automate.
data: object { finishReason, iterationId, prompt, 8 more } Event data when AI generation occurs
Event data when AI generation occurs
messages: optional array of object { content, role, providerOptions } or object { content, role, providerOptions } or object { content, role, providerOptions } or object { content, role, providerOptions }
System object { content, role, providerOptions } A system message. It can contain system information.
Note: using the “system” part of the prompt is strongly preferred to increase the resilience against prompt injection attacks, and because not all providers support several system messages.
A system message. It can contain system information.
Note: using the “system” part of the prompt is strongly preferred to increase the resilience against prompt injection attacks, and because not all providers support several system messages.
User object { content, role, providerOptions } A user message. It can contain text or a combination of text and images.
A user message. It can contain text or a combination of text and images.
content: string or array of object { text, type, providerOptions } or object { image, type, mediaType, providerOptions } or object { data, mediaType, type, 2 more } Content of a user message. It can be a string or an array of text and image parts.
Content of a user message. It can be a string or an array of text and image parts.
array of object { text, type, providerOptions } or object { image, type, mediaType, providerOptions } or object { data, mediaType, type, 2 more }
Text object { text, type, providerOptions } Text content part of a prompt. It contains a string of text.
Text content part of a prompt. It contains a string of text.
Image object { image, type, mediaType, providerOptions } Image content part of a prompt. It contains an image.
Image content part of a prompt. It contains an image.
image: string or object { buffer, byteLength, byteOffset, 2 more } or object { byteLength } or object { buffer, byteLength, byteOffset, 2 more } Image data. Can either be:
- data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
- URL: a URL that points to the image
Image data. Can either be:
- data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
- URL: a URL that points to the image
File object { data, mediaType, type, 2 more } File content part of a prompt. It contains a file.
File content part of a prompt. It contains a file.
data: string or object { buffer, byteLength, byteOffset, 2 more } or object { byteLength } or object { buffer, byteLength, byteOffset, 2 more } File data. Can either be:
- data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
- URL: a URL that points to the image
File data. Can either be:
- data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
- URL: a URL that points to the image
Assistant object { content, role, providerOptions } An assistant message. It can contain text, tool calls, or a combination of text and tool calls.
An assistant message. It can contain text, tool calls, or a combination of text and tool calls.
content: string or array of object { text, type, providerOptions } or object { data, mediaType, type, 2 more } or object { text, type, providerOptions } or 3 moreContent of an assistant message. It can be a string or an array of text, image, reasoning, redacted reasoning, and tool call parts.
Content of an assistant message. It can be a string or an array of text, image, reasoning, redacted reasoning, and tool call parts.
array of object { text, type, providerOptions } or object { data, mediaType, type, 2 more } or object { text, type, providerOptions } or 3 more
Text object { text, type, providerOptions } Text content part of a prompt. It contains a string of text.
Text content part of a prompt. It contains a string of text.
File object { data, mediaType, type, 2 more } File content part of a prompt. It contains a file.
File content part of a prompt. It contains a file.
data: string or object { buffer, byteLength, byteOffset, 2 more } or object { byteLength } or object { buffer, byteLength, byteOffset, 2 more } File data. Can either be:
- data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
- URL: a URL that points to the image
File data. Can either be:
- data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
- URL: a URL that points to the image
Reasoning object { text, type, providerOptions } Reasoning content part of a prompt. It contains a reasoning.
Reasoning content part of a prompt. It contains a reasoning.
ToolCall object { input, toolCallId, toolName, 3 more } Tool call content part of a prompt. It contains a tool call (usually generated by the AI model).
Tool call content part of a prompt. It contains a tool call (usually generated by the AI model).
Arguments of the tool call. This is a JSON-serializable object that matches the tool’s input schema.
ToolResult object { output, toolCallId, toolName, 2 more } Tool result content part of a prompt. It contains the result of the tool call with the matching ID.
Tool result content part of a prompt. It contains the result of the tool call with the matching ID.
output: object { type, value, providerOptions } or object { type, value, providerOptions } or object { type, providerOptions, reason } or 3 moreResult of the tool call. This is a JSON-serializable object.
Result of the tool call. This is a JSON-serializable object.
Text object { type, value, providerOptions }
Json object { type, value, providerOptions }
value: string or number or boolean or 2 moreA JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.
A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.
ExecutionDenied object { type, providerOptions, reason }
ErrorText object { type, value, providerOptions }
ErrorJson object { type, value, providerOptions }
value: string or number or boolean or 2 moreA JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.
A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.
Content object { type, value }
value: array of object { text, type, providerOptions } or object { data, mediaType, type } or object { data, mediaType, type, 2 more } or 6 more
Text object { text, type, providerOptions }
FileData object { data, mediaType, type, 2 more }
FileURL object { type, url, providerOptions }
FileID object { fileId, type, providerOptions }
fileId: string or map[string]ID of the file.
If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.
ID of the file.
If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.
ImageData object { data, mediaType, type, providerOptions }
ImageURL object { type, url, providerOptions }
ImageFileID object { fileId, type, providerOptions }
fileId: string or map[string]Image that is referenced using a provider file id.
If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.
Image that is referenced using a provider file id.
If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.
Tool object { content, role, providerOptions } A tool message. It contains the result of one or more tool calls.
A tool message. It contains the result of one or more tool calls.
content: array of object { output, toolCallId, toolName, 2 more } or object { approvalId, approved, type, 2 more } Content of a tool message. It is an array of tool result parts.
Content of a tool message. It is an array of tool result parts.
ToolResult object { output, toolCallId, toolName, 2 more } Tool result content part of a prompt. It contains the result of the tool call with the matching ID.
Tool result content part of a prompt. It contains the result of the tool call with the matching ID.
output: object { type, value, providerOptions } or object { type, value, providerOptions } or object { type, providerOptions, reason } or 3 moreResult of the tool call. This is a JSON-serializable object.
Result of the tool call. This is a JSON-serializable object.
Text object { type, value, providerOptions }
Json object { type, value, providerOptions }
value: string or number or boolean or 2 moreA JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.
A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.
ExecutionDenied object { type, providerOptions, reason }
ErrorText object { type, value, providerOptions }
ErrorJson object { type, value, providerOptions }
value: string or number or boolean or 2 moreA JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.
A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.
Content object { type, value }
value: array of object { text, type, providerOptions } or object { data, mediaType, type } or object { data, mediaType, type, 2 more } or 6 more
Text object { text, type, providerOptions }
FileData object { data, mediaType, type, 2 more }
FileURL object { type, url, providerOptions }
FileID object { fileId, type, providerOptions }
fileId: string or map[string]ID of the file.
If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.
ID of the file.
If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.
ImageData object { data, mediaType, type, providerOptions }
ImageURL object { type, url, providerOptions }
ImageFileID object { fileId, type, providerOptions }
fileId: string or map[string]Image that is referenced using a provider file id.
If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.
Image that is referenced using a provider file id.
If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.
AIGenerationError object { data, event } Envelope for the “ai:generation:error” event from /v1/automate.
Envelope for the “ai:generation:error” event from /v1/automate.
BrowserActionCompleted object { data, event } Envelope for the “browser:action_completed” event from /v1/automate.
Envelope for the “browser:action_completed” event from /v1/automate.
BrowserActionStarted object { data, event } Envelope for the “browser:action_started” event from /v1/automate.
Envelope for the “browser:action_started” event from /v1/automate.
BrowserNavigated object { data, event } Envelope for the “browser:navigated” event from /v1/automate.
Envelope for the “browser:navigated” event from /v1/automate.
BrowserReconnected object { data, event } Envelope for the “browser:reconnected” event from /v1/automate.
Envelope for the “browser:reconnected” event from /v1/automate.
BrowserScreenshotCaptured object { data, event } Envelope for the “browser:screenshot_captured” event from /v1/automate.
Envelope for the “browser:screenshot_captured” event from /v1/automate.
BrowserScreenshotCapturedImage object { data, event } Envelope for the “browser:screenshot_captured_image” event from /v1/automate.
Envelope for the “browser:screenshot_captured_image” event from /v1/automate.
CdpEndpointConnected object { data, event } Envelope for the “cdp:endpoint_connected” event from /v1/automate.
Envelope for the “cdp:endpoint_connected” event from /v1/automate.
CdpEndpointCycle object { data, event } Envelope for the “cdp:endpoint_cycle” event from /v1/automate.
Envelope for the “cdp:endpoint_cycle” event from /v1/automate.
Complete object { data, event } Envelope for the “complete” event from /v1/automate.
Envelope for the “complete” event from /v1/automate.
data: object { finalAnswer, stats, success, error } Payload for the complete stream event. Structurally identical to TaskExecutionResult from webAgent.ts — the complete event’s data is the agent’s final TaskExecutionResult, stringified onto the SSE stream.
Payload for the complete stream event. Structurally identical to TaskExecutionResult from webAgent.ts — the complete event’s data is the agent’s final TaskExecutionResult, stringified onto the SSE stream.
Error object { data, event } Envelope for the “error” event from /v1/automate.
Envelope for the “error” event from /v1/automate.
data: object { error, success } Payload for the top-level error stream event. Emitted when an uncaught error escapes the task runner. Mirrors ErrorResponse from the server package’s taskRunner.ts — kept structurally aligned so schema and runtime stay consistent. Distinct from agent-level error events like ai:generation:error and task:validation_error, which are emitted through the normal event emitter during the agent loop.
Payload for the top-level error stream event. Emitted when an uncaught error escapes the task runner. Mirrors ErrorResponse from the server package’s taskRunner.ts — kept structurally aligned so schema and runtime stay consistent. Distinct from agent-level error events like ai:generation:error and task:validation_error, which are emitted through the normal event emitter during the agent loop.
InteractiveFormDataError object { data, event } Envelope for the “interactive:form_data:error” event from /v1/automate.
Envelope for the “interactive:form_data:error” event from /v1/automate.
data: object { fieldErrors, fields, formDescription, 5 more } Event data when form validation fails and the agent re-requests data. Carries both the error context and the fields that need new values. Callers respond to this the same way as a request event.
Event data when form validation fails and the agent re-requests data. Carries both the error context and the fields that need new values. Callers respond to this the same way as a request event.
InteractiveFormDataRequest object { data, event } Envelope for the “interactive:form_data:request” event from /v1/automate.
Envelope for the “interactive:form_data:request” event from /v1/automate.
SystemDebugCompression object { data, event } Envelope for the “system:debug_compression” event from /v1/automate.
Envelope for the “system:debug_compression” event from /v1/automate.
SystemDebugMessage object { data, event } Envelope for the “system:debug_message” event from /v1/automate.
Envelope for the “system:debug_message” event from /v1/automate.
TaskMetricsIncremental object { data, event } Envelope for the “task:metrics_incremental” event from /v1/automate.
Envelope for the “task:metrics_incremental” event from /v1/automate.
TaskSetup object { data, event } Envelope for the “task:setup” event from /v1/automate.
Envelope for the “task:setup” event from /v1/automate.
TaskTraceContext object { data, event } Envelope for the “task:trace_context” event from /v1/automate.
Envelope for the “task:trace_context” event from /v1/automate.
TaskValidated object { data, event } Envelope for the “task:validated” event from /v1/automate.
Envelope for the “task:validated” event from /v1/automate.
ResearchEvent = object { data, event } or object { data, event } or object { data, event } or 20 moreA Server-Sent Event from /v1/research. Typed discriminated union keyed on event.
A Server-Sent Event from /v1/research. Typed discriminated union keyed on event.
AnalyzingEnd object { data, event } Envelope for the “analyzing:end” event from /v1/research.
Envelope for the “analyzing:end” event from /v1/research.
Complete object { data, event } Envelope for the “complete” event from /v1/research.
Envelope for the “complete” event from /v1/research.
data: object { message, metadata, report, timestamp } complete - Research finished successfully
complete - Research finished successfully
metadata: object { executedQueries, mode, prompt, 11 more } Research metadata
Note: citedPages, gapEvaluations, outline, and judgments are optional to support fast mode, which skips these phases for maximum speed.
Research metadata
Note: citedPages, gapEvaluations, outline, and judgments are optional to support fast mode, which skips these phases for maximum speed.
mode: "fast" or "balanced" or "deep" or 2 moreResearch mode determines depth, thinking budget, and quality controls
Modes (in order of cost/thoroughness):
- fast: Quick answers with minimal validation (~$2, 1 iteration, no judge)
- balanced: Standard research with moderate depth (~$8, 3 iterations, Flash models, no judge)
- deep: Thorough research with judge review (~$15, 5 iterations, Flash models, with judge)
- max: Maximum quality with Pro models (~$40, 5 iterations, Pro models, with judge)
- ultra: Ultimate tier - all Pro models, 10 iterations (expensive, for when accuracy is paramount)
Research mode determines depth, thinking budget, and quality controls
Modes (in order of cost/thoroughness):
- fast: Quick answers with minimal validation (~$2, 1 iteration, no judge)
- balanced: Standard research with moderate depth (~$8, 3 iterations, Flash models, no judge)
- deep: Thorough research with judge review (~$15, 5 iterations, Flash models, with judge)
- max: Maximum quality with Pro models (~$40, 5 iterations, Pro models, with judge)
- ultra: Ultimate tier - all Pro models, 10 iterations (expensive, for when accuracy is paramount)
citedPages: optional array of object { id, claims, sourceQueries, 9 more } Pages cited in the report, ordered by first citation appearance
Pages cited in the report, ordered by first citation appearance
gapEvaluations: optional array of object { gapDescription, questionAssessments, researchCoverage, 3 more }
Based on unanswered/partial questions, what specific information is still needed?
questionAssessments: array of object { findings, question, status } Assessment of each research question’s status and findings
Assessment of each research question’s status and findings
researchCoverage: "Light" or "Moderate" or "Solid" or "Comprehensive"Research coverage level - assesses quality across all questions.
Hierarchy: Light < Moderate < Solid < Comprehensive
- Light: Basic info on some questions, most need more depth → Continue
- Moderate: Multiple questions answered, some remain partial → Continue
- Solid: Most questions well-answered with validated sources → Sufficient to stop
- Comprehensive: All questions thoroughly answered, exceptional depth → Definitely stop
Research coverage level - assesses quality across all questions.
Hierarchy: Light < Moderate < Solid < Comprehensive
- Light: Basic info on some questions, most need more depth → Continue
- Moderate: Multiple questions answered, some remain partial → Continue
- Solid: Most questions well-answered with validated sources → Sufficient to stop
- Comprehensive: All questions thoroughly answered, exceptional depth → Definitely stop
Explicit decision: should research continue with another iteration?
- Considers: how many questions unanswered/partial, coverage for mode, remaining iterations
- Drives query generation: true → generate queries, false → stop researching
New research questions to add (optional, use sparingly)
- Only if original decomposition missed something critical
- Maximum 2-3 new questions total across all iterations
- Most iterations should return empty array or omit this field
Search queries to address identified gaps (only when shouldContinueResearch is true)
- Target unanswered questions first, then partial questions
- 3-10 targeted queries if shouldContinueResearch is true
- Omit or provide empty array if shouldContinueResearch is false
metrics: optional object { cachedFetches, cachedSearches, fetches, 7 more } Complete research metrics
Complete research metrics
Error object { data, event } Envelope for the “error” event from /v1/research.
Envelope for the “error” event from /v1/research.
EvaluatingEnd object { data, event } Envelope for the “evaluating:end” event from /v1/research.
Envelope for the “evaluating:end” event from /v1/research.
FollowingEnd object { data, event } Envelope for the “following:end” event from /v1/research.
Envelope for the “following:end” event from /v1/research.
IterationEnd object { data, event } Envelope for the “iteration:end” event from /v1/research.
Envelope for the “iteration:end” event from /v1/research.
PrefetchingStart object { data, event } Envelope for the “prefetching:start” event from /v1/research.
Envelope for the “prefetching:start” event from /v1/research.