Skip to content
Get started

AI Task

client.agent.automate(AgentAutomateParams { task, data, geo_target, 5 more } body, RequestOptionsoptions?): AutomateEvent | Stream<AutomateEvent>
POST/automate

Execute AI-powered browser automation tasks using natural language with optional geotargeting. This endpoint always streams responses using Server-Sent Events (SSE).

Streaming Response:

  • All responses are streamed using Server-Sent Events (text/event-stream)
  • Real-time progress updates and results as they’re generated

Geotargeting:

  • Optionally specify a country code for geotargeted browsing

Use Cases:

  • Web scraping and data extraction
  • Form filling and interaction
  • Navigation and information gathering
  • Multi-step web workflows
  • Content analysis from web pages
ParametersExpand Collapse
body: AgentAutomateParams { task, data, geo_target, 5 more }
task: string

The task description in natural language

data?: unknown

JSON data to provide context for form filling or complex tasks

geo_target?: GeoTarget

Optional geotargeting parameters for proxy requests

country?: string

Country code using ISO 3166-1 alpha-2 standard (2 letters, e.g., “US”, “GB”, “JP”). See: https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2

guardrails?: string

Safety constraints for execution

interactive?: boolean

Enable interactive mode to allow human-in-the-loop input during task execution

maxIterations?: number

Maximum task iterations

maximum100
minimum1
maxValidationAttempts?: number

Maximum validation attempts

maximum10
minimum1
url?: string

Starting URL for the task

formaturi
ReturnsExpand Collapse
AutomateEvent = V1AutomateEventAgentAction { data, event } | V1AutomateEventAgentExtracted { data, event } | V1AutomateEventAgentProcessing { data, event } | 30 more

A Server-Sent Event from /v1/automate. Typed discriminated union keyed on event.

One of the following:
V1AutomateEventAgentAction { data, event }

Envelope for the “agent:action” event from /v1/automate.

data: Data { action, iterationId, timestamp, 2 more }

Event data for action execution

action: string
iterationId: string
timestamp: number
ref?: string | null
value?: string | null
event: "agent:action"
V1AutomateEventAgentExtracted { data, event }

Envelope for the “agent:extracted” event from /v1/automate.

data: Data { extractedData, iterationId, timestamp }

Event data for extracted data

extractedData: string
iterationId: string
timestamp: number
event: "agent:extracted"
V1AutomateEventAgentProcessing { data, event }

Envelope for the “agent:processing” event from /v1/automate.

data: Data { hasScreenshot, iterationId, operation, timestamp }

Event data for when the agent is waiting for model generation

hasScreenshot: boolean
iterationId: string
operation: string
timestamp: number
event: "agent:processing"
V1AutomateEventAgentReasoned { data, event }

Envelope for the “agent:reasoned” event from /v1/automate.

data: Data { iterationId, reasoning, timestamp }

Event data for agent reasoning

iterationId: string
reasoning: string
timestamp: number
event: "agent:reasoned"
V1AutomateEventAgentStatus { data, event }

Envelope for the “agent:status” event from /v1/automate.

data: Data { iterationId, message, timestamp }

Event data for status messages

iterationId: string
message: string
timestamp: number
event: "agent:status"
V1AutomateEventAgentStep { data, event }

Envelope for the “agent:step” event from /v1/automate.

data: Data { currentIteration, iterationId, timestamp }

Event data for agent step tracking (each loop iteration)

currentIteration: number
iterationId: string
timestamp: number
event: "agent:step"
V1AutomateEventAgentWaiting { data, event }

Envelope for the “agent:waiting” event from /v1/automate.

data: Data { iterationId, seconds, timestamp }

Event data for waiting notifications

iterationId: string
seconds: number
timestamp: number
event: "agent:waiting"
V1AutomateEventAIGeneration { data, event }

Envelope for the “ai:generation” event from /v1/automate.

data: Data { finishReason, iterationId, prompt, 8 more }

Event data when AI generation occurs

finishReason: "stop" | "length" | "content-filter" | 3 more
One of the following:
"stop"
"length"
"content-filter"
"tool-calls"
"error"
"other"
iterationId: string
prompt: string
schema: unknown
timestamp: number
usage: Usage { inputTokens, outputTokens, totalTokens }
inputTokens?: number
outputTokens?: number
totalTokens?: number
messages?: Array<System { content, role, providerOptions } | User { content, role, providerOptions } | Assistant { content, role, providerOptions } | Tool { content, role, providerOptions } >
One of the following:
System { content, role, providerOptions }

A system message. It can contain system information.

Note: using the “system” part of the prompt is strongly preferred to increase the resilience against prompt injection attacks, and because not all providers support several system messages.

content: string
role: "system"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
User { content, role, providerOptions }

A user message. It can contain text or a combination of text and images.

content: string | Array<Text { text, type, providerOptions } | Image { image, type, mediaType, providerOptions } | File { data, mediaType, type, 2 more } >

Content of a user message. It can be a string or an array of text and image parts.

One of the following:
string
Array<Text { text, type, providerOptions } | Image { image, type, mediaType, providerOptions } | File { data, mediaType, type, 2 more } >
Text { text, type, providerOptions }

Text content part of a prompt. It contains a string of text.

text: string

The text content.

type: "text"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Image { image, type, mediaType, providerOptions }

Image content part of a prompt. It contains an image.

image: string | UnionMember1 { buffer, byteLength, byteOffset, 2 more } | ByteLength { byteLength } | V1GlobalBuffer { buffer, byteLength, byteOffset, 2 more }

Image data. Can either be:

  • data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
  • URL: a URL that points to the image
One of the following:
string
UnionMember1 { buffer, byteLength, byteOffset, 2 more }
buffer: Buffer { byteLength }
byteLength: number
byteLength: number
byteOffset: number
BYTES_PER_ELEMENT: number
length: number
ByteLength { byteLength }
byteLength: number
V1GlobalBuffer { buffer, byteLength, byteOffset, 2 more }
buffer: Buffer { byteLength }
byteLength: number
byteLength: number
byteOffset: number
BYTES_PER_ELEMENT: number
length: number
type: "image"
mediaType?: string

Optional IANA media type of the image.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
File { data, mediaType, type, 2 more }

File content part of a prompt. It contains a file.

data: string | UnionMember1 { buffer, byteLength, byteOffset, 2 more } | ByteLength { byteLength } | V1GlobalBuffer { buffer, byteLength, byteOffset, 2 more }

File data. Can either be:

  • data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
  • URL: a URL that points to the image
One of the following:
string
UnionMember1 { buffer, byteLength, byteOffset, 2 more }
buffer: Buffer { byteLength }
byteLength: number
byteLength: number
byteOffset: number
BYTES_PER_ELEMENT: number
length: number
ByteLength { byteLength }
byteLength: number
V1GlobalBuffer { buffer, byteLength, byteOffset, 2 more }
buffer: Buffer { byteLength }
byteLength: number
byteLength: number
byteOffset: number
BYTES_PER_ELEMENT: number
length: number
mediaType: string

IANA media type of the file.

type: "file"
filename?: string

Optional filename of the file.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
role: "user"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Assistant { content, role, providerOptions }

An assistant message. It can contain text, tool calls, or a combination of text and tool calls.

content: string | Array<Text { text, type, providerOptions } | File { data, mediaType, type, 2 more } | Reasoning { text, type, providerOptions } | 3 more>

Content of an assistant message. It can be a string or an array of text, image, reasoning, redacted reasoning, and tool call parts.

One of the following:
string
Array<Text { text, type, providerOptions } | File { data, mediaType, type, 2 more } | Reasoning { text, type, providerOptions } | 3 more>
Text { text, type, providerOptions }

Text content part of a prompt. It contains a string of text.

text: string

The text content.

type: "text"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
File { data, mediaType, type, 2 more }

File content part of a prompt. It contains a file.

data: string | UnionMember1 { buffer, byteLength, byteOffset, 2 more } | ByteLength { byteLength } | V1GlobalBuffer { buffer, byteLength, byteOffset, 2 more }

File data. Can either be:

  • data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
  • URL: a URL that points to the image
One of the following:
string
UnionMember1 { buffer, byteLength, byteOffset, 2 more }
buffer: Buffer { byteLength }
byteLength: number
byteLength: number
byteOffset: number
BYTES_PER_ELEMENT: number
length: number
ByteLength { byteLength }
byteLength: number
V1GlobalBuffer { buffer, byteLength, byteOffset, 2 more }
buffer: Buffer { byteLength }
byteLength: number
byteLength: number
byteOffset: number
BYTES_PER_ELEMENT: number
length: number
mediaType: string

IANA media type of the file.

type: "file"
filename?: string

Optional filename of the file.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Reasoning { text, type, providerOptions }

Reasoning content part of a prompt. It contains a reasoning.

text: string

The reasoning text.

type: "reasoning"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ToolCall { input, toolCallId, toolName, 3 more }

Tool call content part of a prompt. It contains a tool call (usually generated by the AI model).

input: unknown

Arguments of the tool call. This is a JSON-serializable object that matches the tool’s input schema.

toolCallId: string

ID of the tool call. This ID is used to match the tool call with the tool result.

toolName: string

Name of the tool that is being called.

type: "tool-call"
providerExecuted?: boolean

Whether the tool call was executed by the provider.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ToolResult { output, toolCallId, toolName, 2 more }

Tool result content part of a prompt. It contains the result of the tool call with the matching ID.

output: Text { type, value, providerOptions } | Json { type, value, providerOptions } | ExecutionDenied { type, providerOptions, reason } | 3 more

Result of the tool call. This is a JSON-serializable object.

One of the following:
Text { type, value, providerOptions }
type: "text"

Text tool output that should be directly sent to the API.

value: string
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Json { type, value, providerOptions }
type: "json"
value: string | number | boolean | 2 more | null

A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ExecutionDenied { type, providerOptions, reason }
type: "execution-denied"

Type when the user has denied the execution of the tool call.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
reason?: string

Optional reason for the execution denial.

ErrorText { type, value, providerOptions }
type: "error-text"
value: string
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ErrorJson { type, value, providerOptions }
type: "error-json"
value: string | number | boolean | 2 more | null

A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Content { type, value }
type: "content"
value: Array<Text { text, type, providerOptions } | Media { data, mediaType, type } | FileData { data, mediaType, type, 2 more } | 6 more>
One of the following:
Text { text, type, providerOptions }
text: string

Text content.

type: "text"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Media { data, mediaType, type }
data: string
mediaType: string
Deprecatedtype: "media"
Deprecated by the upstream schema.

Deprecated. Use image-data or file-data instead.

FileData { data, mediaType, type, 2 more }
data: string

Base-64 encoded media data.

mediaType: string

IANA media type.

type: "file-data"
filename?: string

Optional filename of the file.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
FileURL { type, url, providerOptions }
type: "file-url"
url: string

URL of the file.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
FileID { fileId, type, providerOptions }
fileId: string | Record<string, string>

ID of the file.

If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.

One of the following:
string
Record<string, string>
type: "file-id"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ImageData { data, mediaType, type, providerOptions }
data: string

Base-64 encoded image data.

mediaType: string

IANA media type.

type: "image-data"

Images that are referenced using base64 encoded data.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ImageURL { type, url, providerOptions }
type: "image-url"

Images that are referenced using a URL.

url: string

URL of the image.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ImageFileID { fileId, type, providerOptions }
fileId: string | Record<string, string>

Image that is referenced using a provider file id.

If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.

One of the following:
string
Record<string, string>
type: "image-file-id"

Images that are referenced using a provider file id.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Custom { type, providerOptions }
type: "custom"

Custom content part. This can be used to implement provider-specific content parts.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
toolCallId: string

ID of the tool call that this result is associated with.

toolName: string

Name of the tool that generated this result.

type: "tool-result"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ToolApprovalRequest { approvalId, toolCallId, type }

Tool approval request prompt part.

approvalId: string

ID of the tool approval.

toolCallId: string

ID of the tool call that the approval request is for.

type: "tool-approval-request"
role: "assistant"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Tool { content, role, providerOptions }

A tool message. It contains the result of one or more tool calls.

content: Array<ToolResult { output, toolCallId, toolName, 2 more } | ToolApprovalResponse { approvalId, approved, type, 2 more } >

Content of a tool message. It is an array of tool result parts.

One of the following:
ToolResult { output, toolCallId, toolName, 2 more }

Tool result content part of a prompt. It contains the result of the tool call with the matching ID.

output: Text { type, value, providerOptions } | Json { type, value, providerOptions } | ExecutionDenied { type, providerOptions, reason } | 3 more

Result of the tool call. This is a JSON-serializable object.

One of the following:
Text { type, value, providerOptions }
type: "text"

Text tool output that should be directly sent to the API.

value: string
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Json { type, value, providerOptions }
type: "json"
value: string | number | boolean | 2 more | null

A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ExecutionDenied { type, providerOptions, reason }
type: "execution-denied"

Type when the user has denied the execution of the tool call.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
reason?: string

Optional reason for the execution denial.

ErrorText { type, value, providerOptions }
type: "error-text"
value: string
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ErrorJson { type, value, providerOptions }
type: "error-json"
value: string | number | boolean | 2 more | null

A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Content { type, value }
type: "content"
value: Array<Text { text, type, providerOptions } | Media { data, mediaType, type } | FileData { data, mediaType, type, 2 more } | 6 more>
One of the following:
Text { text, type, providerOptions }
text: string

Text content.

type: "text"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Media { data, mediaType, type }
data: string
mediaType: string
Deprecatedtype: "media"
Deprecated by the upstream schema.

Deprecated. Use image-data or file-data instead.

FileData { data, mediaType, type, 2 more }
data: string

Base-64 encoded media data.

mediaType: string

IANA media type.

type: "file-data"
filename?: string

Optional filename of the file.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
FileURL { type, url, providerOptions }
type: "file-url"
url: string

URL of the file.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
FileID { fileId, type, providerOptions }
fileId: string | Record<string, string>

ID of the file.

If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.

One of the following:
string
Record<string, string>
type: "file-id"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ImageData { data, mediaType, type, providerOptions }
data: string

Base-64 encoded image data.

mediaType: string

IANA media type.

type: "image-data"

Images that are referenced using base64 encoded data.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ImageURL { type, url, providerOptions }
type: "image-url"

Images that are referenced using a URL.

url: string

URL of the image.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ImageFileID { fileId, type, providerOptions }
fileId: string | Record<string, string>

Image that is referenced using a provider file id.

If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.

One of the following:
string
Record<string, string>
type: "image-file-id"

Images that are referenced using a provider file id.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Custom { type, providerOptions }
type: "custom"

Custom content part. This can be used to implement provider-specific content parts.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
toolCallId: string

ID of the tool call that this result is associated with.

toolName: string

Name of the tool that generated this result.

type: "tool-result"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ToolApprovalResponse { approvalId, approved, type, 2 more }

Tool approval response prompt part.

approvalId: string

ID of the tool approval.

approved: boolean

Flag indicating whether the approval was granted or denied.

type: "tool-approval-response"
providerExecuted?: boolean

Flag indicating whether the tool call is provider-executed. Only provider-executed tool approval responses should be sent to the model.

reason?: string

Optional reason for the approval or denial.

role: "tool"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
object?: unknown
providerMetadata?: Record<string, unknown>
temperature?: number
warnings?: Array<unknown>
event: "ai:generation"
V1AutomateEventAIGenerationError { data, event }

Envelope for the “ai:generation:error” event from /v1/automate.

data: Data { error, iterationId, prompt, 3 more }

Event data when AI generation error occurs

error: string
iterationId: string
prompt: string
schema: unknown
timestamp: number
messages?: Array<unknown>
event: "ai:generation:error"
V1AutomateEventBrowserActionCompleted { data, event }

Envelope for the “browser:action_completed” event from /v1/automate.

data: Data { iterationId, success, timestamp, error }

Event data for action results

iterationId: string
success: boolean
timestamp: number
error?: string
event: "browser:action_completed"
V1AutomateEventBrowserActionStarted { data, event }

Envelope for the “browser:action_started” event from /v1/automate.

data: Data { action, iterationId, timestamp, 2 more }

Event data for action execution

action: string
iterationId: string
timestamp: number
ref?: string | null
value?: string | null
event: "browser:action_started"
V1AutomateEventBrowserNavigated { data, event }

Envelope for the “browser:navigated” event from /v1/automate.

data: Data { iterationId, timestamp, title, url }

Event data when navigating to a page

iterationId: string
timestamp: number
title: string
url: string
event: "browser:navigated"
V1AutomateEventBrowserReconnected { data, event }

Envelope for the “browser:reconnected” event from /v1/automate.

data: Data { endpointIndex, iterationId, startingUrl, 2 more }

Event data when the browser reconnects after a mid-task disconnect

endpointIndex: number

1-based index of the CDP endpoint now in use

iterationId: string
startingUrl: string

The original starting URL the agent is restarting execution from

timestamp: number
total: number

Total number of configured CDP endpoints

event: "browser:reconnected"
V1AutomateEventBrowserScreenshotCaptured { data, event }

Envelope for the “browser:screenshot_captured” event from /v1/automate.

data: Data { format, iterationId, size, timestamp }

Event data for screenshot capture

format: "jpeg" | "png"
One of the following:
"jpeg"
"png"
iterationId: string
size: number
timestamp: number
event: "browser:screenshot_captured"
V1AutomateEventBrowserScreenshotCapturedImage { data, event }

Envelope for the “browser:screenshot_captured_image” event from /v1/automate.

data: Data { image, iterationId, mediaType, timestamp }

Event data for screenshot image capture with full image data This event contains the complete screenshot and can be very large

image: string
iterationId: string
mediaType: "image/jpeg" | "image/png"
One of the following:
"image/jpeg"
"image/png"
timestamp: number
event: "browser:screenshot_captured_image"
V1AutomateEventCdpEndpointConnected { data, event }

Envelope for the “cdp:endpoint_connected” event from /v1/automate.

data: Data { endpointIndex, iterationId, timestamp, total }

Event data when a CDP endpoint is successfully connected to

endpointIndex: number

1-based index of the endpoint that connected

iterationId: string
timestamp: number
total: number

Total number of configured CDP endpoints

event: "cdp:endpoint_connected"
V1AutomateEventCdpEndpointCycle { data, event }

Envelope for the “cdp:endpoint_cycle” event from /v1/automate.

data: Data { attempt, error, iterationId, 2 more }

Event data when a CDP endpoint fails and the next one is being tried

attempt: number

1-based index of the endpoint attempt that failed

error: string

Sanitized error identifier from the failed connection attempt (error.name, not error.message — full messages may contain endpoint URLs)

iterationId: string
timestamp: number
total: number

Total number of configured CDP endpoints

event: "cdp:endpoint_cycle"
V1AutomateEventComplete { data, event }

Envelope for the “complete” event from /v1/automate.

data: Data { finalAnswer, stats, success, error }

Payload for the complete stream event. Structurally identical to TaskExecutionResult from webAgent.ts — the complete event’s data is the agent’s final TaskExecutionResult, stringified onto the SSE stream.

finalAnswer: string | null

Final answer or result from the agent

stats: Stats { actions, durationMs, endTime, 2 more }

Execution statistics

actions: number
durationMs: number
endTime: number
iterations: number
startTime: number
success: boolean

Whether the task completed successfully

error?: Error { code, message }

Structured error information for failed tasks

code: "TASK_ABORTED" | "MAX_ITERATIONS" | "MAX_ERRORS" | "TASK_FAILED"

Error codes for task failures

One of the following:
"TASK_ABORTED"
"MAX_ITERATIONS"
"MAX_ERRORS"
"TASK_FAILED"
message: string

Human-readable error message

event: "complete"
V1AutomateEventDone { data, event }

Envelope for the “done” event from /v1/automate.

data: Record<string, unknown>

Payload for the done stream terminator event. Empty today; reserved for future metadata.

event: "done"
V1AutomateEventError { data, event }

Envelope for the “error” event from /v1/automate.

data: Data { error, success }

Payload for the top-level error stream event. Emitted when an uncaught error escapes the task runner. Mirrors ErrorResponse from the server package’s taskRunner.ts — kept structurally aligned so schema and runtime stay consistent. Distinct from agent-level error events like ai:generation:error and task:validation_error, which are emitted through the normal event emitter during the agent loop.

error: Error { code, message, timestamp }
code: string
message: string
timestamp: string

ISO-8601 timestamp

success: false
event: "error"
V1AutomateEventInteractiveFormDataError { data, event }

Envelope for the “interactive:form_data:error” event from /v1/automate.

data: Data { fieldErrors, fields, formDescription, 5 more }

Event data when form validation fails and the agent re-requests data. Carries both the error context and the fields that need new values. Callers respond to this the same way as a request event.

fieldErrors: Record<string, string>

Per-field error messages from validation (field ref -> error text)

fields: Array<Field>
fieldType: "text" | "email" | "phone" | 8 more

Semantic field type

One of the following:
"text"
"email"
"phone"
"date"
"number"
"select"
"checkbox"
"radio"
"textarea"
"password"
"other"
label: string

The field’s visible label

ref: string

Element ref from the accessibility tree (e.g., “E42”)

required: boolean

Whether this field must be filled

currentValue?: string

Current value if already partially filled

description?: string

Additional context (e.g., validation error message on re-request)

options?: Array<string>

Available options for select/radio fields

formDescription: string
iterationId: string
pageTitle: string
pageUrl: string
requestId: string
timestamp: number
event: "interactive:form_data:error"
V1AutomateEventInteractiveFormDataRequest { data, event }

Envelope for the “interactive:form_data:request” event from /v1/automate.

data: Data { fields, formDescription, iterationId, 4 more }

Event data when the agent requests user data for form fields

fields: Array<Field>
fieldType: "text" | "email" | "phone" | 8 more

Semantic field type

One of the following:
"text"
"email"
"phone"
"date"
"number"
"select"
"checkbox"
"radio"
"textarea"
"password"
"other"
label: string

The field’s visible label

ref: string

Element ref from the accessibility tree (e.g., “E42”)

required: boolean

Whether this field must be filled

currentValue?: string

Current value if already partially filled

description?: string

Additional context (e.g., validation error message on re-request)

options?: Array<string>

Available options for select/radio fields

formDescription: string
iterationId: string
pageTitle: string
pageUrl: string
requestId: string
timestamp: number
event: "interactive:form_data:request"
V1AutomateEventSystemDebugCompression { data, event }

Envelope for the “system:debug_compression” event from /v1/automate.

data: Data { compressedSize, compressionPercent, iterationId, 2 more }

Event data for compression debug info

compressedSize: number
compressionPercent: number
iterationId: string
originalSize: number
timestamp: number
event: "system:debug_compression"
V1AutomateEventSystemDebugMessage { data, event }

Envelope for the “system:debug_message” event from /v1/automate.

data: Data { iterationId, messages, timestamp }

Event data for message debug info

iterationId: string
messages: Array<unknown>
timestamp: number
event: "system:debug_message"
V1AutomateEventTaskAborted { data, event }

Envelope for the “task:aborted” event from /v1/automate.

data: Data { finalAnswer, iterationId, reason, timestamp }

Event data when a task is aborted

finalAnswer: string
iterationId: string
reason: string
timestamp: number
event: "task:aborted"
V1AutomateEventTaskCompleted { data, event }

Envelope for the “task:completed” event from /v1/automate.

data: Data { finalAnswer, iterationId, timestamp, success }

Event data when a task is completed

finalAnswer: string | null
iterationId: string
timestamp: number
success?: boolean
event: "task:completed"
V1AutomateEventTaskMetrics { data, event }

Envelope for the “task:metrics” event from /v1/automate.

data: Data { aiGenerationCount, aiGenerationErrorCount, eventCounts, 5 more }
aiGenerationCount: number
aiGenerationErrorCount: number
eventCounts: Record<string, number>
iterationId: string
stepCount: number
timestamp: number
totalInputTokens: number
totalOutputTokens: number
event: "task:metrics"
V1AutomateEventTaskMetricsIncremental { data, event }

Envelope for the “task:metrics_incremental” event from /v1/automate.

data: Data { aiGenerationCount, aiGenerationErrorCount, eventCounts, 5 more }
aiGenerationCount: number
aiGenerationErrorCount: number
eventCounts: Record<string, number>
iterationId: string
stepCount: number
timestamp: number
totalInputTokens: number
totalOutputTokens: number
event: "task:metrics_incremental"
V1AutomateEventTaskSetup { data, event }

Envelope for the “task:setup” event from /v1/automate.

data: Data { browserName, iterationId, task, 14 more }

Event data when a task is setup

browserName: string
iterationId: string
task: string
timestamp: number
data?: unknown
guardrails?: string
hasApiKey?: boolean
keySource?: "global" | "env" | "not_set"
One of the following:
"global"
"env"
"not_set"
model?: string
provider?: string
proxy?: string
pwCdpEndpoint?: string
pwCdpEndpointCount?: number

Total number of CDP endpoints configured (index, not URLs)

pwCdpEndpoints?: Array<string>
pwEndpoint?: string
url?: string
vision?: boolean
event: "task:setup"
V1AutomateEventTaskStarted { data, event }

Envelope for the “task:started” event from /v1/automate.

data: Data { iterationId, plan, successCriteria, 4 more }

Event data when a task is started

iterationId: string
plan: string
successCriteria: string
task: string
timestamp: number
url: string
actionItems?: Array<string>
event: "task:started"
V1AutomateEventTaskTraceContext { data, event }

Envelope for the “task:trace_context” event from /v1/automate.

data: Data { traceId }

Payload for the task:trace_context event. Carries the OpenTelemetry trace ID for this /v1/automate request so consumers can deep-link to distributed-tracing UIs (e.g. Cloud Trace, Cloud Logging) for the run.

traceId: string

W3C trace ID — 32-character lowercase hexadecimal string.

event: "task:trace_context"
V1AutomateEventTaskValidated { data, event }

Envelope for the “task:validated” event from /v1/automate.

data: Data { completionQuality, finalAnswer, iterationId, 3 more }

Event data for task validation

completionQuality: "failed" | "partial" | "complete" | "excellent"
One of the following:
"failed"
"partial"
"complete"
"excellent"
finalAnswer: string
iterationId: string
observation: string
timestamp: number
feedback?: string
event: "task:validated"
V1AutomateEventTaskValidationError { data, event }

Envelope for the “task:validation_error” event from /v1/automate.

data: Data { errors, iterationId, rawResponse, 2 more }

Event data for validation errors during action response processing

errors: Array<string>
iterationId: string
rawResponse: unknown
retryCount: number
timestamp: number
event: "task:validation_error"
AutomateEvent = V1AutomateEventAgentAction { data, event } | V1AutomateEventAgentExtracted { data, event } | V1AutomateEventAgentProcessing { data, event } | 30 more

A Server-Sent Event from /v1/automate. Typed discriminated union keyed on event.

One of the following:
V1AutomateEventAgentAction { data, event }

Envelope for the “agent:action” event from /v1/automate.

data: Data { action, iterationId, timestamp, 2 more }

Event data for action execution

action: string
iterationId: string
timestamp: number
ref?: string | null
value?: string | null
event: "agent:action"
V1AutomateEventAgentExtracted { data, event }

Envelope for the “agent:extracted” event from /v1/automate.

data: Data { extractedData, iterationId, timestamp }

Event data for extracted data

extractedData: string
iterationId: string
timestamp: number
event: "agent:extracted"
V1AutomateEventAgentProcessing { data, event }

Envelope for the “agent:processing” event from /v1/automate.

data: Data { hasScreenshot, iterationId, operation, timestamp }

Event data for when the agent is waiting for model generation

hasScreenshot: boolean
iterationId: string
operation: string
timestamp: number
event: "agent:processing"
V1AutomateEventAgentReasoned { data, event }

Envelope for the “agent:reasoned” event from /v1/automate.

data: Data { iterationId, reasoning, timestamp }

Event data for agent reasoning

iterationId: string
reasoning: string
timestamp: number
event: "agent:reasoned"
V1AutomateEventAgentStatus { data, event }

Envelope for the “agent:status” event from /v1/automate.

data: Data { iterationId, message, timestamp }

Event data for status messages

iterationId: string
message: string
timestamp: number
event: "agent:status"
V1AutomateEventAgentStep { data, event }

Envelope for the “agent:step” event from /v1/automate.

data: Data { currentIteration, iterationId, timestamp }

Event data for agent step tracking (each loop iteration)

currentIteration: number
iterationId: string
timestamp: number
event: "agent:step"
V1AutomateEventAgentWaiting { data, event }

Envelope for the “agent:waiting” event from /v1/automate.

data: Data { iterationId, seconds, timestamp }

Event data for waiting notifications

iterationId: string
seconds: number
timestamp: number
event: "agent:waiting"
V1AutomateEventAIGeneration { data, event }

Envelope for the “ai:generation” event from /v1/automate.

data: Data { finishReason, iterationId, prompt, 8 more }

Event data when AI generation occurs

finishReason: "stop" | "length" | "content-filter" | 3 more
One of the following:
"stop"
"length"
"content-filter"
"tool-calls"
"error"
"other"
iterationId: string
prompt: string
schema: unknown
timestamp: number
usage: Usage { inputTokens, outputTokens, totalTokens }
inputTokens?: number
outputTokens?: number
totalTokens?: number
messages?: Array<System { content, role, providerOptions } | User { content, role, providerOptions } | Assistant { content, role, providerOptions } | Tool { content, role, providerOptions } >
One of the following:
System { content, role, providerOptions }

A system message. It can contain system information.

Note: using the “system” part of the prompt is strongly preferred to increase the resilience against prompt injection attacks, and because not all providers support several system messages.

content: string
role: "system"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
User { content, role, providerOptions }

A user message. It can contain text or a combination of text and images.

content: string | Array<Text { text, type, providerOptions } | Image { image, type, mediaType, providerOptions } | File { data, mediaType, type, 2 more } >

Content of a user message. It can be a string or an array of text and image parts.

One of the following:
string
Array<Text { text, type, providerOptions } | Image { image, type, mediaType, providerOptions } | File { data, mediaType, type, 2 more } >
Text { text, type, providerOptions }

Text content part of a prompt. It contains a string of text.

text: string

The text content.

type: "text"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Image { image, type, mediaType, providerOptions }

Image content part of a prompt. It contains an image.

image: string | UnionMember1 { buffer, byteLength, byteOffset, 2 more } | ByteLength { byteLength } | V1GlobalBuffer { buffer, byteLength, byteOffset, 2 more }

Image data. Can either be:

  • data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
  • URL: a URL that points to the image
One of the following:
string
UnionMember1 { buffer, byteLength, byteOffset, 2 more }
buffer: Buffer { byteLength }
byteLength: number
byteLength: number
byteOffset: number
BYTES_PER_ELEMENT: number
length: number
ByteLength { byteLength }
byteLength: number
V1GlobalBuffer { buffer, byteLength, byteOffset, 2 more }
buffer: Buffer { byteLength }
byteLength: number
byteLength: number
byteOffset: number
BYTES_PER_ELEMENT: number
length: number
type: "image"
mediaType?: string

Optional IANA media type of the image.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
File { data, mediaType, type, 2 more }

File content part of a prompt. It contains a file.

data: string | UnionMember1 { buffer, byteLength, byteOffset, 2 more } | ByteLength { byteLength } | V1GlobalBuffer { buffer, byteLength, byteOffset, 2 more }

File data. Can either be:

  • data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
  • URL: a URL that points to the image
One of the following:
string
UnionMember1 { buffer, byteLength, byteOffset, 2 more }
buffer: Buffer { byteLength }
byteLength: number
byteLength: number
byteOffset: number
BYTES_PER_ELEMENT: number
length: number
ByteLength { byteLength }
byteLength: number
V1GlobalBuffer { buffer, byteLength, byteOffset, 2 more }
buffer: Buffer { byteLength }
byteLength: number
byteLength: number
byteOffset: number
BYTES_PER_ELEMENT: number
length: number
mediaType: string

IANA media type of the file.

type: "file"
filename?: string

Optional filename of the file.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
role: "user"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Assistant { content, role, providerOptions }

An assistant message. It can contain text, tool calls, or a combination of text and tool calls.

content: string | Array<Text { text, type, providerOptions } | File { data, mediaType, type, 2 more } | Reasoning { text, type, providerOptions } | 3 more>

Content of an assistant message. It can be a string or an array of text, image, reasoning, redacted reasoning, and tool call parts.

One of the following:
string
Array<Text { text, type, providerOptions } | File { data, mediaType, type, 2 more } | Reasoning { text, type, providerOptions } | 3 more>
Text { text, type, providerOptions }

Text content part of a prompt. It contains a string of text.

text: string

The text content.

type: "text"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
File { data, mediaType, type, 2 more }

File content part of a prompt. It contains a file.

data: string | UnionMember1 { buffer, byteLength, byteOffset, 2 more } | ByteLength { byteLength } | V1GlobalBuffer { buffer, byteLength, byteOffset, 2 more }

File data. Can either be:

  • data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
  • URL: a URL that points to the image
One of the following:
string
UnionMember1 { buffer, byteLength, byteOffset, 2 more }
buffer: Buffer { byteLength }
byteLength: number
byteLength: number
byteOffset: number
BYTES_PER_ELEMENT: number
length: number
ByteLength { byteLength }
byteLength: number
V1GlobalBuffer { buffer, byteLength, byteOffset, 2 more }
buffer: Buffer { byteLength }
byteLength: number
byteLength: number
byteOffset: number
BYTES_PER_ELEMENT: number
length: number
mediaType: string

IANA media type of the file.

type: "file"
filename?: string

Optional filename of the file.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Reasoning { text, type, providerOptions }

Reasoning content part of a prompt. It contains a reasoning.

text: string

The reasoning text.

type: "reasoning"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ToolCall { input, toolCallId, toolName, 3 more }

Tool call content part of a prompt. It contains a tool call (usually generated by the AI model).

input: unknown

Arguments of the tool call. This is a JSON-serializable object that matches the tool’s input schema.

toolCallId: string

ID of the tool call. This ID is used to match the tool call with the tool result.

toolName: string

Name of the tool that is being called.

type: "tool-call"
providerExecuted?: boolean

Whether the tool call was executed by the provider.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ToolResult { output, toolCallId, toolName, 2 more }

Tool result content part of a prompt. It contains the result of the tool call with the matching ID.

output: Text { type, value, providerOptions } | Json { type, value, providerOptions } | ExecutionDenied { type, providerOptions, reason } | 3 more

Result of the tool call. This is a JSON-serializable object.

One of the following:
Text { type, value, providerOptions }
type: "text"

Text tool output that should be directly sent to the API.

value: string
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Json { type, value, providerOptions }
type: "json"
value: string | number | boolean | 2 more | null

A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ExecutionDenied { type, providerOptions, reason }
type: "execution-denied"

Type when the user has denied the execution of the tool call.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
reason?: string

Optional reason for the execution denial.

ErrorText { type, value, providerOptions }
type: "error-text"
value: string
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ErrorJson { type, value, providerOptions }
type: "error-json"
value: string | number | boolean | 2 more | null

A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Content { type, value }
type: "content"
value: Array<Text { text, type, providerOptions } | Media { data, mediaType, type } | FileData { data, mediaType, type, 2 more } | 6 more>
One of the following:
Text { text, type, providerOptions }
text: string

Text content.

type: "text"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Media { data, mediaType, type }
data: string
mediaType: string
Deprecatedtype: "media"
Deprecated by the upstream schema.

Deprecated. Use image-data or file-data instead.

FileData { data, mediaType, type, 2 more }
data: string

Base-64 encoded media data.

mediaType: string

IANA media type.

type: "file-data"
filename?: string

Optional filename of the file.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
FileURL { type, url, providerOptions }
type: "file-url"
url: string

URL of the file.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
FileID { fileId, type, providerOptions }
fileId: string | Record<string, string>

ID of the file.

If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.

One of the following:
string
Record<string, string>
type: "file-id"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ImageData { data, mediaType, type, providerOptions }
data: string

Base-64 encoded image data.

mediaType: string

IANA media type.

type: "image-data"

Images that are referenced using base64 encoded data.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ImageURL { type, url, providerOptions }
type: "image-url"

Images that are referenced using a URL.

url: string

URL of the image.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ImageFileID { fileId, type, providerOptions }
fileId: string | Record<string, string>

Image that is referenced using a provider file id.

If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.

One of the following:
string
Record<string, string>
type: "image-file-id"

Images that are referenced using a provider file id.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Custom { type, providerOptions }
type: "custom"

Custom content part. This can be used to implement provider-specific content parts.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
toolCallId: string

ID of the tool call that this result is associated with.

toolName: string

Name of the tool that generated this result.

type: "tool-result"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ToolApprovalRequest { approvalId, toolCallId, type }

Tool approval request prompt part.

approvalId: string

ID of the tool approval.

toolCallId: string

ID of the tool call that the approval request is for.

type: "tool-approval-request"
role: "assistant"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Tool { content, role, providerOptions }

A tool message. It contains the result of one or more tool calls.

content: Array<ToolResult { output, toolCallId, toolName, 2 more } | ToolApprovalResponse { approvalId, approved, type, 2 more } >

Content of a tool message. It is an array of tool result parts.

One of the following:
ToolResult { output, toolCallId, toolName, 2 more }

Tool result content part of a prompt. It contains the result of the tool call with the matching ID.

output: Text { type, value, providerOptions } | Json { type, value, providerOptions } | ExecutionDenied { type, providerOptions, reason } | 3 more

Result of the tool call. This is a JSON-serializable object.

One of the following:
Text { type, value, providerOptions }
type: "text"

Text tool output that should be directly sent to the API.

value: string
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Json { type, value, providerOptions }
type: "json"
value: string | number | boolean | 2 more | null

A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ExecutionDenied { type, providerOptions, reason }
type: "execution-denied"

Type when the user has denied the execution of the tool call.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
reason?: string

Optional reason for the execution denial.

ErrorText { type, value, providerOptions }
type: "error-text"
value: string
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ErrorJson { type, value, providerOptions }
type: "error-json"
value: string | number | boolean | 2 more | null

A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Content { type, value }
type: "content"
value: Array<Text { text, type, providerOptions } | Media { data, mediaType, type } | FileData { data, mediaType, type, 2 more } | 6 more>
One of the following:
Text { text, type, providerOptions }
text: string

Text content.

type: "text"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Media { data, mediaType, type }
data: string
mediaType: string
Deprecatedtype: "media"
Deprecated by the upstream schema.

Deprecated. Use image-data or file-data instead.

FileData { data, mediaType, type, 2 more }
data: string

Base-64 encoded media data.

mediaType: string

IANA media type.

type: "file-data"
filename?: string

Optional filename of the file.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
FileURL { type, url, providerOptions }
type: "file-url"
url: string

URL of the file.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
FileID { fileId, type, providerOptions }
fileId: string | Record<string, string>

ID of the file.

If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.

One of the following:
string
Record<string, string>
type: "file-id"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ImageData { data, mediaType, type, providerOptions }
data: string

Base-64 encoded image data.

mediaType: string

IANA media type.

type: "image-data"

Images that are referenced using base64 encoded data.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ImageURL { type, url, providerOptions }
type: "image-url"

Images that are referenced using a URL.

url: string

URL of the image.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ImageFileID { fileId, type, providerOptions }
fileId: string | Record<string, string>

Image that is referenced using a provider file id.

If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.

One of the following:
string
Record<string, string>
type: "image-file-id"

Images that are referenced using a provider file id.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
Custom { type, providerOptions }
type: "custom"

Custom content part. This can be used to implement provider-specific content parts.

providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Provider-specific options.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
toolCallId: string

ID of the tool call that this result is associated with.

toolName: string

Name of the tool that generated this result.

type: "tool-result"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
ToolApprovalResponse { approvalId, approved, type, 2 more }

Tool approval response prompt part.

approvalId: string

ID of the tool approval.

approved: boolean

Flag indicating whether the approval was granted or denied.

type: "tool-approval-response"
providerExecuted?: boolean

Flag indicating whether the tool call is provider-executed. Only provider-executed tool approval responses should be sent to the model.

reason?: string

Optional reason for the approval or denial.

role: "tool"
providerOptions?: Record<string, Record<string, string | number | boolean | 2 more>>

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:
string
number
boolean
Record<string, string | number | boolean | 2 more>
string
number
boolean
unknown
Array<unknown>
Array<string | number | boolean | 2 more | null>
string
number
boolean
unknown
Array<unknown>
object?: unknown
providerMetadata?: Record<string, unknown>
temperature?: number
warnings?: Array<unknown>
event: "ai:generation"
V1AutomateEventAIGenerationError { data, event }

Envelope for the “ai:generation:error” event from /v1/automate.

data: Data { error, iterationId, prompt, 3 more }

Event data when AI generation error occurs

error: string
iterationId: string
prompt: string
schema: unknown
timestamp: number
messages?: Array<unknown>
event: "ai:generation:error"
V1AutomateEventBrowserActionCompleted { data, event }

Envelope for the “browser:action_completed” event from /v1/automate.

data: Data { iterationId, success, timestamp, error }

Event data for action results

iterationId: string
success: boolean
timestamp: number
error?: string
event: "browser:action_completed"
V1AutomateEventBrowserActionStarted { data, event }

Envelope for the “browser:action_started” event from /v1/automate.

data: Data { action, iterationId, timestamp, 2 more }

Event data for action execution

action: string
iterationId: string
timestamp: number
ref?: string | null
value?: string | null
event: "browser:action_started"
V1AutomateEventBrowserNavigated { data, event }

Envelope for the “browser:navigated” event from /v1/automate.

data: Data { iterationId, timestamp, title, url }

Event data when navigating to a page

iterationId: string
timestamp: number
title: string
url: string
event: "browser:navigated"
V1AutomateEventBrowserReconnected { data, event }

Envelope for the “browser:reconnected” event from /v1/automate.

data: Data { endpointIndex, iterationId, startingUrl, 2 more }

Event data when the browser reconnects after a mid-task disconnect

endpointIndex: number

1-based index of the CDP endpoint now in use

iterationId: string
startingUrl: string

The original starting URL the agent is restarting execution from

timestamp: number
total: number

Total number of configured CDP endpoints

event: "browser:reconnected"
V1AutomateEventBrowserScreenshotCaptured { data, event }

Envelope for the “browser:screenshot_captured” event from /v1/automate.

data: Data { format, iterationId, size, timestamp }

Event data for screenshot capture

format: "jpeg" | "png"
One of the following:
"jpeg"
"png"
iterationId: string
size: number
timestamp: number
event: "browser:screenshot_captured"
V1AutomateEventBrowserScreenshotCapturedImage { data, event }

Envelope for the “browser:screenshot_captured_image” event from /v1/automate.

data: Data { image, iterationId, mediaType, timestamp }

Event data for screenshot image capture with full image data This event contains the complete screenshot and can be very large

image: string
iterationId: string
mediaType: "image/jpeg" | "image/png"
One of the following:
"image/jpeg"
"image/png"
timestamp: number
event: "browser:screenshot_captured_image"
V1AutomateEventCdpEndpointConnected { data, event }

Envelope for the “cdp:endpoint_connected” event from /v1/automate.

data: Data { endpointIndex, iterationId, timestamp, total }

Event data when a CDP endpoint is successfully connected to

endpointIndex: number

1-based index of the endpoint that connected

iterationId: string
timestamp: number
total: number

Total number of configured CDP endpoints

event: "cdp:endpoint_connected"
V1AutomateEventCdpEndpointCycle { data, event }

Envelope for the “cdp:endpoint_cycle” event from /v1/automate.

data: Data { attempt, error, iterationId, 2 more }

Event data when a CDP endpoint fails and the next one is being tried

attempt: number

1-based index of the endpoint attempt that failed

error: string

Sanitized error identifier from the failed connection attempt (error.name, not error.message — full messages may contain endpoint URLs)

iterationId: string
timestamp: number
total: number

Total number of configured CDP endpoints

event: "cdp:endpoint_cycle"
V1AutomateEventComplete { data, event }

Envelope for the “complete” event from /v1/automate.

data: Data { finalAnswer, stats, success, error }

Payload for the complete stream event. Structurally identical to TaskExecutionResult from webAgent.ts — the complete event’s data is the agent’s final TaskExecutionResult, stringified onto the SSE stream.

finalAnswer: string | null

Final answer or result from the agent

stats: Stats { actions, durationMs, endTime, 2 more }

Execution statistics

actions: number
durationMs: number
endTime: number
iterations: number
startTime: number
success: boolean

Whether the task completed successfully

error?: Error { code, message }

Structured error information for failed tasks

code: "TASK_ABORTED" | "MAX_ITERATIONS" | "MAX_ERRORS" | "TASK_FAILED"

Error codes for task failures

One of the following:
"TASK_ABORTED"
"MAX_ITERATIONS"
"MAX_ERRORS"
"TASK_FAILED"
message: string

Human-readable error message

event: "complete"
V1AutomateEventDone { data, event }

Envelope for the “done” event from /v1/automate.

data: Record<string, unknown>

Payload for the done stream terminator event. Empty today; reserved for future metadata.

event: "done"
V1AutomateEventError { data, event }

Envelope for the “error” event from /v1/automate.

data: Data { error, success }

Payload for the top-level error stream event. Emitted when an uncaught error escapes the task runner. Mirrors ErrorResponse from the server package’s taskRunner.ts — kept structurally aligned so schema and runtime stay consistent. Distinct from agent-level error events like ai:generation:error and task:validation_error, which are emitted through the normal event emitter during the agent loop.

error: Error { code, message, timestamp }
code: string
message: string
timestamp: string

ISO-8601 timestamp

success: false
event: "error"
V1AutomateEventInteractiveFormDataError { data, event }

Envelope for the “interactive:form_data:error” event from /v1/automate.

data: Data { fieldErrors, fields, formDescription, 5 more }

Event data when form validation fails and the agent re-requests data. Carries both the error context and the fields that need new values. Callers respond to this the same way as a request event.

fieldErrors: Record<string, string>

Per-field error messages from validation (field ref -> error text)

fields: Array<Field>
fieldType: "text" | "email" | "phone" | 8 more

Semantic field type

One of the following:
"text"
"email"
"phone"
"date"
"number"
"select"
"checkbox"
"radio"
"textarea"
"password"
"other"
label: string

The field’s visible label

ref: string

Element ref from the accessibility tree (e.g., “E42”)

required: boolean

Whether this field must be filled

currentValue?: string

Current value if already partially filled

description?: string

Additional context (e.g., validation error message on re-request)

options?: Array<string>

Available options for select/radio fields

formDescription: string
iterationId: string
pageTitle: string
pageUrl: string
requestId: string
timestamp: number
event: "interactive:form_data:error"
V1AutomateEventInteractiveFormDataRequest { data, event }

Envelope for the “interactive:form_data:request” event from /v1/automate.

data: Data { fields, formDescription, iterationId, 4 more }

Event data when the agent requests user data for form fields

fields: Array<Field>
fieldType: "text" | "email" | "phone" | 8 more

Semantic field type

One of the following:
"text"
"email"
"phone"
"date"
"number"
"select"
"checkbox"
"radio"
"textarea"
"password"
"other"
label: string

The field’s visible label

ref: string

Element ref from the accessibility tree (e.g., “E42”)

required: boolean

Whether this field must be filled

currentValue?: string

Current value if already partially filled

description?: string

Additional context (e.g., validation error message on re-request)

options?: Array<string>

Available options for select/radio fields

formDescription: string
iterationId: string
pageTitle: string
pageUrl: string
requestId: string
timestamp: number
event: "interactive:form_data:request"
V1AutomateEventSystemDebugCompression { data, event }

Envelope for the “system:debug_compression” event from /v1/automate.

data: Data { compressedSize, compressionPercent, iterationId, 2 more }

Event data for compression debug info

compressedSize: number
compressionPercent: number
iterationId: string
originalSize: number
timestamp: number
event: "system:debug_compression"
V1AutomateEventSystemDebugMessage { data, event }

Envelope for the “system:debug_message” event from /v1/automate.

data: Data { iterationId, messages, timestamp }

Event data for message debug info

iterationId: string
messages: Array<unknown>
timestamp: number
event: "system:debug_message"
V1AutomateEventTaskAborted { data, event }

Envelope for the “task:aborted” event from /v1/automate.

data: Data { finalAnswer, iterationId, reason, timestamp }

Event data when a task is aborted

finalAnswer: string
iterationId: string
reason: string
timestamp: number
event: "task:aborted"
V1AutomateEventTaskCompleted { data, event }

Envelope for the “task:completed” event from /v1/automate.

data: Data { finalAnswer, iterationId, timestamp, success }

Event data when a task is completed

finalAnswer: string | null
iterationId: string
timestamp: number
success?: boolean
event: "task:completed"
V1AutomateEventTaskMetrics { data, event }

Envelope for the “task:metrics” event from /v1/automate.

data: Data { aiGenerationCount, aiGenerationErrorCount, eventCounts, 5 more }
aiGenerationCount: number
aiGenerationErrorCount: number
eventCounts: Record<string, number>
iterationId: string
stepCount: number
timestamp: number
totalInputTokens: number
totalOutputTokens: number
event: "task:metrics"
V1AutomateEventTaskMetricsIncremental { data, event }

Envelope for the “task:metrics_incremental” event from /v1/automate.

data: Data { aiGenerationCount, aiGenerationErrorCount, eventCounts, 5 more }
aiGenerationCount: number
aiGenerationErrorCount: number
eventCounts: Record<string, number>
iterationId: string
stepCount: number
timestamp: number
totalInputTokens: number
totalOutputTokens: number
event: "task:metrics_incremental"
V1AutomateEventTaskSetup { data, event }

Envelope for the “task:setup” event from /v1/automate.

data: Data { browserName, iterationId, task, 14 more }

Event data when a task is setup

browserName: string
iterationId: string
task: string
timestamp: number
data?: unknown
guardrails?: string
hasApiKey?: boolean
keySource?: "global" | "env" | "not_set"
One of the following:
"global"
"env"
"not_set"
model?: string
provider?: string
proxy?: string
pwCdpEndpoint?: string
pwCdpEndpointCount?: number

Total number of CDP endpoints configured (index, not URLs)

pwCdpEndpoints?: Array<string>
pwEndpoint?: string
url?: string
vision?: boolean
event: "task:setup"
V1AutomateEventTaskStarted { data, event }

Envelope for the “task:started” event from /v1/automate.

data: Data { iterationId, plan, successCriteria, 4 more }

Event data when a task is started

iterationId: string
plan: string
successCriteria: string
task: string
timestamp: number
url: string
actionItems?: Array<string>
event: "task:started"
V1AutomateEventTaskTraceContext { data, event }

Envelope for the “task:trace_context” event from /v1/automate.

data: Data { traceId }

Payload for the task:trace_context event. Carries the OpenTelemetry trace ID for this /v1/automate request so consumers can deep-link to distributed-tracing UIs (e.g. Cloud Trace, Cloud Logging) for the run.

traceId: string

W3C trace ID — 32-character lowercase hexadecimal string.

event: "task:trace_context"
V1AutomateEventTaskValidated { data, event }

Envelope for the “task:validated” event from /v1/automate.

data: Data { completionQuality, finalAnswer, iterationId, 3 more }

Event data for task validation

completionQuality: "failed" | "partial" | "complete" | "excellent"
One of the following:
"failed"
"partial"
"complete"
"excellent"
finalAnswer: string
iterationId: string
observation: string
timestamp: number
feedback?: string
event: "task:validated"
V1AutomateEventTaskValidationError { data, event }

Envelope for the “task:validation_error” event from /v1/automate.

data: Data { errors, iterationId, rawResponse, 2 more }

Event data for validation errors during action response processing

errors: Array<string>
iterationId: string
rawResponse: unknown
retryCount: number
timestamp: number
event: "task:validation_error"

AI Task

import Tabstack from '@tabstack/sdk';

const client = new Tabstack({
  apiKey: process.env['TABSTACK_API_KEY'], // This is the default and can be omitted
});

const automateEvent = await client.agent.automate({
  task: 'Find the top 3 trending repositories and extract their names, descriptions, and star counts',
  guardrails: "browse and extract only, don't interact with repositories",
  url: 'https://github.com/trending',
});

console.log(automateEvent);
event: start data: {"task": "Find the top 3 trending repositories", "url": "https://github.com/trending"} event: agent:processing data: {"operation": "Creating task plan", "hasScreenshot": false} event: browser:navigated data: {"title": "Trending - GitHub", "url": "https://github.com/trending"} event: agent:extracted data: {"extractedData": "[{\"name\": \"awesome-ai\", \"stars\": \"45.2k\"}]"} event: task:completed data: {"success": true, "finalAnswer": "Top 3 trending repos extracted"} event: complete data: {"success": true} event: done data: {}
event: start data: {"task": "Find product details", "url": "https://store.com/product"} event: browser:navigated data: {"title": "Product Page", "url": "https://store.com/product"} event: agent:extracted data: {"extractedData": "{\"name\": \"Headphones\", \"price\": \"$299.99\"}"} event: task:completed data: {"success": true} event: done data: {}
{
  "error": "maxIterations must be between 1 and 100"
}
{
  "error": "invalid URL format"
}
{
  "error": "task is required"
}
Returns Examples
event: start data: {"task": "Find the top 3 trending repositories", "url": "https://github.com/trending"} event: agent:processing data: {"operation": "Creating task plan", "hasScreenshot": false} event: browser:navigated data: {"title": "Trending - GitHub", "url": "https://github.com/trending"} event: agent:extracted data: {"extractedData": "[{\"name\": \"awesome-ai\", \"stars\": \"45.2k\"}]"} event: task:completed data: {"success": true, "finalAnswer": "Top 3 trending repos extracted"} event: complete data: {"success": true} event: done data: {}
event: start data: {"task": "Find product details", "url": "https://store.com/product"} event: browser:navigated data: {"title": "Product Page", "url": "https://store.com/product"} event: agent:extracted data: {"extractedData": "{\"name\": \"Headphones\", \"price\": \"$299.99\"}"} event: task:completed data: {"success": true} event: done data: {}
{
  "error": "maxIterations must be between 1 and 100"
}
{
  "error": "invalid URL format"
}
{
  "error": "task is required"
}