AI Task

POST/automate

Execute AI-powered browser automation tasks using natural language with optional geotargeting. This endpoint always streams responses using Server-Sent Events (SSE).

Streaming Response:

All responses are streamed using Server-Sent Events (text/event-stream)
Real-time progress updates and results as they’re generated

Geotargeting:

Optionally specify a country code for geotargeted browsing

Use Cases:

Web scraping and data extraction
Form filling and interaction
Navigation and information gathering
Multi-step web workflows
Content analysis from web pages

Body ParametersJSONExpand Collapse

task: string

The task description in natural language

data: optional unknown

JSON data to provide context for form filling or complex tasks

geo_target: optional object { country }

Optional geotargeting parameters for proxy requests

country: optional string

Country code using ISO 3166-1 alpha-2 standard (2 letters, e.g., “US”, “GB”, “JP”). See: https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2

guardrails: optional string

Safety constraints for execution

interactive: optional boolean

Enable interactive mode to allow human-in-the-loop input during task execution

maxIterations: optional number

Maximum task iterations

maximum100

minimum1

maxValidationAttempts: optional number

Maximum validation attempts

maximum10

minimum1

url: optional string

Starting URL for the task

formaturi

ReturnsExpand Collapse

AutomateEvent = object { data, event } or object { data, event } or object { data, event } or 30 more

A Server-Sent Event from /v1/automate. Typed discriminated union keyed on event.

One of the following:

AgentAction object { data, event }

Envelope for the “agent:action” event from /v1/automate.

data: object { action, iterationId, timestamp, 2 more }

Event data for action execution

action: string

iterationId: string

timestamp: number

ref: optional string

value: optional string

event: "agent:action"

AgentExtracted object { data, event }

Envelope for the “agent:extracted” event from /v1/automate.

data: object { extractedData, iterationId, timestamp }

Event data for extracted data

extractedData: string

iterationId: string

timestamp: number

event: "agent:extracted"

AgentProcessing object { data, event }

Envelope for the “agent:processing” event from /v1/automate.

data: object { hasScreenshot, iterationId, operation, timestamp }

Event data for when the agent is waiting for model generation

hasScreenshot: boolean

iterationId: string

operation: string

timestamp: number

event: "agent:processing"

AgentReasoned object { data, event }

Envelope for the “agent:reasoned” event from /v1/automate.

data: object { iterationId, reasoning, timestamp }

Event data for agent reasoning

iterationId: string

reasoning: string

timestamp: number

event: "agent:reasoned"

AgentStatus object { data, event }

Envelope for the “agent:status” event from /v1/automate.

data: object { iterationId, message, timestamp }

Event data for status messages

iterationId: string

message: string

timestamp: number

event: "agent:status"

AgentStep object { data, event }

Envelope for the “agent:step” event from /v1/automate.

data: object { currentIteration, iterationId, timestamp }

Event data for agent step tracking (each loop iteration)

currentIteration: number

iterationId: string

timestamp: number

event: "agent:step"

AgentWaiting object { data, event }

Envelope for the “agent:waiting” event from /v1/automate.

data: object { iterationId, seconds, timestamp }

Event data for waiting notifications

iterationId: string

seconds: number

timestamp: number

event: "agent:waiting"

AIGeneration object { data, event }

Envelope for the “ai:generation” event from /v1/automate.

data: object { finishReason, iterationId, prompt, 8 more }

Event data when AI generation occurs

finishReason: "stop" or "length" or "content-filter" or 3 more

One of the following:

"stop"

"length"

"content-filter"

"tool-calls"

"error"

"other"

iterationId: string

prompt: string

schema: unknown

timestamp: number

usage: object { inputTokens, outputTokens, totalTokens }

inputTokens: optional number

outputTokens: optional number

totalTokens: optional number

messages: optional array of object { content, role, providerOptions } or object { content, role, providerOptions } or object { content, role, providerOptions } or object { content, role, providerOptions }

One of the following:

System object { content, role, providerOptions }

A system message. It can contain system information.

Note: using the “system” part of the prompt is strongly preferred to increase the resilience against prompt injection attacks, and because not all providers support several system messages.

content: string

role: "system"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

User object { content, role, providerOptions }

A user message. It can contain text or a combination of text and images.

content: string or array of object { text, type, providerOptions } or object { image, type, mediaType, providerOptions } or object { data, mediaType, type, 2 more }

Content of a user message. It can be a string or an array of text and image parts.

One of the following:

string

array of object { text, type, providerOptions } or object { image, type, mediaType, providerOptions } or object { data, mediaType, type, 2 more }

One of the following:

Text object { text, type, providerOptions }

Text content part of a prompt. It contains a string of text.

text: string

The text content.

type: "text"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

Image object { image, type, mediaType, providerOptions }

Image content part of a prompt. It contains an image.

image: string or object { buffer, byteLength, byteOffset, 2 more } or object { byteLength } or object { buffer, byteLength, byteOffset, 2 more }

Image data. Can either be:

data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
URL: a URL that points to the image

One of the following:

string

object { buffer, byteLength, byteOffset, 2 more }

buffer: object { byteLength }

byteLength: number

byteOffset: number

BYTES_PER_ELEMENT: number

length: number

ByteLength object { byteLength }

byteLength: number

V1GlobalBuffer object { buffer, byteLength, byteOffset, 2 more }

buffer: object { byteLength }

byteLength: number

byteOffset: number

BYTES_PER_ELEMENT: number

length: number

type: "image"

mediaType: optional string

Optional IANA media type of the image.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

File object { data, mediaType, type, 2 more }

File content part of a prompt. It contains a file.

data: string or object { buffer, byteLength, byteOffset, 2 more } or object { byteLength } or object { buffer, byteLength, byteOffset, 2 more }

File data. Can either be:

data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
URL: a URL that points to the image

One of the following:

string

object { buffer, byteLength, byteOffset, 2 more }

buffer: object { byteLength }

byteLength: number

byteOffset: number

BYTES_PER_ELEMENT: number

length: number

ByteLength object { byteLength }

byteLength: number

V1GlobalBuffer object { buffer, byteLength, byteOffset, 2 more }

buffer: object { byteLength }

byteLength: number

byteOffset: number

BYTES_PER_ELEMENT: number

length: number

mediaType: string

IANA media type of the file.

type: "file"

filename: optional string

Optional filename of the file.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

role: "user"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

Assistant object { content, role, providerOptions }

An assistant message. It can contain text, tool calls, or a combination of text and tool calls.

content: string or array of object { text, type, providerOptions } or object { data, mediaType, type, 2 more } or object { text, type, providerOptions } or 3 more

Content of an assistant message. It can be a string or an array of text, image, reasoning, redacted reasoning, and tool call parts.

One of the following:

string

array of object { text, type, providerOptions } or object { data, mediaType, type, 2 more } or object { text, type, providerOptions } or 3 more

One of the following:

Text object { text, type, providerOptions }

Text content part of a prompt. It contains a string of text.

text: string

The text content.

type: "text"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

File object { data, mediaType, type, 2 more }

File content part of a prompt. It contains a file.

data: string or object { buffer, byteLength, byteOffset, 2 more } or object { byteLength } or object { buffer, byteLength, byteOffset, 2 more }

File data. Can either be:

data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
URL: a URL that points to the image

One of the following:

string

object { buffer, byteLength, byteOffset, 2 more }

buffer: object { byteLength }

byteLength: number

byteOffset: number

BYTES_PER_ELEMENT: number

length: number

ByteLength object { byteLength }

byteLength: number

V1GlobalBuffer object { buffer, byteLength, byteOffset, 2 more }

buffer: object { byteLength }

byteLength: number

byteOffset: number

BYTES_PER_ELEMENT: number

length: number

mediaType: string

IANA media type of the file.

type: "file"

filename: optional string

Optional filename of the file.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

Reasoning object { text, type, providerOptions }

Reasoning content part of a prompt. It contains a reasoning.

text: string

The reasoning text.

type: "reasoning"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ToolCall object { input, toolCallId, toolName, 3 more }

Tool call content part of a prompt. It contains a tool call (usually generated by the AI model).

input: unknown

Arguments of the tool call. This is a JSON-serializable object that matches the tool’s input schema.

toolCallId: string

ID of the tool call. This ID is used to match the tool call with the tool result.

toolName: string

Name of the tool that is being called.

type: "tool-call"

providerExecuted: optional boolean

Whether the tool call was executed by the provider.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ToolResult object { output, toolCallId, toolName, 2 more }

Tool result content part of a prompt. It contains the result of the tool call with the matching ID.

output: object { type, value, providerOptions } or object { type, value, providerOptions } or object { type, providerOptions, reason } or 3 more

Result of the tool call. This is a JSON-serializable object.

One of the following:

Text object { type, value, providerOptions }

type: "text"

Text tool output that should be directly sent to the API.

value: string

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

Json object { type, value, providerOptions }

type: "json"

value: string or number or boolean or 2 more

A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.

One of the following:

string

number

boolean

map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ExecutionDenied object { type, providerOptions, reason }

type: "execution-denied"

Type when the user has denied the execution of the tool call.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

reason: optional string

Optional reason for the execution denial.

ErrorText object { type, value, providerOptions }

type: "error-text"

value: string

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ErrorJson object { type, value, providerOptions }

type: "error-json"

value: string or number or boolean or 2 more

A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.

One of the following:

string

number

boolean

map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

Content object { type, value }

type: "content"

value: array of object { text, type, providerOptions } or object { data, mediaType, type } or object { data, mediaType, type, 2 more } or 6 more

One of the following:

Text object { text, type, providerOptions }

text: string

Text content.

type: "text"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

Media object { data, mediaType, type }

data: string

mediaType: string

Deprecatedtype: "media"

Deprecated by the upstream schema.

Deprecated. Use image-data or file-data instead.

FileData object { data, mediaType, type, 2 more }

data: string

Base-64 encoded media data.

mediaType: string

IANA media type.

type: "file-data"

filename: optional string

Optional filename of the file.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

FileURL object { type, url, providerOptions }

type: "file-url"

url: string

URL of the file.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

FileID object { fileId, type, providerOptions }

fileId: string or map[string]

ID of the file.

If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.

One of the following:

string

map[string]

type: "file-id"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ImageData object { data, mediaType, type, providerOptions }

data: string

Base-64 encoded image data.

mediaType: string

IANA media type.

type: "image-data"

Images that are referenced using base64 encoded data.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ImageURL object { type, url, providerOptions }

type: "image-url"

Images that are referenced using a URL.

url: string

URL of the image.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ImageFileID object { fileId, type, providerOptions }

fileId: string or map[string]

Image that is referenced using a provider file id.

If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.

One of the following:

string

map[string]

type: "image-file-id"

Images that are referenced using a provider file id.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

Custom object { type, providerOptions }

type: "custom"

Custom content part. This can be used to implement provider-specific content parts.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

toolCallId: string

ID of the tool call that this result is associated with.

toolName: string

Name of the tool that generated this result.

type: "tool-result"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ToolApprovalRequest object { approvalId, toolCallId, type }

Tool approval request prompt part.

approvalId: string

ID of the tool approval.

toolCallId: string

ID of the tool call that the approval request is for.

type: "tool-approval-request"

role: "assistant"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

Tool object { content, role, providerOptions }

A tool message. It contains the result of one or more tool calls.

content: array of object { output, toolCallId, toolName, 2 more } or object { approvalId, approved, type, 2 more }

Content of a tool message. It is an array of tool result parts.

One of the following:

ToolResult object { output, toolCallId, toolName, 2 more }

Tool result content part of a prompt. It contains the result of the tool call with the matching ID.

output: object { type, value, providerOptions } or object { type, value, providerOptions } or object { type, providerOptions, reason } or 3 more

Result of the tool call. This is a JSON-serializable object.

One of the following:

Text object { type, value, providerOptions }

type: "text"

Text tool output that should be directly sent to the API.

value: string

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

Json object { type, value, providerOptions }

type: "json"

value: string or number or boolean or 2 more

A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.

One of the following:

string

number

boolean

map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ExecutionDenied object { type, providerOptions, reason }

type: "execution-denied"

Type when the user has denied the execution of the tool call.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

reason: optional string

Optional reason for the execution denial.

ErrorText object { type, value, providerOptions }

type: "error-text"

value: string

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ErrorJson object { type, value, providerOptions }

type: "error-json"

value: string or number or boolean or 2 more

A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.

One of the following:

string

number

boolean

map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

Content object { type, value }

type: "content"

value: array of object { text, type, providerOptions } or object { data, mediaType, type } or object { data, mediaType, type, 2 more } or 6 more

One of the following:

Text object { text, type, providerOptions }

text: string

Text content.

type: "text"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

Media object { data, mediaType, type }

data: string

mediaType: string

Deprecatedtype: "media"

Deprecated by the upstream schema.

Deprecated. Use image-data or file-data instead.

FileData object { data, mediaType, type, 2 more }

data: string

Base-64 encoded media data.

mediaType: string

IANA media type.

type: "file-data"

filename: optional string

Optional filename of the file.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

FileURL object { type, url, providerOptions }

type: "file-url"

url: string

URL of the file.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

FileID object { fileId, type, providerOptions }

fileId: string or map[string]

ID of the file.

If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.

One of the following:

string

map[string]

type: "file-id"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ImageData object { data, mediaType, type, providerOptions }

data: string

Base-64 encoded image data.

mediaType: string

IANA media type.

type: "image-data"

Images that are referenced using base64 encoded data.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ImageURL object { type, url, providerOptions }

type: "image-url"

Images that are referenced using a URL.

url: string

URL of the image.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ImageFileID object { fileId, type, providerOptions }

fileId: string or map[string]

Image that is referenced using a provider file id.

If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.

One of the following:

string

map[string]

type: "image-file-id"

Images that are referenced using a provider file id.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

Custom object { type, providerOptions }

type: "custom"

Custom content part. This can be used to implement provider-specific content parts.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

toolCallId: string

ID of the tool call that this result is associated with.

toolName: string

Name of the tool that generated this result.

type: "tool-result"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ToolApprovalResponse object { approvalId, approved, type, 2 more }

Tool approval response prompt part.

approvalId: string

ID of the tool approval.

approved: boolean

Flag indicating whether the approval was granted or denied.

type: "tool-approval-response"

providerExecuted: optional boolean

Flag indicating whether the tool call is provider-executed. Only provider-executed tool approval responses should be sent to the model.

reason: optional string

Optional reason for the approval or denial.

role: "tool"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

object: optional unknown

providerMetadata: optional map[unknown]

temperature: optional number

warnings: optional array of unknown

event: "ai:generation"

AIGenerationError object { data, event }

Envelope for the “ai:generation:error” event from /v1/automate.

data: object { error, iterationId, prompt, 3 more }

Event data when AI generation error occurs

error: string

iterationId: string

prompt: string

schema: unknown

timestamp: number

messages: optional array of unknown

event: "ai:generation:error"

BrowserActionCompleted object { data, event }

Envelope for the “browser:action_completed” event from /v1/automate.

data: object { iterationId, success, timestamp, error }

Event data for action results

iterationId: string

success: boolean

timestamp: number

error: optional string

event: "browser:action_completed"

BrowserActionStarted object { data, event }

Envelope for the “browser:action_started” event from /v1/automate.

data: object { action, iterationId, timestamp, 2 more }

Event data for action execution

action: string

iterationId: string

timestamp: number

ref: optional string

value: optional string

event: "browser:action_started"

BrowserNavigated object { data, event }

Envelope for the “browser:navigated” event from /v1/automate.

data: object { iterationId, timestamp, title, url }

Event data when navigating to a page

iterationId: string

timestamp: number

title: string

url: string

event: "browser:navigated"

BrowserReconnected object { data, event }

Envelope for the “browser:reconnected” event from /v1/automate.

data: object { endpointIndex, iterationId, startingUrl, 2 more }

Event data when the browser reconnects after a mid-task disconnect

endpointIndex: number

1-based index of the CDP endpoint now in use

iterationId: string

startingUrl: string

The original starting URL the agent is restarting execution from

timestamp: number

total: number

Total number of configured CDP endpoints

event: "browser:reconnected"

BrowserScreenshotCaptured object { data, event }

Envelope for the “browser:screenshot_captured” event from /v1/automate.

data: object { format, iterationId, size, timestamp }

Event data for screenshot capture

format: "jpeg" or "png"

One of the following:

"jpeg"

"png"

iterationId: string

size: number

timestamp: number

event: "browser:screenshot_captured"

BrowserScreenshotCapturedImage object { data, event }

Envelope for the “browser:screenshot_captured_image” event from /v1/automate.

data: object { image, iterationId, mediaType, timestamp }

Event data for screenshot image capture with full image data This event contains the complete screenshot and can be very large

image: string

iterationId: string

mediaType: "image/jpeg" or "image/png"

One of the following:

"image/jpeg"

"image/png"

timestamp: number

event: "browser:screenshot_captured_image"

CdpEndpointConnected object { data, event }

Envelope for the “cdp:endpoint_connected” event from /v1/automate.

data: object { endpointIndex, iterationId, timestamp, total }

Event data when a CDP endpoint is successfully connected to

endpointIndex: number

1-based index of the endpoint that connected

iterationId: string

timestamp: number

total: number

Total number of configured CDP endpoints

event: "cdp:endpoint_connected"

CdpEndpointCycle object { data, event }

Envelope for the “cdp:endpoint_cycle” event from /v1/automate.

data: object { attempt, error, iterationId, 2 more }

Event data when a CDP endpoint fails and the next one is being tried

attempt: number

1-based index of the endpoint attempt that failed

error: string

Sanitized error identifier from the failed connection attempt (error.name, not error.message — full messages may contain endpoint URLs)

iterationId: string

timestamp: number

total: number

Total number of configured CDP endpoints

event: "cdp:endpoint_cycle"

Complete object { data, event }

Envelope for the “complete” event from /v1/automate.

data: object { finalAnswer, stats, success, error }

Payload for the complete stream event. Structurally identical to TaskExecutionResult from webAgent.ts — the complete event’s data is the agent’s final TaskExecutionResult, stringified onto the SSE stream.

finalAnswer: string

Final answer or result from the agent

stats: object { actions, durationMs, endTime, 2 more }

Execution statistics

actions: number

durationMs: number

endTime: number

iterations: number

startTime: number

success: boolean

Whether the task completed successfully

error: optional object { code, message }

Structured error information for failed tasks

code: "TASK_ABORTED" or "MAX_ITERATIONS" or "MAX_ERRORS" or "TASK_FAILED"

Error codes for task failures

One of the following:

"TASK_ABORTED"

"MAX_ITERATIONS"

"MAX_ERRORS"

"TASK_FAILED"

message: string

Human-readable error message

event: "complete"

Done object { data, event }

Envelope for the “done” event from /v1/automate.

data: map[unknown]

Payload for the done stream terminator event. Empty today; reserved for future metadata.

event: "done"

Error object { data, event }

Envelope for the “error” event from /v1/automate.

data: object { error, success }

Payload for the top-level error stream event. Emitted when an uncaught error escapes the task runner. Mirrors ErrorResponse from the server package’s taskRunner.ts — kept structurally aligned so schema and runtime stay consistent. Distinct from agent-level error events like ai:generation:error and task:validation_error, which are emitted through the normal event emitter during the agent loop.

error: object { code, message, timestamp }

code: string

message: string

timestamp: string

ISO-8601 timestamp

success: false

event: "error"

InteractiveFormDataError object { data, event }

Envelope for the “interactive:form_data:error” event from /v1/automate.

data: object { fieldErrors, fields, formDescription, 5 more }

Event data when form validation fails and the agent re-requests data. Carries both the error context and the fields that need new values. Callers respond to this the same way as a request event.

fieldErrors: map[string]

Per-field error messages from validation (field ref -> error text)

fields: array of object { fieldType, label, ref, 4 more }

fieldType: "text" or "email" or "phone" or 8 more

Semantic field type

One of the following:

"text"

"email"

"phone"

"date"

"number"

"select"

"checkbox"

"radio"

"textarea"

"password"

"other"

label: string

The field’s visible label

ref: string

Element ref from the accessibility tree (e.g., “E42”)

required: boolean

Whether this field must be filled

currentValue: optional string

Current value if already partially filled

description: optional string

Additional context (e.g., validation error message on re-request)

options: optional array of string

Available options for select/radio fields

formDescription: string

iterationId: string

pageTitle: string

pageUrl: string

requestId: string

timestamp: number

event: "interactive:form_data:error"

InteractiveFormDataRequest object { data, event }

Envelope for the “interactive:form_data:request” event from /v1/automate.

data: object { fields, formDescription, iterationId, 4 more }

Event data when the agent requests user data for form fields

fields: array of object { fieldType, label, ref, 4 more }

fieldType: "text" or "email" or "phone" or 8 more

Semantic field type

One of the following:

"text"

"email"

"phone"

"date"

"number"

"select"

"checkbox"

"radio"

"textarea"

"password"

"other"

label: string

The field’s visible label

ref: string

Element ref from the accessibility tree (e.g., “E42”)

required: boolean

Whether this field must be filled

currentValue: optional string

Current value if already partially filled

description: optional string

Additional context (e.g., validation error message on re-request)

options: optional array of string

Available options for select/radio fields

formDescription: string

iterationId: string

pageTitle: string

pageUrl: string

requestId: string

timestamp: number

event: "interactive:form_data:request"

SystemDebugCompression object { data, event }

Envelope for the “system:debug_compression” event from /v1/automate.

data: object { compressedSize, compressionPercent, iterationId, 2 more }

Event data for compression debug info

compressedSize: number

compressionPercent: number

iterationId: string

originalSize: number

timestamp: number

event: "system:debug_compression"

SystemDebugMessage object { data, event }

Envelope for the “system:debug_message” event from /v1/automate.

data: object { iterationId, messages, timestamp }

Event data for message debug info

iterationId: string

messages: array of unknown

timestamp: number

event: "system:debug_message"

TaskAborted object { data, event }

Envelope for the “task:aborted” event from /v1/automate.

data: object { finalAnswer, iterationId, reason, timestamp }

Event data when a task is aborted

finalAnswer: string

iterationId: string

reason: string

timestamp: number

event: "task:aborted"

TaskCompleted object { data, event }

Envelope for the “task:completed” event from /v1/automate.

data: object { finalAnswer, iterationId, timestamp, success }

Event data when a task is completed

finalAnswer: string

iterationId: string

timestamp: number

success: optional boolean

event: "task:completed"

TaskMetrics object { data, event }

Envelope for the “task:metrics” event from /v1/automate.

data: object { aiGenerationCount, aiGenerationErrorCount, eventCounts, 5 more }

aiGenerationCount: number

aiGenerationErrorCount: number

eventCounts: map[number]

iterationId: string

stepCount: number

timestamp: number

totalInputTokens: number

totalOutputTokens: number

event: "task:metrics"

TaskMetricsIncremental object { data, event }

Envelope for the “task:metrics_incremental” event from /v1/automate.

data: object { aiGenerationCount, aiGenerationErrorCount, eventCounts, 5 more }

aiGenerationCount: number

aiGenerationErrorCount: number

eventCounts: map[number]

iterationId: string

stepCount: number

timestamp: number

totalInputTokens: number

totalOutputTokens: number

event: "task:metrics_incremental"

TaskSetup object { data, event }

Envelope for the “task:setup” event from /v1/automate.

data: object { browserName, iterationId, task, 14 more }

Event data when a task is setup

browserName: string

iterationId: string

task: string

timestamp: number

data: optional unknown

guardrails: optional string

hasApiKey: optional boolean

keySource: optional "global" or "env" or "not_set"

One of the following:

"global"

"env"

"not_set"

model: optional string

provider: optional string

proxy: optional string

pwCdpEndpoint: optional string

pwCdpEndpointCount: optional number

Total number of CDP endpoints configured (index, not URLs)

pwCdpEndpoints: optional array of string

pwEndpoint: optional string

url: optional string

vision: optional boolean

event: "task:setup"

TaskStarted object { data, event }

Envelope for the “task:started” event from /v1/automate.

data: object { iterationId, plan, successCriteria, 4 more }

Event data when a task is started

iterationId: string

plan: string

successCriteria: string

task: string

timestamp: number

url: string

actionItems: optional array of string

event: "task:started"

TaskTraceContext object { data, event }

Envelope for the “task:trace_context” event from /v1/automate.

data: object { traceId }

Payload for the task:trace_context event. Carries the OpenTelemetry trace ID for this /v1/automate request so consumers can deep-link to distributed-tracing UIs (e.g. Cloud Trace, Cloud Logging) for the run.

traceId: string

W3C trace ID — 32-character lowercase hexadecimal string.

event: "task:trace_context"

TaskValidated object { data, event }

Envelope for the “task:validated” event from /v1/automate.

data: object { completionQuality, finalAnswer, iterationId, 3 more }

Event data for task validation

completionQuality: "failed" or "partial" or "complete" or "excellent"

One of the following:

"failed"

"partial"

"complete"

"excellent"

finalAnswer: string

iterationId: string

observation: string

timestamp: number

feedback: optional string

event: "task:validated"

TaskValidationError object { data, event }

Envelope for the “task:validation_error” event from /v1/automate.

data: object { errors, iterationId, rawResponse, 2 more }

Event data for validation errors during action response processing

errors: array of string

iterationId: string

rawResponse: unknown

retryCount: number

timestamp: number

event: "task:validation_error"

AutomateEvent = object { data, event } or object { data, event } or object { data, event } or 30 more

A Server-Sent Event from /v1/automate. Typed discriminated union keyed on event.

One of the following:

AgentAction object { data, event }

Envelope for the “agent:action” event from /v1/automate.

data: object { action, iterationId, timestamp, 2 more }

Event data for action execution

action: string

iterationId: string

timestamp: number

ref: optional string

value: optional string

event: "agent:action"

AgentExtracted object { data, event }

Envelope for the “agent:extracted” event from /v1/automate.

data: object { extractedData, iterationId, timestamp }

Event data for extracted data

extractedData: string

iterationId: string

timestamp: number

event: "agent:extracted"

AgentProcessing object { data, event }

Envelope for the “agent:processing” event from /v1/automate.

data: object { hasScreenshot, iterationId, operation, timestamp }

Event data for when the agent is waiting for model generation

hasScreenshot: boolean

iterationId: string

operation: string

timestamp: number

event: "agent:processing"

AgentReasoned object { data, event }

Envelope for the “agent:reasoned” event from /v1/automate.

data: object { iterationId, reasoning, timestamp }

Event data for agent reasoning

iterationId: string

reasoning: string

timestamp: number

event: "agent:reasoned"

AgentStatus object { data, event }

Envelope for the “agent:status” event from /v1/automate.

data: object { iterationId, message, timestamp }

Event data for status messages

iterationId: string

message: string

timestamp: number

event: "agent:status"

AgentStep object { data, event }

Envelope for the “agent:step” event from /v1/automate.

data: object { currentIteration, iterationId, timestamp }

Event data for agent step tracking (each loop iteration)

currentIteration: number

iterationId: string

timestamp: number

event: "agent:step"

AgentWaiting object { data, event }

Envelope for the “agent:waiting” event from /v1/automate.

data: object { iterationId, seconds, timestamp }

Event data for waiting notifications

iterationId: string

seconds: number

timestamp: number

event: "agent:waiting"

AIGeneration object { data, event }

Envelope for the “ai:generation” event from /v1/automate.

data: object { finishReason, iterationId, prompt, 8 more }

Event data when AI generation occurs

finishReason: "stop" or "length" or "content-filter" or 3 more

One of the following:

"stop"

"length"

"content-filter"

"tool-calls"

"error"

"other"

iterationId: string

prompt: string

schema: unknown

timestamp: number

usage: object { inputTokens, outputTokens, totalTokens }

inputTokens: optional number

outputTokens: optional number

totalTokens: optional number

One of the following:

System object { content, role, providerOptions }

A system message. It can contain system information.

Note: using the “system” part of the prompt is strongly preferred to increase the resilience against prompt injection attacks, and because not all providers support several system messages.

content: string

role: "system"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

User object { content, role, providerOptions }

A user message. It can contain text or a combination of text and images.

content: string or array of object { text, type, providerOptions } or object { image, type, mediaType, providerOptions } or object { data, mediaType, type, 2 more }

Content of a user message. It can be a string or an array of text and image parts.

One of the following:

string

array of object { text, type, providerOptions } or object { image, type, mediaType, providerOptions } or object { data, mediaType, type, 2 more }

One of the following:

Text object { text, type, providerOptions }

Text content part of a prompt. It contains a string of text.

text: string

The text content.

type: "text"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

Image object { image, type, mediaType, providerOptions }

Image content part of a prompt. It contains an image.

image: string or object { buffer, byteLength, byteOffset, 2 more } or object { byteLength } or object { buffer, byteLength, byteOffset, 2 more }

Image data. Can either be:

data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
URL: a URL that points to the image

One of the following:

string

object { buffer, byteLength, byteOffset, 2 more }

buffer: object { byteLength }

byteLength: number

byteOffset: number

BYTES_PER_ELEMENT: number

length: number

ByteLength object { byteLength }

byteLength: number

V1GlobalBuffer object { buffer, byteLength, byteOffset, 2 more }

buffer: object { byteLength }

byteLength: number

byteOffset: number

BYTES_PER_ELEMENT: number

length: number

type: "image"

mediaType: optional string

Optional IANA media type of the image.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

File object { data, mediaType, type, 2 more }

File content part of a prompt. It contains a file.

data: string or object { buffer, byteLength, byteOffset, 2 more } or object { byteLength } or object { buffer, byteLength, byteOffset, 2 more }

File data. Can either be:

data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
URL: a URL that points to the image

One of the following:

string

object { buffer, byteLength, byteOffset, 2 more }

buffer: object { byteLength }

byteLength: number

byteOffset: number

BYTES_PER_ELEMENT: number

length: number

ByteLength object { byteLength }

byteLength: number

V1GlobalBuffer object { buffer, byteLength, byteOffset, 2 more }

buffer: object { byteLength }

byteLength: number

byteOffset: number

BYTES_PER_ELEMENT: number

length: number

mediaType: string

IANA media type of the file.

type: "file"

filename: optional string

Optional filename of the file.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

role: "user"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

Assistant object { content, role, providerOptions }

An assistant message. It can contain text, tool calls, or a combination of text and tool calls.

content: string or array of object { text, type, providerOptions } or object { data, mediaType, type, 2 more } or object { text, type, providerOptions } or 3 more

Content of an assistant message. It can be a string or an array of text, image, reasoning, redacted reasoning, and tool call parts.

One of the following:

string

array of object { text, type, providerOptions } or object { data, mediaType, type, 2 more } or object { text, type, providerOptions } or 3 more

One of the following:

Text object { text, type, providerOptions }

Text content part of a prompt. It contains a string of text.

text: string

The text content.

type: "text"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

File object { data, mediaType, type, 2 more }

File content part of a prompt. It contains a file.

data: string or object { buffer, byteLength, byteOffset, 2 more } or object { byteLength } or object { buffer, byteLength, byteOffset, 2 more }

File data. Can either be:

data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
URL: a URL that points to the image

One of the following:

string

object { buffer, byteLength, byteOffset, 2 more }

buffer: object { byteLength }

byteLength: number

byteOffset: number

BYTES_PER_ELEMENT: number

length: number

ByteLength object { byteLength }

byteLength: number

V1GlobalBuffer object { buffer, byteLength, byteOffset, 2 more }

buffer: object { byteLength }

byteLength: number

byteOffset: number

BYTES_PER_ELEMENT: number

length: number

mediaType: string

IANA media type of the file.

type: "file"

filename: optional string

Optional filename of the file.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

Reasoning object { text, type, providerOptions }

Reasoning content part of a prompt. It contains a reasoning.

text: string

The reasoning text.

type: "reasoning"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ToolCall object { input, toolCallId, toolName, 3 more }

Tool call content part of a prompt. It contains a tool call (usually generated by the AI model).

input: unknown

Arguments of the tool call. This is a JSON-serializable object that matches the tool’s input schema.

toolCallId: string

ID of the tool call. This ID is used to match the tool call with the tool result.

toolName: string

Name of the tool that is being called.

type: "tool-call"

providerExecuted: optional boolean

Whether the tool call was executed by the provider.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ToolResult object { output, toolCallId, toolName, 2 more }

Tool result content part of a prompt. It contains the result of the tool call with the matching ID.

output: object { type, value, providerOptions } or object { type, value, providerOptions } or object { type, providerOptions, reason } or 3 more

Result of the tool call. This is a JSON-serializable object.

One of the following:

Text object { type, value, providerOptions }

type: "text"

Text tool output that should be directly sent to the API.

value: string

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

Json object { type, value, providerOptions }

type: "json"

value: string or number or boolean or 2 more

A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.

One of the following:

string

number

boolean

map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ExecutionDenied object { type, providerOptions, reason }

type: "execution-denied"

Type when the user has denied the execution of the tool call.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

reason: optional string

Optional reason for the execution denial.

ErrorText object { type, value, providerOptions }

type: "error-text"

value: string

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ErrorJson object { type, value, providerOptions }

type: "error-json"

value: string or number or boolean or 2 more

A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.

One of the following:

string

number

boolean

map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

Content object { type, value }

type: "content"

value: array of object { text, type, providerOptions } or object { data, mediaType, type } or object { data, mediaType, type, 2 more } or 6 more

One of the following:

Text object { text, type, providerOptions }

text: string

Text content.

type: "text"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

Media object { data, mediaType, type }

data: string

mediaType: string

Deprecatedtype: "media"

Deprecated by the upstream schema.

Deprecated. Use image-data or file-data instead.

FileData object { data, mediaType, type, 2 more }

data: string

Base-64 encoded media data.

mediaType: string

IANA media type.

type: "file-data"

filename: optional string

Optional filename of the file.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

FileURL object { type, url, providerOptions }

type: "file-url"

url: string

URL of the file.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

FileID object { fileId, type, providerOptions }

fileId: string or map[string]

ID of the file.

If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.

One of the following:

string

map[string]

type: "file-id"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ImageData object { data, mediaType, type, providerOptions }

data: string

Base-64 encoded image data.

mediaType: string

IANA media type.

type: "image-data"

Images that are referenced using base64 encoded data.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ImageURL object { type, url, providerOptions }

type: "image-url"

Images that are referenced using a URL.

url: string

URL of the image.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ImageFileID object { fileId, type, providerOptions }

fileId: string or map[string]

Image that is referenced using a provider file id.

If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.

One of the following:

string

map[string]

type: "image-file-id"

Images that are referenced using a provider file id.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

Custom object { type, providerOptions }

type: "custom"

Custom content part. This can be used to implement provider-specific content parts.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

toolCallId: string

ID of the tool call that this result is associated with.

toolName: string

Name of the tool that generated this result.

type: "tool-result"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ToolApprovalRequest object { approvalId, toolCallId, type }

Tool approval request prompt part.

approvalId: string

ID of the tool approval.

toolCallId: string

ID of the tool call that the approval request is for.

type: "tool-approval-request"

role: "assistant"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

Tool object { content, role, providerOptions }

A tool message. It contains the result of one or more tool calls.

content: array of object { output, toolCallId, toolName, 2 more } or object { approvalId, approved, type, 2 more }

Content of a tool message. It is an array of tool result parts.

One of the following:

ToolResult object { output, toolCallId, toolName, 2 more }

Tool result content part of a prompt. It contains the result of the tool call with the matching ID.

output: object { type, value, providerOptions } or object { type, value, providerOptions } or object { type, providerOptions, reason } or 3 more

Result of the tool call. This is a JSON-serializable object.

One of the following:

Text object { type, value, providerOptions }

type: "text"

Text tool output that should be directly sent to the API.

value: string

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

Json object { type, value, providerOptions }

type: "json"

value: string or number or boolean or 2 more

A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.

One of the following:

string

number

boolean

map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ExecutionDenied object { type, providerOptions, reason }

type: "execution-denied"

Type when the user has denied the execution of the tool call.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

reason: optional string

Optional reason for the execution denial.

ErrorText object { type, value, providerOptions }

type: "error-text"

value: string

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ErrorJson object { type, value, providerOptions }

type: "error-json"

value: string or number or boolean or 2 more

A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.

One of the following:

string

number

boolean

map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

Content object { type, value }

type: "content"

value: array of object { text, type, providerOptions } or object { data, mediaType, type } or object { data, mediaType, type, 2 more } or 6 more

One of the following:

Text object { text, type, providerOptions }

text: string

Text content.

type: "text"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

Media object { data, mediaType, type }

data: string

mediaType: string

Deprecatedtype: "media"

Deprecated by the upstream schema.

Deprecated. Use image-data or file-data instead.

FileData object { data, mediaType, type, 2 more }

data: string

Base-64 encoded media data.

mediaType: string

IANA media type.

type: "file-data"

filename: optional string

Optional filename of the file.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

FileURL object { type, url, providerOptions }

type: "file-url"

url: string

URL of the file.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

FileID object { fileId, type, providerOptions }

fileId: string or map[string]

ID of the file.

If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.

One of the following:

string

map[string]

type: "file-id"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ImageData object { data, mediaType, type, providerOptions }

data: string

Base-64 encoded image data.

mediaType: string

IANA media type.

type: "image-data"

Images that are referenced using base64 encoded data.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ImageURL object { type, url, providerOptions }

type: "image-url"

Images that are referenced using a URL.

url: string

URL of the image.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ImageFileID object { fileId, type, providerOptions }

fileId: string or map[string]

Image that is referenced using a provider file id.

If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.

One of the following:

string

map[string]

type: "image-file-id"

Images that are referenced using a provider file id.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

Custom object { type, providerOptions }

type: "custom"

Custom content part. This can be used to implement provider-specific content parts.

providerOptions: optional map[map[string or number or boolean or 2 more]]

Provider-specific options.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

toolCallId: string

ID of the tool call that this result is associated with.

toolName: string

Name of the tool that generated this result.

type: "tool-result"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

ToolApprovalResponse object { approvalId, approved, type, 2 more }

Tool approval response prompt part.

approvalId: string

ID of the tool approval.

approved: boolean

Flag indicating whether the approval was granted or denied.

type: "tool-approval-response"

providerExecuted: optional boolean

Flag indicating whether the tool call is provider-executed. Only provider-executed tool approval responses should be sent to the model.

reason: optional string

Optional reason for the approval or denial.

role: "tool"

providerOptions: optional map[map[string or number or boolean or 2 more]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

string

number

boolean

V1Alias922155206_411_476_922155206_0_129727 = map[string or number or boolean or 2 more]

One of the following:

string

number

boolean

unknown

array of unknown

array of string or number or boolean or 2 more

One of the following:

string

number

boolean

unknown

array of unknown

object: optional unknown

providerMetadata: optional map[unknown]

temperature: optional number

warnings: optional array of unknown

event: "ai:generation"

AIGenerationError object { data, event }

Envelope for the “ai:generation:error” event from /v1/automate.

data: object { error, iterationId, prompt, 3 more }

Event data when AI generation error occurs

error: string

iterationId: string

prompt: string

schema: unknown

timestamp: number

messages: optional array of unknown

event: "ai:generation:error"

BrowserActionCompleted object { data, event }

Envelope for the “browser:action_completed” event from /v1/automate.

data: object { iterationId, success, timestamp, error }

Event data for action results

iterationId: string

success: boolean

timestamp: number

error: optional string

event: "browser:action_completed"

BrowserActionStarted object { data, event }

Envelope for the “browser:action_started” event from /v1/automate.

data: object { action, iterationId, timestamp, 2 more }

Event data for action execution

action: string

iterationId: string

timestamp: number

ref: optional string

value: optional string

event: "browser:action_started"

BrowserNavigated object { data, event }

Envelope for the “browser:navigated” event from /v1/automate.

data: object { iterationId, timestamp, title, url }

Event data when navigating to a page

iterationId: string

timestamp: number

title: string

url: string

event: "browser:navigated"

BrowserReconnected object { data, event }

Envelope for the “browser:reconnected” event from /v1/automate.

data: object { endpointIndex, iterationId, startingUrl, 2 more }

Event data when the browser reconnects after a mid-task disconnect

endpointIndex: number

1-based index of the CDP endpoint now in use

iterationId: string

startingUrl: string

The original starting URL the agent is restarting execution from

timestamp: number

total: number

Total number of configured CDP endpoints

event: "browser:reconnected"

BrowserScreenshotCaptured object { data, event }

Envelope for the “browser:screenshot_captured” event from /v1/automate.

data: object { format, iterationId, size, timestamp }

Event data for screenshot capture

format: "jpeg" or "png"

One of the following:

"jpeg"

"png"

iterationId: string

size: number

timestamp: number

event: "browser:screenshot_captured"

BrowserScreenshotCapturedImage object { data, event }

Envelope for the “browser:screenshot_captured_image” event from /v1/automate.

data: object { image, iterationId, mediaType, timestamp }

Event data for screenshot image capture with full image data This event contains the complete screenshot and can be very large

image: string

iterationId: string

mediaType: "image/jpeg" or "image/png"

One of the following:

"image/jpeg"

"image/png"

timestamp: number

event: "browser:screenshot_captured_image"

CdpEndpointConnected object { data, event }

Envelope for the “cdp:endpoint_connected” event from /v1/automate.

data: object { endpointIndex, iterationId, timestamp, total }

Event data when a CDP endpoint is successfully connected to

endpointIndex: number

1-based index of the endpoint that connected

iterationId: string

timestamp: number

total: number

Total number of configured CDP endpoints

event: "cdp:endpoint_connected"

CdpEndpointCycle object { data, event }

Envelope for the “cdp:endpoint_cycle” event from /v1/automate.

data: object { attempt, error, iterationId, 2 more }

Event data when a CDP endpoint fails and the next one is being tried

attempt: number

1-based index of the endpoint attempt that failed

error: string

Sanitized error identifier from the failed connection attempt (error.name, not error.message — full messages may contain endpoint URLs)

iterationId: string

timestamp: number

total: number

Total number of configured CDP endpoints

event: "cdp:endpoint_cycle"

Complete object { data, event }

Envelope for the “complete” event from /v1/automate.

data: object { finalAnswer, stats, success, error }

finalAnswer: string

Final answer or result from the agent

stats: object { actions, durationMs, endTime, 2 more }

Execution statistics

actions: number

durationMs: number

endTime: number

iterations: number

startTime: number

success: boolean

Whether the task completed successfully

error: optional object { code, message }

Structured error information for failed tasks

code: "TASK_ABORTED" or "MAX_ITERATIONS" or "MAX_ERRORS" or "TASK_FAILED"

Error codes for task failures

One of the following:

"TASK_ABORTED"

"MAX_ITERATIONS"

"MAX_ERRORS"

"TASK_FAILED"

message: string

Human-readable error message

event: "complete"

Done object { data, event }

Envelope for the “done” event from /v1/automate.

data: map[unknown]

Payload for the done stream terminator event. Empty today; reserved for future metadata.

event: "done"

Error object { data, event }

Envelope for the “error” event from /v1/automate.

data: object { error, success }

error: object { code, message, timestamp }

code: string

message: string

timestamp: string

ISO-8601 timestamp

success: false

event: "error"

InteractiveFormDataError object { data, event }

Envelope for the “interactive:form_data:error” event from /v1/automate.

data: object { fieldErrors, fields, formDescription, 5 more }

Event data when form validation fails and the agent re-requests data. Carries both the error context and the fields that need new values. Callers respond to this the same way as a request event.

fieldErrors: map[string]

Per-field error messages from validation (field ref -> error text)

fields: array of object { fieldType, label, ref, 4 more }

fieldType: "text" or "email" or "phone" or 8 more

Semantic field type

One of the following:

"text"

"email"

"phone"

"date"

"number"

"select"

"checkbox"

"radio"

"textarea"

"password"

"other"

label: string

The field’s visible label

ref: string

Element ref from the accessibility tree (e.g., “E42”)

required: boolean

Whether this field must be filled

currentValue: optional string

Current value if already partially filled

description: optional string

Additional context (e.g., validation error message on re-request)

options: optional array of string

Available options for select/radio fields

formDescription: string

iterationId: string

pageTitle: string

pageUrl: string

requestId: string

timestamp: number

event: "interactive:form_data:error"

InteractiveFormDataRequest object { data, event }

Envelope for the “interactive:form_data:request” event from /v1/automate.

data: object { fields, formDescription, iterationId, 4 more }

Event data when the agent requests user data for form fields

fields: array of object { fieldType, label, ref, 4 more }

fieldType: "text" or "email" or "phone" or 8 more

Semantic field type

One of the following:

"text"

"email"

"phone"

"date"

"number"

"select"

"checkbox"

"radio"

"textarea"

"password"

"other"

label: string

The field’s visible label

ref: string

Element ref from the accessibility tree (e.g., “E42”)

required: boolean

Whether this field must be filled

currentValue: optional string

Current value if already partially filled

description: optional string

Additional context (e.g., validation error message on re-request)

options: optional array of string

Available options for select/radio fields

formDescription: string

iterationId: string

pageTitle: string

pageUrl: string

requestId: string

timestamp: number

event: "interactive:form_data:request"

SystemDebugCompression object { data, event }

Envelope for the “system:debug_compression” event from /v1/automate.

data: object { compressedSize, compressionPercent, iterationId, 2 more }

Event data for compression debug info

compressedSize: number

compressionPercent: number

iterationId: string

originalSize: number

timestamp: number

event: "system:debug_compression"

SystemDebugMessage object { data, event }

Envelope for the “system:debug_message” event from /v1/automate.

data: object { iterationId, messages, timestamp }

Event data for message debug info

iterationId: string

messages: array of unknown

timestamp: number

event: "system:debug_message"

TaskAborted object { data, event }

Envelope for the “task:aborted” event from /v1/automate.

data: object { finalAnswer, iterationId, reason, timestamp }

Event data when a task is aborted

finalAnswer: string

iterationId: string

reason: string

timestamp: number

event: "task:aborted"

TaskCompleted object { data, event }

Envelope for the “task:completed” event from /v1/automate.

data: object { finalAnswer, iterationId, timestamp, success }

Event data when a task is completed

finalAnswer: string

iterationId: string

timestamp: number

success: optional boolean

event: "task:completed"

TaskMetrics object { data, event }

Envelope for the “task:metrics” event from /v1/automate.

data: object { aiGenerationCount, aiGenerationErrorCount, eventCounts, 5 more }

aiGenerationCount: number

aiGenerationErrorCount: number

eventCounts: map[number]

iterationId: string

stepCount: number

timestamp: number

totalInputTokens: number

totalOutputTokens: number

event: "task:metrics"

TaskMetricsIncremental object { data, event }

Envelope for the “task:metrics_incremental” event from /v1/automate.

data: object { aiGenerationCount, aiGenerationErrorCount, eventCounts, 5 more }

aiGenerationCount: number

aiGenerationErrorCount: number

eventCounts: map[number]

iterationId: string

stepCount: number

timestamp: number

totalInputTokens: number

totalOutputTokens: number

event: "task:metrics_incremental"

TaskSetup object { data, event }

Envelope for the “task:setup” event from /v1/automate.

data: object { browserName, iterationId, task, 14 more }

Event data when a task is setup

browserName: string

iterationId: string

task: string

timestamp: number

data: optional unknown

guardrails: optional string

hasApiKey: optional boolean

keySource: optional "global" or "env" or "not_set"

One of the following:

"global"

"env"

"not_set"

model: optional string

provider: optional string

proxy: optional string

pwCdpEndpoint: optional string

pwCdpEndpointCount: optional number

Total number of CDP endpoints configured (index, not URLs)

pwCdpEndpoints: optional array of string

pwEndpoint: optional string

url: optional string

vision: optional boolean

event: "task:setup"

TaskStarted object { data, event }

Envelope for the “task:started” event from /v1/automate.

data: object { iterationId, plan, successCriteria, 4 more }

Event data when a task is started

iterationId: string

plan: string

successCriteria: string

task: string

timestamp: number

url: string

actionItems: optional array of string

event: "task:started"

TaskTraceContext object { data, event }

Envelope for the “task:trace_context” event from /v1/automate.

data: object { traceId }

traceId: string

W3C trace ID — 32-character lowercase hexadecimal string.

event: "task:trace_context"

TaskValidated object { data, event }

Envelope for the “task:validated” event from /v1/automate.

data: object { completionQuality, finalAnswer, iterationId, 3 more }

Event data for task validation

completionQuality: "failed" or "partial" or "complete" or "excellent"

One of the following:

"failed"

"partial"

"complete"

"excellent"

finalAnswer: string

iterationId: string

observation: string

timestamp: number

feedback: optional string

event: "task:validated"

TaskValidationError object { data, event }

Envelope for the “task:validation_error” event from /v1/automate.

data: object { errors, iterationId, rawResponse, 2 more }

Event data for validation errors during action response processing

errors: array of string

iterationId: string

rawResponse: unknown

retryCount: number

timestamp: number

event: "task:validation_error"

AI Task

curl https://api.tabstack.ai/v1/automate \
    -H 'Content-Type: application/json' \
    -H "Authorization: Bearer $TABSTACK_API_KEY" \
    -d "{
          \"task\": \"Find the top 3 trending repositories and extract their names, descriptions, and star counts\",
          \"guardrails\": \"browse and extract only, don't interact with repositories\",
          \"maxIterations\": 50,
          \"maxValidationAttempts\": 3,
          \"url\": \"https://github.com/trending\"
        }"

event: start data: {"task": "Find the top 3 trending repositories", "url": "https://github.com/trending"} event: agent:processing data: {"operation": "Creating task plan", "hasScreenshot": false} event: browser:navigated data: {"title": "Trending - GitHub", "url": "https://github.com/trending"} event: agent:extracted data: {"extractedData": "[{\"name\": \"awesome-ai\", \"stars\": \"45.2k\"}]"} event: task:completed data: {"success": true, "finalAnswer": "Top 3 trending repos extracted"} event: complete data: {"success": true} event: done data: {}

event: start data: {"task": "Find product details", "url": "https://store.com/product"} event: browser:navigated data: {"title": "Product Page", "url": "https://store.com/product"} event: agent:extracted data: {"extractedData": "{\"name\": \"Headphones\", \"price\": \"$299.99\"}"} event: task:completed data: {"success": true} event: done data: {}

{
  "error": "maxIterations must be between 1 and 100"
}

{
  "error": "invalid URL format"
}

{
  "error": "task is required"
}

Returns Examples

{
  "error": "maxIterations must be between 1 and 100"
}

{
  "error": "invalid URL format"
}

{
  "error": "task is required"
}