API Reference

Agent

Copy Markdown

Open in Claude

Open in ChatGPT

Open in Cursor

---

Copy Markdown

View as Markdown

AI Task

agent.automate(AgentAutomateParams**kwargs) -> AutomateEvent

POST/automate

Execute AI-powered browser automation tasks using natural language with optional geotargeting. This endpoint always streams responses using Server-Sent Events (SSE).

Streaming Response:

• All responses are streamed using Server-Sent Events (text/event-stream)
• Real-time progress updates and results as they’re generated

Geotargeting:

• Optionally specify a country code for geotargeted browsing

Use Cases:

• Web scraping and data extraction
• Form filling and interaction
• Navigation and information gathering
• Multi-step web workflows
• Content analysis from web pages

ParametersExpand Collapse

task: str

The task description in natural language

data: Optional[object]

JSON data to provide context for form filling or complex tasks

geo_target: Optional[GeoTarget]

Optional geotargeting parameters for proxy requests

country: Optional[str]

Country code using ISO 3166-1 alpha-2 standard (2 letters, e.g., “US”, “GB”, “JP”). See: https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2

guardrails: Optional[str]

Safety constraints for execution

interactive: Optional[bool]

Enable interactive mode to allow human-in-the-loop input during task execution

max_iterations: Optional[int]

Maximum task iterations

maximum100

minimum1

max_validation_attempts: Optional[int]

Maximum validation attempts

maximum10

minimum1

url: Optional[str]

Starting URL for the task

formaturi

ReturnsExpand Collapse

AutomateEvent

A Server-Sent Event from /v1/automate. Typed discriminated union keyed on event.

One of the following:

class V1AutomateEventAgentAction: …

Envelope for the “agent:action” event from /v1/automate.

data: V1AutomateEventAgentActionData

Event data for action execution

action: str

iteration_id: str

timestamp: float

ref: Optional[str]

value: Optional[str]

event: Literal["agent:action"]

class V1AutomateEventAgentExtracted: …

Envelope for the “agent:extracted” event from /v1/automate.

data: V1AutomateEventAgentExtractedData

Event data for extracted data

extracted_data: str

iteration_id: str

timestamp: float

event: Literal["agent:extracted"]

class V1AutomateEventAgentProcessing: …

Envelope for the “agent:processing” event from /v1/automate.

data: V1AutomateEventAgentProcessingData

Event data for when the agent is waiting for model generation

has_screenshot: bool

iteration_id: str

operation: str

timestamp: float

event: Literal["agent:processing"]

class V1AutomateEventAgentReasoned: …

Envelope for the “agent:reasoned” event from /v1/automate.

data: V1AutomateEventAgentReasonedData

Event data for agent reasoning

iteration_id: str

reasoning: str

timestamp: float

event: Literal["agent:reasoned"]

class V1AutomateEventAgentStatus: …

Envelope for the “agent:status” event from /v1/automate.

data: V1AutomateEventAgentStatusData

Event data for status messages

iteration_id: str

message: str

timestamp: float

event: Literal["agent:status"]

class V1AutomateEventAgentStep: …

Envelope for the “agent:step” event from /v1/automate.

data: V1AutomateEventAgentStepData

Event data for agent step tracking (each loop iteration)

current_iteration: float

iteration_id: str

timestamp: float

event: Literal["agent:step"]

class V1AutomateEventAgentWaiting: …

Envelope for the “agent:waiting” event from /v1/automate.

data: V1AutomateEventAgentWaitingData

Event data for waiting notifications

iteration_id: str

seconds: float

timestamp: float

event: Literal["agent:waiting"]

class V1AutomateEventAIGeneration: …

Envelope for the “ai:generation” event from /v1/automate.

data: V1AutomateEventAIGenerationData

Event data when AI generation occurs

finish_reason: Literal["stop", "length", "content-filter", 3 more]

One of the following:

"stop"

"length"

"content-filter"

"tool-calls"

"error"

"other"

iteration_id: str

prompt: str

schema: object

timestamp: float

usage: V1AutomateEventAIGenerationDataUsage

input_tokens: Optional[float]

output_tokens: Optional[float]

total_tokens: Optional[float]

messages: Optional[List[V1AutomateEventAIGenerationDataMessage]]

One of the following:

class V1AutomateEventAIGenerationDataMessageSystem: …

A system message. It can contain system information.

Note: using the “system” part of the prompt is strongly preferred to increase the resilience against prompt injection attacks, and because not all providers support several system messages.

content: str

role: Literal["system"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageUser: …

A user message. It can contain text or a combination of text and images.

content: Union[str, List[V1AutomateEventAIGenerationDataMessageUserContentUnionMember1]]

Content of a user message. It can be a string or an array of text and image parts.

One of the following:

str

List[V1AutomateEventAIGenerationDataMessageUserContentUnionMember1]

One of the following:

class V1AutomateEventAIGenerationDataMessageUserContentUnionMember1Text: …

Text content part of a prompt. It contains a string of text.

text: str

The text content.

type: Literal["text"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageUserContentUnionMember1Image: …

Image content part of a prompt. It contains an image.

image: V1AutomateEventAIGenerationDataMessageUserContentUnionMember1ImageImage

Image data. Can either be:

• data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
• URL: a URL that points to the image

One of the following:

str

class V1AutomateEventAIGenerationDataMessageUserContentUnionMember1ImageImageUnionMember1: …

buffer: V1AutomateEventAIGenerationDataMessageUserContentUnionMember1ImageImageUnionMember1Buffer

byte_length: float

byte_length: float

byte_offset: float

bytes_per_element: float

length: float

class V1AutomateEventAIGenerationDataMessageUserContentUnionMember1ImageImageByteLength: …

byte_length: float

class V1AutomateEventAIGenerationDataMessageUserContentUnionMember1ImageImageV1GlobalBuffer: …

buffer: V1AutomateEventAIGenerationDataMessageUserContentUnionMember1ImageImageV1GlobalBufferBuffer

byte_length: float

byte_length: float

byte_offset: float

bytes_per_element: float

length: float

type: Literal["image"]

media_type: Optional[str]

Optional IANA media type of the image.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageUserContentUnionMember1File: …

File content part of a prompt. It contains a file.

data: V1AutomateEventAIGenerationDataMessageUserContentUnionMember1FileData

File data. Can either be:

• data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
• URL: a URL that points to the image

One of the following:

str

class V1AutomateEventAIGenerationDataMessageUserContentUnionMember1FileDataUnionMember1: …

buffer: V1AutomateEventAIGenerationDataMessageUserContentUnionMember1FileDataUnionMember1Buffer

byte_length: float

byte_length: float

byte_offset: float

bytes_per_element: float

length: float

class V1AutomateEventAIGenerationDataMessageUserContentUnionMember1FileDataByteLength: …

byte_length: float

class V1AutomateEventAIGenerationDataMessageUserContentUnionMember1FileDataV1GlobalBuffer: …

buffer: V1AutomateEventAIGenerationDataMessageUserContentUnionMember1FileDataV1GlobalBufferBuffer

byte_length: float

byte_length: float

byte_offset: float

bytes_per_element: float

length: float

media_type: str

IANA media type of the file.

type: Literal["file"]

filename: Optional[str]

Optional filename of the file.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

role: Literal["user"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistant: …

An assistant message. It can contain text, tool calls, or a combination of text and tool calls.

content: Union[str, List[V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1]]

Content of an assistant message. It can be a string or an array of text, image, reasoning, redacted reasoning, and tool call parts.

One of the following:

str

List[V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1]

One of the following:

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1Text: …

Text content part of a prompt. It contains a string of text.

text: str

The text content.

type: Literal["text"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1File: …

File content part of a prompt. It contains a file.

data: V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1FileData

File data. Can either be:

• data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
• URL: a URL that points to the image

One of the following:

str

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1FileDataUnionMember1: …

buffer: V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1FileDataUnionMember1Buffer

byte_length: float

byte_length: float

byte_offset: float

bytes_per_element: float

length: float

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1FileDataByteLength: …

byte_length: float

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1FileDataV1GlobalBuffer: …

buffer: V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1FileDataV1GlobalBufferBuffer

byte_length: float

byte_length: float

byte_offset: float

bytes_per_element: float

length: float

media_type: str

IANA media type of the file.

type: Literal["file"]

filename: Optional[str]

Optional filename of the file.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1Reasoning: …

Reasoning content part of a prompt. It contains a reasoning.

text: str

The reasoning text.

type: Literal["reasoning"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolCall: …

Tool call content part of a prompt. It contains a tool call (usually generated by the AI model).

input: object

Arguments of the tool call. This is a JSON-serializable object that matches the tool’s input schema.

tool_call_id: str

ID of the tool call. This ID is used to match the tool call with the tool result.

tool_name: str

Name of the tool that is being called.

type: Literal["tool-call"]

provider_executed: Optional[bool]

Whether the tool call was executed by the provider.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResult: …

Tool result content part of a prompt. It contains the result of the tool call with the matching ID.

output: V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutput

Result of the tool call. This is a JSON-serializable object.

One of the following:

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputText: …

type: Literal["text"]

Text tool output that should be directly sent to the API.

value: str

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputJson: …

type: Literal["json"]

value: Union[str, float, bool, 3 more]

A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputExecutionDenied: …

type: Literal["execution-denied"]

Type when the user has denied the execution of the tool call.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

reason: Optional[str]

Optional reason for the execution denial.

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputErrorText: …

type: Literal["error-text"]

value: str

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputErrorJson: …

type: Literal["error-json"]

value: Union[str, float, bool, 3 more]

A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputContent: …

type: Literal["content"]

value: List[V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputContentValue]

One of the following:

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputContentValueText: …

text: str

Text content.

type: Literal["text"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputContentValueMedia: …

data: str

media_type: str

Deprecatedtype: Literal["media"]

Deprecated by the upstream schema.

Deprecated. Use image-data or file-data instead.

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputContentValueFileData: …

data: str

Base-64 encoded media data.

media_type: str

IANA media type.

type: Literal["file-data"]

filename: Optional[str]

Optional filename of the file.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputContentValueFileURL: …

type: Literal["file-url"]

url: str

URL of the file.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputContentValueFileID: …

file_id: Union[str, Dict[str, str]]

ID of the file.

If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.

One of the following:

str

Dict[str, str]

type: Literal["file-id"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputContentValueImageData: …

data: str

Base-64 encoded image data.

media_type: str

IANA media type.

type: Literal["image-data"]

Images that are referenced using base64 encoded data.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputContentValueImageURL: …

type: Literal["image-url"]

Images that are referenced using a URL.

url: str

URL of the image.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputContentValueImageFileID: …

file_id: Union[str, Dict[str, str]]

Image that is referenced using a provider file id.

If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.

One of the following:

str

Dict[str, str]

type: Literal["image-file-id"]

Images that are referenced using a provider file id.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputContentValueCustom: …

type: Literal["custom"]

Custom content part. This can be used to implement provider-specific content parts.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

tool_call_id: str

ID of the tool call that this result is associated with.

tool_name: str

Name of the tool that generated this result.

type: Literal["tool-result"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolApprovalRequest: …

Tool approval request prompt part.

approval_id: str

ID of the tool approval.

tool_call_id: str

ID of the tool call that the approval request is for.

type: Literal["tool-approval-request"]

role: Literal["assistant"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageTool: …

A tool message. It contains the result of one or more tool calls.

content: List[V1AutomateEventAIGenerationDataMessageToolContent]

Content of a tool message. It is an array of tool result parts.

One of the following:

class V1AutomateEventAIGenerationDataMessageToolContentToolResult: …

Tool result content part of a prompt. It contains the result of the tool call with the matching ID.

output: V1AutomateEventAIGenerationDataMessageToolContentToolResultOutput

Result of the tool call. This is a JSON-serializable object.

One of the following:

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputText: …

type: Literal["text"]

Text tool output that should be directly sent to the API.

value: str

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputJson: …

type: Literal["json"]

value: Union[str, float, bool, 3 more]

A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputExecutionDenied: …

type: Literal["execution-denied"]

Type when the user has denied the execution of the tool call.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

reason: Optional[str]

Optional reason for the execution denial.

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputErrorText: …

type: Literal["error-text"]

value: str

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputErrorJson: …

type: Literal["error-json"]

value: Union[str, float, bool, 3 more]

A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputContent: …

type: Literal["content"]

value: List[V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputContentValue]

One of the following:

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputContentValueText: …

text: str

Text content.

type: Literal["text"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputContentValueMedia: …

data: str

media_type: str

Deprecatedtype: Literal["media"]

Deprecated by the upstream schema.

Deprecated. Use image-data or file-data instead.

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputContentValueFileData: …

data: str

Base-64 encoded media data.

media_type: str

IANA media type.

type: Literal["file-data"]

filename: Optional[str]

Optional filename of the file.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputContentValueFileURL: …

type: Literal["file-url"]

url: str

URL of the file.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputContentValueFileID: …

file_id: Union[str, Dict[str, str]]

ID of the file.

If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.

One of the following:

str

Dict[str, str]

type: Literal["file-id"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputContentValueImageData: …

data: str

Base-64 encoded image data.

media_type: str

IANA media type.

type: Literal["image-data"]

Images that are referenced using base64 encoded data.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputContentValueImageURL: …

type: Literal["image-url"]

Images that are referenced using a URL.

url: str

URL of the image.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputContentValueImageFileID: …

file_id: Union[str, Dict[str, str]]

Image that is referenced using a provider file id.

If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.

One of the following:

str

Dict[str, str]

type: Literal["image-file-id"]

Images that are referenced using a provider file id.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputContentValueCustom: …

type: Literal["custom"]

Custom content part. This can be used to implement provider-specific content parts.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

tool_call_id: str

ID of the tool call that this result is associated with.

tool_name: str

Name of the tool that generated this result.

type: Literal["tool-result"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageToolContentToolApprovalResponse: …

Tool approval response prompt part.

approval_id: str

ID of the tool approval.

approved: bool

Flag indicating whether the approval was granted or denied.

type: Literal["tool-approval-response"]

provider_executed: Optional[bool]

Flag indicating whether the tool call is provider-executed. Only provider-executed tool approval responses should be sent to the model.

reason: Optional[str]

Optional reason for the approval or denial.

role: Literal["tool"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

object: Optional[object]

provider_metadata: Optional[Dict[str, object]]

temperature: Optional[float]

warnings: Optional[List[object]]

event: Literal["ai:generation"]

class V1AutomateEventAIGenerationError: …

Envelope for the “ai:generation:error” event from /v1/automate.

data: V1AutomateEventAIGenerationErrorData

Event data when AI generation error occurs

error: str

iteration_id: str

prompt: str

schema: object

timestamp: float

messages: Optional[List[object]]

event: Literal["ai:generation:error"]

class V1AutomateEventBrowserActionCompleted: …

Envelope for the “browser:action_completed” event from /v1/automate.

data: V1AutomateEventBrowserActionCompletedData

Event data for action results

iteration_id: str

success: bool

timestamp: float

error: Optional[str]

event: Literal["browser:action_completed"]

class V1AutomateEventBrowserActionStarted: …

Envelope for the “browser:action_started” event from /v1/automate.

data: V1AutomateEventBrowserActionStartedData

Event data for action execution

action: str

iteration_id: str

timestamp: float

ref: Optional[str]

value: Optional[str]

event: Literal["browser:action_started"]

class V1AutomateEventBrowserNavigated: …

Envelope for the “browser:navigated” event from /v1/automate.

data: V1AutomateEventBrowserNavigatedData

Event data when navigating to a page

iteration_id: str

timestamp: float

title: str

url: str

event: Literal["browser:navigated"]

class V1AutomateEventBrowserReconnected: …

Envelope for the “browser:reconnected” event from /v1/automate.

data: V1AutomateEventBrowserReconnectedData

Event data when the browser reconnects after a mid-task disconnect

endpoint_index: float

1-based index of the CDP endpoint now in use

iteration_id: str

starting_url: str

The original starting URL the agent is restarting execution from

timestamp: float

total: float

Total number of configured CDP endpoints

event: Literal["browser:reconnected"]

class V1AutomateEventBrowserScreenshotCaptured: …

Envelope for the “browser:screenshot_captured” event from /v1/automate.

data: V1AutomateEventBrowserScreenshotCapturedData

Event data for screenshot capture

format: Literal["jpeg", "png"]

One of the following:

"jpeg"

"png"

iteration_id: str

size: float

timestamp: float

event: Literal["browser:screenshot_captured"]

class V1AutomateEventBrowserScreenshotCapturedImage: …

Envelope for the “browser:screenshot_captured_image” event from /v1/automate.

data: V1AutomateEventBrowserScreenshotCapturedImageData

Event data for screenshot image capture with full image data This event contains the complete screenshot and can be very large

image: str

iteration_id: str

media_type: Literal["image/jpeg", "image/png"]

One of the following:

"image/jpeg"

"image/png"

timestamp: float

event: Literal["browser:screenshot_captured_image"]

class V1AutomateEventCdpEndpointConnected: …

Envelope for the “cdp:endpoint_connected” event from /v1/automate.

data: V1AutomateEventCdpEndpointConnectedData

Event data when a CDP endpoint is successfully connected to

endpoint_index: float

1-based index of the endpoint that connected

iteration_id: str

timestamp: float

total: float

Total number of configured CDP endpoints

event: Literal["cdp:endpoint_connected"]

class V1AutomateEventCdpEndpointCycle: …

Envelope for the “cdp:endpoint_cycle” event from /v1/automate.

data: V1AutomateEventCdpEndpointCycleData

Event data when a CDP endpoint fails and the next one is being tried

attempt: float

1-based index of the endpoint attempt that failed

error: str

Sanitized error identifier from the failed connection attempt (error.name, not error.message — full messages may contain endpoint URLs)

iteration_id: str

timestamp: float

total: float

Total number of configured CDP endpoints

event: Literal["cdp:endpoint_cycle"]

class V1AutomateEventComplete: …

Envelope for the “complete” event from /v1/automate.

data: V1AutomateEventCompleteData

Payload for the complete stream event. Structurally identical to TaskExecutionResult from webAgent.ts — the complete event’s data is the agent’s final TaskExecutionResult, stringified onto the SSE stream.

final_answer: Optional[str]

Final answer or result from the agent

stats: V1AutomateEventCompleteDataStats

Execution statistics

actions: float

duration_ms: float

end_time: float

iterations: float

start_time: float

success: bool

Whether the task completed successfully

error: Optional[V1AutomateEventCompleteDataError]

Structured error information for failed tasks

code: Literal["TASK_ABORTED", "MAX_ITERATIONS", "MAX_ERRORS", "TASK_FAILED"]

Error codes for task failures

One of the following:

"TASK_ABORTED"

"MAX_ITERATIONS"

"MAX_ERRORS"

"TASK_FAILED"

message: str

Human-readable error message

event: Literal["complete"]

class V1AutomateEventDone: …

Envelope for the “done” event from /v1/automate.

data: Dict[str, object]

Payload for the done stream terminator event. Empty today; reserved for future metadata.

event: Literal["done"]

class V1AutomateEventError: …

Envelope for the “error” event from /v1/automate.

data: V1AutomateEventErrorData

Payload for the top-level error stream event. Emitted when an uncaught error escapes the task runner. Mirrors ErrorResponse from the server package’s taskRunner.ts — kept structurally aligned so schema and runtime stay consistent. Distinct from agent-level error events like ai:generation:error and task:validation_error, which are emitted through the normal event emitter during the agent loop.

error: V1AutomateEventErrorDataError

code: str

message: str

timestamp: str

ISO-8601 timestamp

success: Literal[false]

event: Literal["error"]

class V1AutomateEventInteractiveFormDataError: …

Envelope for the “interactive:form_data:error” event from /v1/automate.

data: V1AutomateEventInteractiveFormDataErrorData

Event data when form validation fails and the agent re-requests data. Carries both the error context and the fields that need new values. Callers respond to this the same way as a request event.

field_errors: Dict[str, str]

Per-field error messages from validation (field ref -> error text)

fields: List[V1AutomateEventInteractiveFormDataErrorDataField]

field_type: Literal["text", "email", "phone", 8 more]

Semantic field type

One of the following:

"text"

"email"

"phone"

"date"

"number"

"select"

"checkbox"

"radio"

"textarea"

"password"

"other"

label: str

The field’s visible label

ref: str

Element ref from the accessibility tree (e.g., “E42”)

required: bool

Whether this field must be filled

current_value: Optional[str]

Current value if already partially filled

description: Optional[str]

Additional context (e.g., validation error message on re-request)

options: Optional[List[str]]

Available options for select/radio fields

form_description: str

iteration_id: str

page_title: str

page_url: str

request_id: str

timestamp: float

event: Literal["interactive:form_data:error"]

class V1AutomateEventInteractiveFormDataRequest: …

Envelope for the “interactive:form_data:request” event from /v1/automate.

data: V1AutomateEventInteractiveFormDataRequestData

Event data when the agent requests user data for form fields

fields: List[V1AutomateEventInteractiveFormDataRequestDataField]

field_type: Literal["text", "email", "phone", 8 more]

Semantic field type

One of the following:

"text"

"email"

"phone"

"date"

"number"

"select"

"checkbox"

"radio"

"textarea"

"password"

"other"

label: str

The field’s visible label

ref: str

Element ref from the accessibility tree (e.g., “E42”)

required: bool

Whether this field must be filled

current_value: Optional[str]

Current value if already partially filled

description: Optional[str]

Additional context (e.g., validation error message on re-request)

options: Optional[List[str]]

Available options for select/radio fields

form_description: str

iteration_id: str

page_title: str

page_url: str

request_id: str

timestamp: float

event: Literal["interactive:form_data:request"]

class V1AutomateEventSystemDebugCompression: …

Envelope for the “system:debug_compression” event from /v1/automate.

data: V1AutomateEventSystemDebugCompressionData

Event data for compression debug info

compressed_size: float

compression_percent: float

iteration_id: str

original_size: float

timestamp: float

event: Literal["system:debug_compression"]

class V1AutomateEventSystemDebugMessage: …

Envelope for the “system:debug_message” event from /v1/automate.

data: V1AutomateEventSystemDebugMessageData

Event data for message debug info

iteration_id: str

messages: List[object]

timestamp: float

event: Literal["system:debug_message"]

class V1AutomateEventTaskAborted: …

Envelope for the “task:aborted” event from /v1/automate.

data: V1AutomateEventTaskAbortedData

Event data when a task is aborted

final_answer: str

iteration_id: str

reason: str

timestamp: float

event: Literal["task:aborted"]

class V1AutomateEventTaskCompleted: …

Envelope for the “task:completed” event from /v1/automate.

data: V1AutomateEventTaskCompletedData

Event data when a task is completed

final_answer: Optional[str]

iteration_id: str

timestamp: float

success: Optional[bool]

event: Literal["task:completed"]

class V1AutomateEventTaskMetrics: …

Envelope for the “task:metrics” event from /v1/automate.

data: V1AutomateEventTaskMetricsData

ai_generation_count: float

ai_generation_error_count: float

event_counts: Dict[str, float]

iteration_id: str

step_count: float

timestamp: float

total_input_tokens: float

total_output_tokens: float

event: Literal["task:metrics"]

class V1AutomateEventTaskMetricsIncremental: …

Envelope for the “task:metrics_incremental” event from /v1/automate.

data: V1AutomateEventTaskMetricsIncrementalData

ai_generation_count: float

ai_generation_error_count: float

event_counts: Dict[str, float]

iteration_id: str

step_count: float

timestamp: float

total_input_tokens: float

total_output_tokens: float

event: Literal["task:metrics_incremental"]

class V1AutomateEventTaskSetup: …

Envelope for the “task:setup” event from /v1/automate.

data: V1AutomateEventTaskSetupData

Event data when a task is setup

browser_name: str

iteration_id: str

task: str

timestamp: float

data: Optional[object]

guardrails: Optional[str]

has_api_key: Optional[bool]

key_source: Optional[Literal["global", "env", "not_set"]]

One of the following:

"global"

"env"

"not_set"

model: Optional[str]

provider: Optional[str]

proxy: Optional[str]

pw_cdp_endpoint: Optional[str]

pw_cdp_endpoint_count: Optional[float]

Total number of CDP endpoints configured (index, not URLs)

pw_cdp_endpoints: Optional[List[str]]

pw_endpoint: Optional[str]

url: Optional[str]

vision: Optional[bool]

event: Literal["task:setup"]

class V1AutomateEventTaskStarted: …

Envelope for the “task:started” event from /v1/automate.

data: V1AutomateEventTaskStartedData

Event data when a task is started

iteration_id: str

plan: str

success_criteria: str

task: str

timestamp: float

url: str

action_items: Optional[List[str]]

event: Literal["task:started"]

class V1AutomateEventTaskTraceContext: …

Envelope for the “task:trace_context” event from /v1/automate.

data: V1AutomateEventTaskTraceContextData

Payload for the task:trace_context event. Carries the OpenTelemetry trace ID for this /v1/automate request so consumers can deep-link to distributed-tracing UIs (e.g. Cloud Trace, Cloud Logging) for the run.

trace_id: str

W3C trace ID — 32-character lowercase hexadecimal string.

event: Literal["task:trace_context"]

class V1AutomateEventTaskValidated: …

Envelope for the “task:validated” event from /v1/automate.

data: V1AutomateEventTaskValidatedData

Event data for task validation

completion_quality: Literal["failed", "partial", "complete", "excellent"]

One of the following:

"failed"

"partial"

"complete"

"excellent"

final_answer: str

iteration_id: str

observation: str

timestamp: float

feedback: Optional[str]

event: Literal["task:validated"]

class V1AutomateEventTaskValidationError: …

Envelope for the “task:validation_error” event from /v1/automate.

data: V1AutomateEventTaskValidationErrorData

Event data for validation errors during action response processing

errors: List[str]

iteration_id: str

raw_response: object

retry_count: float

timestamp: float

event: Literal["task:validation_error"]

AutomateEvent

A Server-Sent Event from /v1/automate. Typed discriminated union keyed on event.

One of the following:

class V1AutomateEventAgentAction: …

Envelope for the “agent:action” event from /v1/automate.

data: V1AutomateEventAgentActionData

Event data for action execution

action: str

iteration_id: str

timestamp: float

ref: Optional[str]

value: Optional[str]

event: Literal["agent:action"]

class V1AutomateEventAgentExtracted: …

Envelope for the “agent:extracted” event from /v1/automate.

data: V1AutomateEventAgentExtractedData

Event data for extracted data

extracted_data: str

iteration_id: str

timestamp: float

event: Literal["agent:extracted"]

class V1AutomateEventAgentProcessing: …

Envelope for the “agent:processing” event from /v1/automate.

data: V1AutomateEventAgentProcessingData

Event data for when the agent is waiting for model generation

has_screenshot: bool

iteration_id: str

operation: str

timestamp: float

event: Literal["agent:processing"]

class V1AutomateEventAgentReasoned: …

Envelope for the “agent:reasoned” event from /v1/automate.

data: V1AutomateEventAgentReasonedData

Event data for agent reasoning

iteration_id: str

reasoning: str

timestamp: float

event: Literal["agent:reasoned"]

class V1AutomateEventAgentStatus: …

Envelope for the “agent:status” event from /v1/automate.

data: V1AutomateEventAgentStatusData

Event data for status messages

iteration_id: str

message: str

timestamp: float

event: Literal["agent:status"]

class V1AutomateEventAgentStep: …

Envelope for the “agent:step” event from /v1/automate.

data: V1AutomateEventAgentStepData

Event data for agent step tracking (each loop iteration)

current_iteration: float

iteration_id: str

timestamp: float

event: Literal["agent:step"]

class V1AutomateEventAgentWaiting: …

Envelope for the “agent:waiting” event from /v1/automate.

data: V1AutomateEventAgentWaitingData

Event data for waiting notifications

iteration_id: str

seconds: float

timestamp: float

event: Literal["agent:waiting"]

class V1AutomateEventAIGeneration: …

Envelope for the “ai:generation” event from /v1/automate.

data: V1AutomateEventAIGenerationData

Event data when AI generation occurs

finish_reason: Literal["stop", "length", "content-filter", 3 more]

One of the following:

"stop"

"length"

"content-filter"

"tool-calls"

"error"

"other"

iteration_id: str

prompt: str

schema: object

timestamp: float

usage: V1AutomateEventAIGenerationDataUsage

input_tokens: Optional[float]

output_tokens: Optional[float]

total_tokens: Optional[float]

messages: Optional[List[V1AutomateEventAIGenerationDataMessage]]

One of the following:

class V1AutomateEventAIGenerationDataMessageSystem: …

A system message. It can contain system information.

Note: using the “system” part of the prompt is strongly preferred to increase the resilience against prompt injection attacks, and because not all providers support several system messages.

content: str

role: Literal["system"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageUser: …

A user message. It can contain text or a combination of text and images.

content: Union[str, List[V1AutomateEventAIGenerationDataMessageUserContentUnionMember1]]

Content of a user message. It can be a string or an array of text and image parts.

One of the following:

str

List[V1AutomateEventAIGenerationDataMessageUserContentUnionMember1]

One of the following:

class V1AutomateEventAIGenerationDataMessageUserContentUnionMember1Text: …

Text content part of a prompt. It contains a string of text.

text: str

The text content.

type: Literal["text"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageUserContentUnionMember1Image: …

Image content part of a prompt. It contains an image.

image: V1AutomateEventAIGenerationDataMessageUserContentUnionMember1ImageImage

Image data. Can either be:

• data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
• URL: a URL that points to the image

One of the following:

str

class V1AutomateEventAIGenerationDataMessageUserContentUnionMember1ImageImageUnionMember1: …

buffer: V1AutomateEventAIGenerationDataMessageUserContentUnionMember1ImageImageUnionMember1Buffer

byte_length: float

byte_length: float

byte_offset: float

bytes_per_element: float

length: float

class V1AutomateEventAIGenerationDataMessageUserContentUnionMember1ImageImageByteLength: …

byte_length: float

class V1AutomateEventAIGenerationDataMessageUserContentUnionMember1ImageImageV1GlobalBuffer: …

buffer: V1AutomateEventAIGenerationDataMessageUserContentUnionMember1ImageImageV1GlobalBufferBuffer

byte_length: float

byte_length: float

byte_offset: float

bytes_per_element: float

length: float

type: Literal["image"]

media_type: Optional[str]

Optional IANA media type of the image.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageUserContentUnionMember1File: …

File content part of a prompt. It contains a file.

data: V1AutomateEventAIGenerationDataMessageUserContentUnionMember1FileData

File data. Can either be:

• data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
• URL: a URL that points to the image

One of the following:

str

class V1AutomateEventAIGenerationDataMessageUserContentUnionMember1FileDataUnionMember1: …

buffer: V1AutomateEventAIGenerationDataMessageUserContentUnionMember1FileDataUnionMember1Buffer

byte_length: float

byte_length: float

byte_offset: float

bytes_per_element: float

length: float

class V1AutomateEventAIGenerationDataMessageUserContentUnionMember1FileDataByteLength: …

byte_length: float

class V1AutomateEventAIGenerationDataMessageUserContentUnionMember1FileDataV1GlobalBuffer: …

buffer: V1AutomateEventAIGenerationDataMessageUserContentUnionMember1FileDataV1GlobalBufferBuffer

byte_length: float

byte_length: float

byte_offset: float

bytes_per_element: float

length: float

media_type: str

IANA media type of the file.

type: Literal["file"]

filename: Optional[str]

Optional filename of the file.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

role: Literal["user"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistant: …

An assistant message. It can contain text, tool calls, or a combination of text and tool calls.

content: Union[str, List[V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1]]

Content of an assistant message. It can be a string or an array of text, image, reasoning, redacted reasoning, and tool call parts.

One of the following:

str

List[V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1]

One of the following:

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1Text: …

Text content part of a prompt. It contains a string of text.

text: str

The text content.

type: Literal["text"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1File: …

File content part of a prompt. It contains a file.

data: V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1FileData

File data. Can either be:

• data: a base64-encoded string, a Uint8Array, an ArrayBuffer, or a Buffer
• URL: a URL that points to the image

One of the following:

str

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1FileDataUnionMember1: …

buffer: V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1FileDataUnionMember1Buffer

byte_length: float

byte_length: float

byte_offset: float

bytes_per_element: float

length: float

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1FileDataByteLength: …

byte_length: float

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1FileDataV1GlobalBuffer: …

buffer: V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1FileDataV1GlobalBufferBuffer

byte_length: float

byte_length: float

byte_offset: float

bytes_per_element: float

length: float

media_type: str

IANA media type of the file.

type: Literal["file"]

filename: Optional[str]

Optional filename of the file.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1Reasoning: …

Reasoning content part of a prompt. It contains a reasoning.

text: str

The reasoning text.

type: Literal["reasoning"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolCall: …

Tool call content part of a prompt. It contains a tool call (usually generated by the AI model).

input: object

Arguments of the tool call. This is a JSON-serializable object that matches the tool’s input schema.

tool_call_id: str

ID of the tool call. This ID is used to match the tool call with the tool result.

tool_name: str

Name of the tool that is being called.

type: Literal["tool-call"]

provider_executed: Optional[bool]

Whether the tool call was executed by the provider.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResult: …

Tool result content part of a prompt. It contains the result of the tool call with the matching ID.

output: V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutput

Result of the tool call. This is a JSON-serializable object.

One of the following:

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputText: …

type: Literal["text"]

Text tool output that should be directly sent to the API.

value: str

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputJson: …

type: Literal["json"]

value: Union[str, float, bool, 3 more]

A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputExecutionDenied: …

type: Literal["execution-denied"]

Type when the user has denied the execution of the tool call.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

reason: Optional[str]

Optional reason for the execution denial.

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputErrorText: …

type: Literal["error-text"]

value: str

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputErrorJson: …

type: Literal["error-json"]

value: Union[str, float, bool, 3 more]

A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputContent: …

type: Literal["content"]

value: List[V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputContentValue]

One of the following:

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputContentValueText: …

text: str

Text content.

type: Literal["text"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputContentValueMedia: …

data: str

media_type: str

Deprecatedtype: Literal["media"]

Deprecated by the upstream schema.

Deprecated. Use image-data or file-data instead.

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputContentValueFileData: …

data: str

Base-64 encoded media data.

media_type: str

IANA media type.

type: Literal["file-data"]

filename: Optional[str]

Optional filename of the file.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputContentValueFileURL: …

type: Literal["file-url"]

url: str

URL of the file.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputContentValueFileID: …

file_id: Union[str, Dict[str, str]]

ID of the file.

If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.

One of the following:

str

Dict[str, str]

type: Literal["file-id"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputContentValueImageData: …

data: str

Base-64 encoded image data.

media_type: str

IANA media type.

type: Literal["image-data"]

Images that are referenced using base64 encoded data.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputContentValueImageURL: …

type: Literal["image-url"]

Images that are referenced using a URL.

url: str

URL of the image.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputContentValueImageFileID: …

file_id: Union[str, Dict[str, str]]

Image that is referenced using a provider file id.

If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.

One of the following:

str

Dict[str, str]

type: Literal["image-file-id"]

Images that are referenced using a provider file id.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolResultOutputContentValueCustom: …

type: Literal["custom"]

Custom content part. This can be used to implement provider-specific content parts.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

tool_call_id: str

ID of the tool call that this result is associated with.

tool_name: str

Name of the tool that generated this result.

type: Literal["tool-result"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageAssistantContentUnionMember1ToolApprovalRequest: …

Tool approval request prompt part.

approval_id: str

ID of the tool approval.

tool_call_id: str

ID of the tool call that the approval request is for.

type: Literal["tool-approval-request"]

role: Literal["assistant"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageTool: …

A tool message. It contains the result of one or more tool calls.

content: List[V1AutomateEventAIGenerationDataMessageToolContent]

Content of a tool message. It is an array of tool result parts.

One of the following:

class V1AutomateEventAIGenerationDataMessageToolContentToolResult: …

Tool result content part of a prompt. It contains the result of the tool call with the matching ID.

output: V1AutomateEventAIGenerationDataMessageToolContentToolResultOutput

Result of the tool call. This is a JSON-serializable object.

One of the following:

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputText: …

type: Literal["text"]

Text tool output that should be directly sent to the API.

value: str

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputJson: …

type: Literal["json"]

value: Union[str, float, bool, 3 more]

A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputExecutionDenied: …

type: Literal["execution-denied"]

Type when the user has denied the execution of the tool call.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

reason: Optional[str]

Optional reason for the execution denial.

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputErrorText: …

type: Literal["error-text"]

value: str

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputErrorJson: …

type: Literal["error-json"]

value: Union[str, float, bool, 3 more]

A JSON value can be a string, number, boolean, object, array, or null. JSON values can be serialized and deserialized by the JSON.stringify and JSON.parse methods.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputContent: …

type: Literal["content"]

value: List[V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputContentValue]

One of the following:

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputContentValueText: …

text: str

Text content.

type: Literal["text"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputContentValueMedia: …

data: str

media_type: str

Deprecatedtype: Literal["media"]

Deprecated by the upstream schema.

Deprecated. Use image-data or file-data instead.

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputContentValueFileData: …

data: str

Base-64 encoded media data.

media_type: str

IANA media type.

type: Literal["file-data"]

filename: Optional[str]

Optional filename of the file.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputContentValueFileURL: …

type: Literal["file-url"]

url: str

URL of the file.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputContentValueFileID: …

file_id: Union[str, Dict[str, str]]

ID of the file.

If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.

One of the following:

str

Dict[str, str]

type: Literal["file-id"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputContentValueImageData: …

data: str

Base-64 encoded image data.

media_type: str

IANA media type.

type: Literal["image-data"]

Images that are referenced using base64 encoded data.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputContentValueImageURL: …

type: Literal["image-url"]

Images that are referenced using a URL.

url: str

URL of the image.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputContentValueImageFileID: …

file_id: Union[str, Dict[str, str]]

Image that is referenced using a provider file id.

If you use multiple providers, you need to specify the provider specific ids using the Record option. The key is the provider name, e.g. ‘openai’ or ‘anthropic’.

One of the following:

str

Dict[str, str]

type: Literal["image-file-id"]

Images that are referenced using a provider file id.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageToolContentToolResultOutputContentValueCustom: …

type: Literal["custom"]

Custom content part. This can be used to implement provider-specific content parts.

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Provider-specific options.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

tool_call_id: str

ID of the tool call that this result is associated with.

tool_name: str

Name of the tool that generated this result.

type: Literal["tool-result"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

class V1AutomateEventAIGenerationDataMessageToolContentToolApprovalResponse: …

Tool approval response prompt part.

approval_id: str

ID of the tool approval.

approved: bool

Flag indicating whether the approval was granted or denied.

type: Literal["tool-approval-response"]

provider_executed: Optional[bool]

Flag indicating whether the tool call is provider-executed. Only provider-executed tool approval responses should be sent to the model.

reason: Optional[str]

Optional reason for the approval or denial.

role: Literal["tool"]

provider_options: Optional[Dict[str, Dict[str, Union[str, float, bool, 2 more]]]]

Additional provider-specific metadata. They are passed through to the provider from the AI SDK and enable provider-specific functionality that can be fully encapsulated in the provider.

One of the following:

str

float

bool

Dict[str, Union[str, float, bool, 2 more]]

One of the following:

str

float

bool

List[object]

object

List[Union[str, float, bool, 3 more]]

One of the following:

str

float

bool

List[object]

object

object: Optional[object]

provider_metadata: Optional[Dict[str, object]]

temperature: Optional[float]

warnings: Optional[List[object]]

event: Literal["ai:generation"]

class V1AutomateEventAIGenerationError: …

Envelope for the “ai:generation:error” event from /v1/automate.

data: V1AutomateEventAIGenerationErrorData

Event data when AI generation error occurs

error: str

iteration_id: str

prompt: str

schema: object

timestamp: float

messages: Optional[List[object]]

event: Literal["ai:generation:error"]

class V1AutomateEventBrowserActionCompleted: …

Envelope for the “browser:action_completed” event from /v1/automate.

data: V1AutomateEventBrowserActionCompletedData

Event data for action results

iteration_id: str

success: bool

timestamp: float

error: Optional[str]

event: Literal["browser:action_completed"]

class V1AutomateEventBrowserActionStarted: …

Envelope for the “browser:action_started” event from /v1/automate.

data: V1AutomateEventBrowserActionStartedData

Event data for action execution

action: str

iteration_id: str

timestamp: float

ref: Optional[str]

value: Optional[str]

event: Literal["browser:action_started"]

class V1AutomateEventBrowserNavigated: …

Envelope for the “browser:navigated” event from /v1/automate.

data: V1AutomateEventBrowserNavigatedData

Event data when navigating to a page

iteration_id: str

timestamp: float

title: str

url: str

event: Literal["browser:navigated"]

class V1AutomateEventBrowserReconnected: …

Envelope for the “browser:reconnected” event from /v1/automate.

data: V1AutomateEventBrowserReconnectedData

Event data when the browser reconnects after a mid-task disconnect

endpoint_index: float

1-based index of the CDP endpoint now in use

iteration_id: str

starting_url: str

The original starting URL the agent is restarting execution from

timestamp: float

total: float

Total number of configured CDP endpoints

event: Literal["browser:reconnected"]

class V1AutomateEventBrowserScreenshotCaptured: …

Envelope for the “browser:screenshot_captured” event from /v1/automate.

data: V1AutomateEventBrowserScreenshotCapturedData

Event data for screenshot capture

format: Literal["jpeg", "png"]

One of the following:

"jpeg"

"png"

iteration_id: str

size: float

timestamp: float

event: Literal["browser:screenshot_captured"]

class V1AutomateEventBrowserScreenshotCapturedImage: …

Envelope for the “browser:screenshot_captured_image” event from /v1/automate.

data: V1AutomateEventBrowserScreenshotCapturedImageData

Event data for screenshot image capture with full image data This event contains the complete screenshot and can be very large

image: str

iteration_id: str

media_type: Literal["image/jpeg", "image/png"]

One of the following:

"image/jpeg"

"image/png"

timestamp: float

event: Literal["browser:screenshot_captured_image"]

class V1AutomateEventCdpEndpointConnected: …

Envelope for the “cdp:endpoint_connected” event from /v1/automate.

data: V1AutomateEventCdpEndpointConnectedData

Event data when a CDP endpoint is successfully connected to

endpoint_index: float

1-based index of the endpoint that connected

iteration_id: str

timestamp: float

total: float

Total number of configured CDP endpoints

event: Literal["cdp:endpoint_connected"]

class V1AutomateEventCdpEndpointCycle: …

Envelope for the “cdp:endpoint_cycle” event from /v1/automate.

data: V1AutomateEventCdpEndpointCycleData

Event data when a CDP endpoint fails and the next one is being tried

attempt: float

1-based index of the endpoint attempt that failed

error: str

Sanitized error identifier from the failed connection attempt (error.name, not error.message — full messages may contain endpoint URLs)

iteration_id: str

timestamp: float

total: float

Total number of configured CDP endpoints

event: Literal["cdp:endpoint_cycle"]

class V1AutomateEventComplete: …

Envelope for the “complete” event from /v1/automate.

data: V1AutomateEventCompleteData

Payload for the complete stream event. Structurally identical to TaskExecutionResult from webAgent.ts — the complete event’s data is the agent’s final TaskExecutionResult, stringified onto the SSE stream.

final_answer: Optional[str]

Final answer or result from the agent

stats: V1AutomateEventCompleteDataStats

Execution statistics

actions: float

duration_ms: float

end_time: float

iterations: float

start_time: float

success: bool

Whether the task completed successfully

error: Optional[V1AutomateEventCompleteDataError]

Structured error information for failed tasks

code: Literal["TASK_ABORTED", "MAX_ITERATIONS", "MAX_ERRORS", "TASK_FAILED"]

Error codes for task failures

One of the following:

"TASK_ABORTED"

"MAX_ITERATIONS"

"MAX_ERRORS"

"TASK_FAILED"

message: str

Human-readable error message

event: Literal["complete"]

class V1AutomateEventDone: …

Envelope for the “done” event from /v1/automate.

data: Dict[str, object]

Payload for the done stream terminator event. Empty today; reserved for future metadata.

event: Literal["done"]

class V1AutomateEventError: …

Envelope for the “error” event from /v1/automate.

data: V1AutomateEventErrorData

Payload for the top-level error stream event. Emitted when an uncaught error escapes the task runner. Mirrors ErrorResponse from the server package’s taskRunner.ts — kept structurally aligned so schema and runtime stay consistent. Distinct from agent-level error events like ai:generation:error and task:validation_error, which are emitted through the normal event emitter during the agent loop.

error: V1AutomateEventErrorDataError

code: str

message: str

timestamp: str

ISO-8601 timestamp

success: Literal[false]

event: Literal["error"]

class V1AutomateEventInteractiveFormDataError: …

Envelope for the “interactive:form_data:error” event from /v1/automate.

data: V1AutomateEventInteractiveFormDataErrorData

Event data when form validation fails and the agent re-requests data. Carries both the error context and the fields that need new values. Callers respond to this the same way as a request event.

field_errors: Dict[str, str]

Per-field error messages from validation (field ref -> error text)

fields: List[V1AutomateEventInteractiveFormDataErrorDataField]

field_type: Literal["text", "email", "phone", 8 more]

Semantic field type

One of the following:

"text"

"email"

"phone"

"date"

"number"

"select"

"checkbox"

"radio"

"textarea"

"password"

"other"

label: str

The field’s visible label

ref: str

Element ref from the accessibility tree (e.g., “E42”)

required: bool

Whether this field must be filled

current_value: Optional[str]

Current value if already partially filled

description: Optional[str]

Additional context (e.g., validation error message on re-request)

options: Optional[List[str]]

Available options for select/radio fields

form_description: str

iteration_id: str

page_title: str

page_url: str

request_id: str

timestamp: float

event: Literal["interactive:form_data:error"]

class V1AutomateEventInteractiveFormDataRequest: …

Envelope for the “interactive:form_data:request” event from /v1/automate.

data: V1AutomateEventInteractiveFormDataRequestData

Event data when the agent requests user data for form fields

fields: List[V1AutomateEventInteractiveFormDataRequestDataField]

field_type: Literal["text", "email", "phone", 8 more]

Semantic field type

One of the following:

"text"

"email"

"phone"

"date"

"number"

"select"

"checkbox"

"radio"

"textarea"

"password"

"other"

label: str

The field’s visible label

ref: str

Element ref from the accessibility tree (e.g., “E42”)

required: bool

Whether this field must be filled

current_value: Optional[str]

Current value if already partially filled

description: Optional[str]

Additional context (e.g., validation error message on re-request)

options: Optional[List[str]]

Available options for select/radio fields

form_description: str

iteration_id: str

page_title: str

page_url: str

request_id: str

timestamp: float

event: Literal["interactive:form_data:request"]

class V1AutomateEventSystemDebugCompression: …

Envelope for the “system:debug_compression” event from /v1/automate.

data: V1AutomateEventSystemDebugCompressionData

Event data for compression debug info

compressed_size: float

compression_percent: float

iteration_id: str

original_size: float

timestamp: float

event: Literal["system:debug_compression"]

class V1AutomateEventSystemDebugMessage: …

Envelope for the “system:debug_message” event from /v1/automate.

data: V1AutomateEventSystemDebugMessageData

Event data for message debug info

iteration_id: str

messages: List[object]

timestamp: float

event: Literal["system:debug_message"]

class V1AutomateEventTaskAborted: …

Envelope for the “task:aborted” event from /v1/automate.

data: V1AutomateEventTaskAbortedData

Event data when a task is aborted

final_answer: str

iteration_id: str

reason: str

timestamp: float

event: Literal["task:aborted"]

class V1AutomateEventTaskCompleted: …

Envelope for the “task:completed” event from /v1/automate.

data: V1AutomateEventTaskCompletedData

Event data when a task is completed

final_answer: Optional[str]

iteration_id: str

timestamp: float

success: Optional[bool]

event: Literal["task:completed"]

class V1AutomateEventTaskMetrics: …

Envelope for the “task:metrics” event from /v1/automate.

data: V1AutomateEventTaskMetricsData

ai_generation_count: float

ai_generation_error_count: float

event_counts: Dict[str, float]

iteration_id: str

step_count: float

timestamp: float

total_input_tokens: float

total_output_tokens: float

event: Literal["task:metrics"]

class V1AutomateEventTaskMetricsIncremental: …

Envelope for the “task:metrics_incremental” event from /v1/automate.

data: V1AutomateEventTaskMetricsIncrementalData

ai_generation_count: float

ai_generation_error_count: float

event_counts: Dict[str, float]

iteration_id: str

step_count: float

timestamp: float

total_input_tokens: float

total_output_tokens: float

event: Literal["task:metrics_incremental"]

class V1AutomateEventTaskSetup: …

Envelope for the “task:setup” event from /v1/automate.

data: V1AutomateEventTaskSetupData

Event data when a task is setup

browser_name: str

iteration_id: str

task: str

timestamp: float

data: Optional[object]

guardrails: Optional[str]

has_api_key: Optional[bool]

key_source: Optional[Literal["global", "env", "not_set"]]

One of the following:

"global"

"env"

"not_set"

model: Optional[str]

provider: Optional[str]

proxy: Optional[str]

pw_cdp_endpoint: Optional[str]

pw_cdp_endpoint_count: Optional[float]

Total number of CDP endpoints configured (index, not URLs)

pw_cdp_endpoints: Optional[List[str]]

pw_endpoint: Optional[str]

url: Optional[str]

vision: Optional[bool]

event: Literal["task:setup"]

class V1AutomateEventTaskStarted: …

Envelope for the “task:started” event from /v1/automate.

data: V1AutomateEventTaskStartedData

Event data when a task is started

iteration_id: str

plan: str

success_criteria: str

task: str

timestamp: float

url: str

action_items: Optional[List[str]]

event: Literal["task:started"]

class V1AutomateEventTaskTraceContext: …

Envelope for the “task:trace_context” event from /v1/automate.

data: V1AutomateEventTaskTraceContextData

Payload for the task:trace_context event. Carries the OpenTelemetry trace ID for this /v1/automate request so consumers can deep-link to distributed-tracing UIs (e.g. Cloud Trace, Cloud Logging) for the run.

trace_id: str

W3C trace ID — 32-character lowercase hexadecimal string.

event: Literal["task:trace_context"]

class V1AutomateEventTaskValidated: …

Envelope for the “task:validated” event from /v1/automate.

data: V1AutomateEventTaskValidatedData

Event data for task validation

completion_quality: Literal["failed", "partial", "complete", "excellent"]

One of the following:

"failed"

"partial"

"complete"

"excellent"

final_answer: str

iteration_id: str

observation: str

timestamp: float

feedback: Optional[str]

event: Literal["task:validated"]

class V1AutomateEventTaskValidationError: …

Envelope for the “task:validation_error” event from /v1/automate.

data: V1AutomateEventTaskValidationErrorData

Event data for validation errors during action response processing

errors: List[str]

iteration_id: str

raw_response: object

retry_count: float

timestamp: float

event: Literal["task:validation_error"]

AI Task

Python

HTTP

TypeScript

Python

Go

import os
from tabstack import Tabstack

client = Tabstack(
    api_key=os.environ.get("TABSTACK_API_KEY"),  # This is the default and can be omitted
)
for agent in client.agent.automate(
    task="Find the top 3 trending repositories and extract their names, descriptions, and star counts",
    guardrails="browse and extract only, don't interact with repositories",
    url="https://github.com/trending",
):
  print(agent)

200 example

200 example

400 example

400 example

400 example

event: start data: {"task": "Find the top 3 trending repositories", "url": "https://github.com/trending"} event: agent:processing data: {"operation": "Creating task plan", "hasScreenshot": false} event: browser:navigated data: {"title": "Trending - GitHub", "url": "https://github.com/trending"} event: agent:extracted data: {"extractedData": "[{\"name\": \"awesome-ai\", \"stars\": \"45.2k\"}]"} event: task:completed data: {"success": true, "finalAnswer": "Top 3 trending repos extracted"} event: complete data: {"success": true} event: done data: {}

event: start data: {"task": "Find product details", "url": "https://store.com/product"} event: browser:navigated data: {"title": "Product Page", "url": "https://store.com/product"} event: agent:extracted data: {"extractedData": "{\"name\": \"Headphones\", \"price\": \"$299.99\"}"} event: task:completed data: {"success": true} event: done data: {}

{
  "error": "maxIterations must be between 1 and 100"
}

{
  "error": "invalid URL format"
}

{
  "error": "task is required"
}

Returns Examples

200 example

200 example

400 example

400 example

400 example

event: start data: {"task": "Find the top 3 trending repositories", "url": "https://github.com/trending"} event: agent:processing data: {"operation": "Creating task plan", "hasScreenshot": false} event: browser:navigated data: {"title": "Trending - GitHub", "url": "https://github.com/trending"} event: agent:extracted data: {"extractedData": "[{\"name\": \"awesome-ai\", \"stars\": \"45.2k\"}]"} event: task:completed data: {"success": true, "finalAnswer": "Top 3 trending repos extracted"} event: complete data: {"success": true} event: done data: {}

event: start data: {"task": "Find product details", "url": "https://store.com/product"} event: browser:navigated data: {"title": "Product Page", "url": "https://store.com/product"} event: agent:extracted data: {"extractedData": "{\"name\": \"Headphones\", \"price\": \"$299.99\"}"} event: task:completed data: {"success": true} event: done data: {}

{
  "error": "maxIterations must be between 1 and 100"
}

{
  "error": "invalid URL format"
}

{
  "error": "task is required"
}