---
title: Orchestrator-in-the-loop | Tabstack
description: Handle interactive form events programmatically in autonomous pipelines. Your orchestrator resolves field values from its own context store and passes them back -- no human required.
---

Interactive Mode is currently in **Beta**. This initial release is optimized for form-filling scenarios. Future iterations will support more complex interactive workflows.

**Password fields don’t work end-to-end yet.** Under the current beta, the automation backend times out when filling `password`-typed fields during interactive submissions, so password-gated login flows will abort. Use this pattern for forms that request contact info, profile data, preferences, or other non-sensitive context. For login flows, fall back to a non-interactive path (e.g. a pre-issued session token supplied via the `data` parameter) until this is resolved.

When a parent agent dispatches sub-tasks to `/automate`, those sub-tasks may encounter forms that require context the automation layer doesn’t have. Instead of failing, `/automate` with `interactive: true` pauses and emits an event. Your orchestrator catches it, resolves the values from its own context store, and passes them back. The automation resumes. No human involvement required.

This guide shows you how to wire that pattern.

## Why this pattern

The standard interactive mode workflow puts a human at the terminal, typing form values on demand. That works for one-off tasks but breaks down in autonomous pipelines where no human is watching.

The orchestrator-in-the-loop pattern replaces the human with code:

- Your top-level agent maintains a context store: secrets, user profiles, preference databases, session tokens.
- It dispatches a `/automate` sub-task and stays on the event stream.
- When `interactive:form_data:request` fires, the orchestrator resolves the required field values from its own context and calls `automateInput`.
- If validation fails, `interactive:form_data:error` fires with the same shape plus a `fieldErrors` map. The orchestrator re-resolves and resubmits against the new request id.
- The stream terminates with `complete` — a typed payload carrying `finalAnswer`, `stats`, `success`, and (when the agent gave up) a structured `error.code`. A separate top-level `error` event fires only if the task runner itself crashed.

The pipeline never stalls. The orchestrator supplies missing context on demand rather than requiring it all upfront, which means you can build workflows against forms whose fields you don’t know ahead of time.

## Complete orchestrator example

- [TypeScript](#tab-panel-111)
- [Python](#tab-panel-112)

```
import Tabstack, { APIError } from '@tabstack/sdk'


const client = new Tabstack()


// Simulates a profile store your orchestrator manages. Keys are normalized
// field labels -- lowercased, punctuation-stripped -- so the lookup matches
// whatever the form presents to the user. Alias rows cover common label
// variations ("Full name" vs "Your name", "E-mail" vs "Email address").
const contextStore: Record<string, string> = {
  'name': process.env.PROFILE_NAME ?? '',
  'full name': process.env.PROFILE_NAME ?? '',
  'customer name': process.env.PROFILE_NAME ?? '',
  'email': process.env.PROFILE_EMAIL ?? '',
  'email address': process.env.PROFILE_EMAIL ?? '',
  'e mail address': process.env.PROFILE_EMAIL ?? '',
  'work email': process.env.PROFILE_EMAIL ?? '',
  'phone': process.env.PROFILE_PHONE ?? '',
  'telephone': process.env.PROFILE_PHONE ?? '',
  'phone number': process.env.PROFILE_PHONE ?? '',
  'company': process.env.PROFILE_COMPANY ?? '',
  'company name': process.env.PROFILE_COMPANY ?? '',
  'organization': process.env.PROFILE_COMPANY ?? '',
}


function normalizeLabel(label: string): string {
  return label.toLowerCase().replace(/[^a-z0-9]+/g, ' ').trim()
}


type ResolvedFields = {
  fields: Array<{ ref: string; value: string }>
  missing: string[]
}


function resolveFields(
  fields: Array<{ ref: string; label: string; required: boolean }>,
): ResolvedFields {
  const resolved: Array<{ ref: string; value: string }> = []
  const missing: string[] = []


  for (const field of fields) {
    // Look up by the human-readable label. `ref` is an accessibility-tree id
    // (e.g. "E42") used to route the value back to the right element on
    // submission -- it is not a semantic key you can match against a secrets
    // store.
    const value = contextStore[normalizeLabel(field.label)] ?? ''


    if (!value) {
      if (field.required) {
        missing.push(`${field.label} (${field.ref})`)
      } else {
        resolved.push({ ref: field.ref, value: '' })
      }
    } else {
      resolved.push({ ref: field.ref, value })
    }
  }


  return { fields: resolved, missing }
}


async function resolveAndSubmit(
  requestId: string,
  fields: Array<{ ref: string; label: string; required: boolean }>,
  fieldErrors?: Record<string, string>,
): Promise<void> {
  if (fieldErrors && Object.keys(fieldErrors).length > 0) {
    console.warn('Retrying after validation errors:', fieldErrors)
  }


  const { fields: resolvedFields, missing } = resolveFields(fields)


  if (missing.length > 0) {
    // Orchestrator also lacks the value -- cancel rather than submit blanks.
    console.warn(
      `Orchestrator missing values for fields: ${missing.join(', ')}. Cancelling.`,
    )
    await client.agent.automateInput(requestId, { cancelled: true })
    throw new Error(
      `Cannot complete task: missing context for fields [${missing.join(', ')}]`,
    )
  }


  try {
    await client.agent.automateInput(requestId, { fields: resolvedFields })
  } catch (err) {
    if (err instanceof APIError && err.status === 410) {
      // 410 Gone -- orchestrator took longer than 2 minutes to resolve.
      throw new Error(
        'Input request expired before the orchestrator could respond. ' +
          'Reduce context lookup latency or restart the task.',
      )
    }
    throw err
  }
}


async function runSubTask(task: string, url: string): Promise<string> {
  const stream = await client.agent.automate({
    task,
    url,
    interactive: true,
  })


  for await (const event of stream) {
    switch (event.event) {
      case 'interactive:form_data:request':
        await resolveAndSubmit(event.data.requestId, event.data.fields)
        break


      case 'interactive:form_data:error':
        // Validation failed. The event carries the fields that need new
        // values and a fieldErrors map keyed by ref. Respond to the new
        // requestId the same way you responded to the original request.
        await resolveAndSubmit(
          event.data.requestId,
          event.data.fields,
          event.data.fieldErrors,
        )
        break


      case 'complete': {
        // Stream terminator with the agent's final result.
        if (!event.data.success) {
          const code = event.data.error?.code ?? 'UNKNOWN'
          const message = event.data.error?.message ?? '(no message)'
          throw new Error(`Sub-task ended without success (${code}): ${message}`)
        }
        const { durationMs, iterations } = event.data.stats
        console.log(`Sub-task finished in ${durationMs}ms across ${iterations} iterations.`)
        return event.data.finalAnswer ?? ''
      }


      case 'error':
        // Top-level runner crash. Distinct from agent-level aborts, which
        // surface inside `complete` with `success: false`.
        throw new Error(
          `Task runner error (${event.data.error.code}): ${event.data.error.message}`,
        )
    }
  }


  throw new Error('Stream ended without a complete event')
}


// Entry point -- called by the parent orchestrator agent.
async function main() {
  try {
    const result = await runSubTask(
      'Submit the contact form on this page using the provided profile data. Leave optional fields blank.',
      'https://www.example.com/contact',
    )
    console.log('Sub-task result:', result)
  } catch (err) {
    console.error('Sub-task failed:', err)
    process.exit(1)
  }
}


main()
```

### What each part does

`runSubTask` dispatches the automation and owns the event loop. Switching on `event.event` narrows `event.data` to the correct variant automatically — the SDK models the stream as a discriminated union, so no casting is needed for typed events.

`resolveFields` is the lookup layer. It normalizes each field’s `label` (lowercase, punctuation stripped) and uses that as the key into `contextStore`. `ref` is retained verbatim in the submitted payload so the agent can map the value back to the correct element. Required fields with no value are collected into `missing` so the orchestrator can cancel cleanly instead of submitting blanks.

`resolveAndSubmit` is shared between the request and error branches. When validation fails, the agent re-emits a fresh event (`interactive:form_data:error`) carrying a new `requestId`, the fields that still need values, and a `fieldErrors` map. The handler logs the error context and runs the same resolve-and-submit flow against the new id. If you skip this branch and treat `interactive:form_data:error` as informational, the task hangs until the 2-minute timeout and then fails.

The `complete` branch is the canonical termination. Its data carries `finalAnswer`, `stats` (duration, iteration count, action count), `success`, and an optional structured `error` with a typed `code` enum (`TASK_ABORTED` / `MAX_ITERATIONS` / `MAX_ERRORS` / `TASK_FAILED`). Branch on `success` to distinguish a clean finish from an agent that gave up; the `error.code` tells you why if it did. The separate top-level `error` event only fires if the task runner itself crashed — distinct from agent-level aborts, which surface inside `complete`.

```
import os
import re
from tabstack import Tabstack, APIStatusError


client = Tabstack()


# Simulates your orchestrator's profile store. Keys are normalized field
# labels -- lowercased, punctuation-stripped -- so the lookup matches
# whatever the form presents to the user. Alias rows cover common label
# variations ("Full name" vs "Your name", "E-mail" vs "Email address").
context_store: dict[str, str] = {
    "name": os.environ.get("PROFILE_NAME", ""),
    "full name": os.environ.get("PROFILE_NAME", ""),
    "customer name": os.environ.get("PROFILE_NAME", ""),
    "email": os.environ.get("PROFILE_EMAIL", ""),
    "email address": os.environ.get("PROFILE_EMAIL", ""),
    "e mail address": os.environ.get("PROFILE_EMAIL", ""),
    "work email": os.environ.get("PROFILE_EMAIL", ""),
    "phone": os.environ.get("PROFILE_PHONE", ""),
    "telephone": os.environ.get("PROFILE_PHONE", ""),
    "phone number": os.environ.get("PROFILE_PHONE", ""),
    "company": os.environ.get("PROFILE_COMPANY", ""),
    "company name": os.environ.get("PROFILE_COMPANY", ""),
    "organization": os.environ.get("PROFILE_COMPANY", ""),
}




def normalize_label(label: str) -> str:
    return re.sub(r"[^a-z0-9]+", " ", label.lower()).strip()




def resolve_fields(fields) -> tuple[list[dict], list[str]]:
    resolved: list[dict] = []
    missing: list[str] = []


    for field in fields:
        # Look up by the human-readable label. `ref` is an accessibility-tree
        # id (e.g. "E42") used to route the value back to the right element
        # on submission -- it is not a semantic key you can match against a
        # secrets store.
        value = context_store.get(normalize_label(field.label), "")


        if not value:
            if field.required:
                missing.append(f"{field.label} ({field.ref})")
            else:
                resolved.append({"ref": field.ref, "value": ""})
        else:
            resolved.append({"ref": field.ref, "value": value})


    return resolved, missing




def resolve_and_submit(request_id: str, fields, field_errors: dict | None = None) -> None:
    if field_errors:
        print(f"Retrying after validation errors: {field_errors}")


    resolved, missing = resolve_fields(fields)


    if missing:
        print(f"Orchestrator missing values for fields: {missing}. Cancelling.")
        client.agent.automate_input(request_id, cancelled=True)
        raise RuntimeError(
            f"Cannot complete task: missing context for fields {missing}"
        )


    try:
        client.agent.automate_input(request_id, fields=resolved)
    except APIStatusError as err:
        if err.status_code == 410:
            raise RuntimeError(
                "Input request expired before the orchestrator could respond. "
                "Reduce context lookup latency or restart the task."
            )
        raise




def run_sub_task(task: str, url: str) -> str:
    stream = client.agent.automate(
        task=task,
        url=url,
        interactive=True,
    )


    for event in stream:
        if event.event == "interactive:form_data:request":
            resolve_and_submit(event.data.request_id, event.data.fields)


        elif event.event == "interactive:form_data:error":
            # Validation failed. The event carries the fields that need new
            # values and a field_errors map keyed by ref. Respond to the new
            # request_id the same way you responded to the original request.
            resolve_and_submit(
                event.data.request_id,
                event.data.fields,
                event.data.field_errors,
            )


        elif event.event == "complete":
            # Stream terminator with the agent's final result.
            if not event.data.success:
                code = event.data.error.code if event.data.error else "UNKNOWN"
                message = event.data.error.message if event.data.error else "(no message)"
                raise RuntimeError(
                    f"Sub-task ended without success ({code}): {message}"
                )
            stats = event.data.stats
            print(
                f"Sub-task finished in {stats.duration_ms}ms across "
                f"{stats.iterations} iterations."
            )
            return event.data.final_answer or ""


        elif event.event == "error":
            # Top-level runner crash. Distinct from agent-level aborts, which
            # surface inside `complete` with `success` set to False.
            raise RuntimeError(
                f"Task runner error ({event.data.error.code}): {event.data.error.message}"
            )


    raise RuntimeError("Stream ended without a complete event")




if __name__ == "__main__":
    result = run_sub_task(
        task="Submit the contact form on this page using the provided profile data. Leave optional fields blank.",
        url="https://www.example.com/contact",
    )
    print("Sub-task result:", result)
```

## Production considerations

### Keying the context store on labels, not refs

`ref` is an accessibility-tree identifier the agent assigns when it captures the form (e.g. `"E42"`). It’s stable within a single task run — you submit it back so the agent knows which element your value belongs to — but it is not a semantic key. Across page re-renders or runs it will change, and it never corresponds to anything meaningful in your secrets store.

Key on the field’s `label` instead. Normalize it before lookup (lowercase, collapse whitespace and punctuation) so `"Email Address"`, `"Email address"`, and `"email-address"` all hit the same store entry. For sites where labels vary in wording across pages, keep an alias table — e.g. `"Email"`, `"Email Address"`, and `"Your email"` all map to the same store entry. The example above does this inline; in production, generate the alias table from a canonical field list rather than maintaining it by hand.

### Handling unknown fields gracefully

When `resolveFields` returns missing required fields, you have two choices:

- Cancel with `{ cancelled: true }` and surface the error to the parent orchestrator. This is the safe default.
- Escalate to a human fallback queue. If your system has a human-in-the-loop tier, this is where you’d route the request before the 2-minute window closes.

Never submit a blank value for a required field. The agent will likely re-prompt immediately (increasing round-trips) or submit an invalid form that triggers a downstream error harder to diagnose than a clean cancellation.

### Timeout management

Input requests expire after 2 minutes. If your context lookup involves a network call (secrets manager, external database), measure that latency and build in a budget. A safe heuristic: if your lookup takes more than 90 seconds under load, add a local cache layer so the first response is fast. When a 410 comes back from `automateInput`, it surfaces as `APIError` with `status === 410` in TypeScript and `APIStatusError` with `status_code == 410` in Python. The request is unrecoverable — you must restart the task.

If you catch a 410, log the `requestId` alongside the task parameters so you can diagnose whether the expiry was consistent (indicating a lookup latency problem) or a one-off.

### When the orchestrator also lacks the value

The `missing` path in `resolveFields` is not an edge case. Design for it. Options:

- **Cancel and re-queue** with the missing fields injected into the next task’s `data` param after a human provides them out-of-band.
- **Cancel and escalate** to a human-approval step in your workflow system.
- **Cancel and fail fast**: if the orchestrator should always have these values, a missing value is a configuration error worth surfacing loudly.

Avoid silently passing empty strings for required fields. The failure will be harder to debug than a clean cancellation with a logged reason.

## Related

- [Interactive mode guide](/guides/interactive-mode/index.md) — how interactive mode works, the full event reference, and how to handle the human-at-terminal case
- [Automation Events reference](/reference/automate-events/index.md) — the complete `AutomateEvent` union and consumption patterns
- [API reference](/api/index.md) — full parameter and response schema for `/automate` and the input endpoint