machine:extension:apply
Turns one model decision into a step and performs it on the live page, then reports what the next turn needs to know. A step the loop may not write, a ref the page does not hold and a step the page refused are all ordinary outcomes here rather than component failures. Each one comes back as Error, which is the text the next prompt shows the model, because a loop that stopped on them would throw away the one thing that lets the model correct itself. A step that hands the page to the person in front of it is the one turn this does not hold to the turn timeout. It waits on them for as long as the step allows, and the turn is given back to the session's budget, so a loop is never shortened by the waiting it asked for.
Options
| Name | Type | Description |
|---|---|---|
decision | string | The model's raw reply for this turn, which is one JSON object holding reasoning, done and step. An empty or unreadable reply is a turn that failed rather than a step that failed, so it is fed back rather than refused here |
observation | PageObservation | The page the decision was made against, whose refs the step's ref is resolved through |
↳ url | string | Is the observed page's address. |
↳ title | string | Is the document title. |
↳ nodes | []ObservedNode | Are the interactive elements, in document order. |
↳ ref | string | Is the short handle for this node ("e1", "e2"), unique inside one observation and meaningless outside it. |
↳ role | string | Is the aria role, explicit or implied by the tag. |
↳ name | string | Is the accessible name, from aria-label, an associated label, the text, the placeholder, the title, or the alt text. |
↳ tag | string | Is the lowercase tag name. |
↳ type | string | Is the input type, for form controls that carry one. |
↳ value | string | Is the current value, capped by the observer and never set for a password input. |
↳ placeholder | string | Is the control's placeholder text. |
↳ disabled | bool | Reports whether the control rejects interaction. |
↳ checked | bool | Is the state of a checkbox or radio. |
↳ expanded | bool | Mirrors aria-expanded. |
↳ options | []string | Are a select's option values, capped by the observer. |
↳ selector | string | Is the stable CSS selector the observer built for this node, the same one the element picker would produce. |
↳ path | string | Is the structural path, for a node CSS cannot address. |
↳ indexes | []int | Narrows a multi-match selector to this node's positions. |
↳ enter | []string | Mirrors Selector.Enter, the shadow hosts and frames to descend through before Selector resolves. Absent for an element in the top document, which is most of them. |
↳ section | string | Names the nearest enclosing landmark or the preceding heading, so a reader can tell two identical controls apart by where they sit. |
↳ context | string | Is the text of the row, card or list item the element sits in, filled in only when the element's own name does not already say which one this is. It is what tells twenty identical add-to-cart buttons apart. |
↳ offscreen | bool | Marks an element laid out outside the current viewport. It is still real and still actionable, since the engine scrolls to it. |
↳ obscured | bool | Marks an element another element covers at its center point, which is what a cookie wall or an open dialog does to the page behind it. |
↳ similar | int | Counts the further siblings left out because they repeat this one's shape. It sits on the last member of a collapsed run. |
↳ headings | []ObservedHeading | Are the page's headings, in document order. |
↳ level | int | Is the heading level, 1 for h1. |
↳ text | string | Is the heading's text. |
↳ text | string | Is the readable page text, present only when the observation asked for it, capped by the observer. |
↳ truncated | bool | Reports that the node or text cap clipped the page, so a reader knows the snapshot is partial. |
↳ modal | string | Names the dialog or consent wall standing over the page, when one is. Everything outside it is marked obscured, because clicking it is what a browser would refuse to do until the wall is dealt with. |
↳ modal_frame | bool | Says the wall is a cross-origin frame, which is how most consent managers ship. Nothing inside one can be listed or clicked from the page it covers, so getting past it needs another way in rather than a ref from this snapshot. |
↳ omitted | int | Counts the elements the caps and the repeat collapsing left out. |
↳ frames | int | Counts the cross-origin iframes on the page, whose contents no observation can reach from here. |
↳ lists | []ObservedList | Are the repeated content blocks on the page. Nothing in Nodes describes a review card or a product tile that holds no control, so this is the only thing that says the page has records on it at all. |
↳ ref | string | Is the short handle for this list inside one observation. |
↳ selector | string | Names the item container. |
↳ count | int | Is how many members the selector matched. |
↳ section | string | Names the region of the page the list sits in, when the observer could tell. |
↳ fields | []ObservedListField | Are the candidate columns, each relative to the item container. |
↳ name | string | Is the column name the observer proposed. |
↳ css | string | Is relative to the item container. |
↳ attribute | string | Names the attribute the column reads, when it reads one. |
↳ sample | string | Is what the column read on the first member, so a caller can tell a price from a rating without opening the page. |
↳ pagination | ObservedPagination | Names the control that reaches the next batch of items, when the page has one. |
↳ kind | string | Is the shape the page uses to reach its next batch, one of link, numbered, more, or infinite. |
↳ selector | string | Targets the control that advances the listing. |
↳ label | string | Is the control's visible text. |
↳ url | string | Is the next page's href when the control is a link. |
↳ pages | int | Counts the numbered page links a pager prints, when it prints any. |
steps | []Step | The steps accepted before this turn, in order |
↳ id | string | Uniquely names the step within its workflow, echoed on its StepResult. |
↳ type | string | Is the action to perform (click, fill, select, captureUrl, ...). |
↳ label | string | Is a human-readable description shown in run progress UIs. |
↳ selector | Selector | Picks the DOM node(s) the step acts on. |
↳ css | string | Is a CSS selector. |
↳ path | string | Is an explicit DOM path, used when CSS cannot express the target. |
↳ indexes | []int | Narrows a multi-match selector to these positions. |
↳ enter | []string | Names the subtrees CSS has to resolve inside, outermost first. Each entry selects a shadow host or an iframe, resolved in the tree the previous entry opened, so a control inside a web component inside a same-origin frame is reachable by one Selector. Empty runs CSS against the top document. |
↳ index | int | Narrows a multi-match selector to a single position. |
↳ all | bool | Expands the step to act on every match instead of the first. |
↳ value | string | Is the input for value-taking steps such as fill and select. It may contain {{ variable }} placeholders. |
↳ format | string | Names the shape extraction steps return their data in. |
↳ attribute | string | Names the DOM attribute an extraction step reads. |
↳ fields | ExtractField | Are the named columns an extractList step reads out of each item its selector matched. Key order is the column order, and every returned item carries every key. |
↳ limit | int | Caps how many items an extractList step returns. The extension applies its own bound when this is zero, so a listing that lazily grew to ten thousand rows truncates and says so. |
↳ url | string | Is the address for navigation steps. |
↳ fn | string | Names the page function an eval step invokes. |
↳ args | []any | Are the arguments passed to Fn. |
↳ body_params | map[string]string | Overrides or adds form fields on the request a captureDownload re-issues, and carries the parameters a captureUrl sends, in the query for a GET and form-encoded otherwise. |
↳ method | string | Is the HTTP method a captureUrl step fetches its URL with, defaulting to GET. |
↳ filename | string | Renames a captured file (captureDownload, captureUrl and capturePageFile). It may contain {{ variable }} placeholders. |
↳ output | string | Chooses where a captured file goes, and the controlling client is its only destination. |
↳ wait | WaitConfig | Blocks the step until a page condition holds. |
↳ selector | string | Waits for a matching element to appear (or disappear, see Absent). |
↳ text | string | Waits for this text to be present on the page. |
↳ url_includes | string | Waits for the page URL to contain this substring. |
↳ absent | bool | Flips the wait to expect the selector to disappear rather than appear. |
↳ delay_ms | int | Is a fixed pause in milliseconds, applied on top of any condition. |
↳ timeout_ms | int | Bounds the wait, in milliseconds. |
↳ prompt | string | Is the text shown at a user gate asking for manual input. |
↳ continue_on | ContinueCondition | Holds the step until a user gate is cleared or a page condition is met. |
↳ manual | bool | Pauses the run for an explicit user continue. |
↳ selector | string | Resumes when a matching element appears. |
↳ text | string | Resumes when this text is present on the page. |
↳ url_includes | string | Resumes when the page URL contains this substring. |
↳ timeout_ms | int | Bounds the hold, in milliseconds. |
↳ timeout_ms | int | Overrides the workflow's default step timeout, in milliseconds. |
↳ optional | bool | Lets the run continue when this step fails. |
↳ condition | Condition | Condition, when set, gates the step. If it does not hold, the step is skipped rather than failed. |
↳ selector | string | Holds when a matching element exists on the page. |
↳ text | string | Holds when the page (or the selected element) contains this text. |
↳ url_includes | string | Holds when the page URL contains this substring. |
↳ match | string | Decides how the set predicates combine, "any" or "all". |
↳ negate | bool | Inverts the combined result. |
↳ store | string | Names a slot the step's data is saved under, readable by later steps and returned on the WorkflowResult. |
↳ resume | ResumePoint | Resume, when set, marks this step as a point a re-run may start at, and carries the predicate recognizing a browser already standing there. |
↳ id | string | Names this point so a later point's Rewind can reach it. It is unique within one workflow. |
↳ when | Condition | Recognizes the browser standing at this point. |
↳ selector | string | Holds when a matching element exists on the page. |
↳ text | string | Holds when the page (or the selected element) contains this text. |
↳ url_includes | string | Holds when the page URL contains this substring. |
↳ match | string | Decides how the set predicates combine, "any" or "all". |
↳ negate | bool | Inverts the combined result. |
↳ rewind | Rewind | Rewind, when set, is how the browser leaves this point for an earlier one. |
↳ to | string | Names the resume point this reaches, which must sit earlier in the step list than the point declaring it. |
↳ url | string | Navigates the tab back to the earlier point. |
↳ selector | string | Clicks a control on the current page that leads back, for a flow whose way back is a button rather than an address. |
↳ wait | WaitConfig | Holds until the page has settled after the rewind, before the target point's own predicate is checked again. |
↳ selector | string | Waits for a matching element to appear (or disappear, see Absent). |
↳ text | string | Waits for this text to be present on the page. |
↳ url_includes | string | Waits for the page URL to contain this substring. |
↳ absent | bool | Flips the wait to expect the selector to disappear rather than appear. |
↳ delay_ms | int | Is a fixed pause in milliseconds, applied on top of any condition. |
↳ timeout_ms | int | Bounds the wait, in milliseconds. |
↳ always | bool | Runs the step even when re-entry starts the run past it, for work the page cannot report as already done. |
last_observation | string | The page as it read on the previous turn, which is how a step that changed nothing visible is told apart from one that moved the page on |
last_url | string | Where the page was on the previous turn, which is how the loop notices that the step it accepted then moved the page |
trail | []ExtTurn | The turns taken before this one, so the record travels with the session |
↳ turn | int | Is the position of this turn in the session, counting from one. |
↳ reasoning | string | Is the model's one-line justification. |
↳ type | string | Is the step type decided, empty when the model called the goal reached or when the answer could not be read. |
↳ label | string | Is the step's own description. |
↳ ref | string | Is the element the step named. |
↳ value | string | Is what a fill or select step typed, as written, so a token stays a token and a real credential never reaches here. |
↳ ok | bool | Reports whether the turn produced a step the page accepted. |
↳ done | bool | Reports the model calling the goal reached. |
↳ error | string | Is why the turn produced nothing, in the words the next turn read. |
↳ url | string | Is where the page was when the turn was decided. |
notes | []string | The model's reasoning for each accepted step, in order |
variables | map[string]string | The {{ }} placeholders the authored workflow may use, name to description |
values | map[string]string | Real values for the declared variables, typed into the live page and never emitted |
workflow_id | string | The id the session is authoring under, which names the inline run this step travels in |
client_id | string | The extension client to drive, a hint the host may honour; empty leaves the choice of browser to the host |
budget | int | How many turns the loop may still take, this one included. A turn that waited on a person is handed back, up to five of them |
failures | int | How many turns in a row have already failed |
max_failures | int | How many consecutive failures end the session (default 3) |
timeout_ms | int | How long to wait for the browser before failing the step |
Outputs
| Name | Type | Description |
|---|---|---|
steps | []Step | Every step accepted so far, this turn's included when it worked |
↳ id | string | Uniquely names the step within its workflow, echoed on its StepResult. |
↳ type | string | Is the action to perform (click, fill, select, captureUrl, ...). |
↳ label | string | Is a human-readable description shown in run progress UIs. |
↳ selector | Selector | Picks the DOM node(s) the step acts on. |
↳ css | string | Is a CSS selector. |
↳ path | string | Is an explicit DOM path, used when CSS cannot express the target. |
↳ indexes | []int | Narrows a multi-match selector to these positions. |
↳ enter | []string | Names the subtrees CSS has to resolve inside, outermost first. Each entry selects a shadow host or an iframe, resolved in the tree the previous entry opened, so a control inside a web component inside a same-origin frame is reachable by one Selector. Empty runs CSS against the top document. |
↳ index | int | Narrows a multi-match selector to a single position. |
↳ all | bool | Expands the step to act on every match instead of the first. |
↳ value | string | Is the input for value-taking steps such as fill and select. It may contain {{ variable }} placeholders. |
↳ format | string | Names the shape extraction steps return their data in. |
↳ attribute | string | Names the DOM attribute an extraction step reads. |
↳ fields | ExtractField | Are the named columns an extractList step reads out of each item its selector matched. Key order is the column order, and every returned item carries every key. |
↳ limit | int | Caps how many items an extractList step returns. The extension applies its own bound when this is zero, so a listing that lazily grew to ten thousand rows truncates and says so. |
↳ url | string | Is the address for navigation steps. |
↳ fn | string | Names the page function an eval step invokes. |
↳ args | []any | Are the arguments passed to Fn. |
↳ body_params | map[string]string | Overrides or adds form fields on the request a captureDownload re-issues, and carries the parameters a captureUrl sends, in the query for a GET and form-encoded otherwise. |
↳ method | string | Is the HTTP method a captureUrl step fetches its URL with, defaulting to GET. |
↳ filename | string | Renames a captured file (captureDownload, captureUrl and capturePageFile). It may contain {{ variable }} placeholders. |
↳ output | string | Chooses where a captured file goes, and the controlling client is its only destination. |
↳ wait | WaitConfig | Blocks the step until a page condition holds. |
↳ selector | string | Waits for a matching element to appear (or disappear, see Absent). |
↳ text | string | Waits for this text to be present on the page. |
↳ url_includes | string | Waits for the page URL to contain this substring. |
↳ absent | bool | Flips the wait to expect the selector to disappear rather than appear. |
↳ delay_ms | int | Is a fixed pause in milliseconds, applied on top of any condition. |
↳ timeout_ms | int | Bounds the wait, in milliseconds. |
↳ prompt | string | Is the text shown at a user gate asking for manual input. |
↳ continue_on | ContinueCondition | Holds the step until a user gate is cleared or a page condition is met. |
↳ manual | bool | Pauses the run for an explicit user continue. |
↳ selector | string | Resumes when a matching element appears. |
↳ text | string | Resumes when this text is present on the page. |
↳ url_includes | string | Resumes when the page URL contains this substring. |
↳ timeout_ms | int | Bounds the hold, in milliseconds. |
↳ timeout_ms | int | Overrides the workflow's default step timeout, in milliseconds. |
↳ optional | bool | Lets the run continue when this step fails. |
↳ condition | Condition | Condition, when set, gates the step. If it does not hold, the step is skipped rather than failed. |
↳ selector | string | Holds when a matching element exists on the page. |
↳ text | string | Holds when the page (or the selected element) contains this text. |
↳ url_includes | string | Holds when the page URL contains this substring. |
↳ match | string | Decides how the set predicates combine, "any" or "all". |
↳ negate | bool | Inverts the combined result. |
↳ store | string | Names a slot the step's data is saved under, readable by later steps and returned on the WorkflowResult. |
↳ resume | ResumePoint | Resume, when set, marks this step as a point a re-run may start at, and carries the predicate recognizing a browser already standing there. |
↳ id | string | Names this point so a later point's Rewind can reach it. It is unique within one workflow. |
↳ when | Condition | Recognizes the browser standing at this point. |
↳ selector | string | Holds when a matching element exists on the page. |
↳ text | string | Holds when the page (or the selected element) contains this text. |
↳ url_includes | string | Holds when the page URL contains this substring. |
↳ match | string | Decides how the set predicates combine, "any" or "all". |
↳ negate | bool | Inverts the combined result. |
↳ rewind | Rewind | Rewind, when set, is how the browser leaves this point for an earlier one. |
↳ to | string | Names the resume point this reaches, which must sit earlier in the step list than the point declaring it. |
↳ url | string | Navigates the tab back to the earlier point. |
↳ selector | string | Clicks a control on the current page that leads back, for a flow whose way back is a button rather than an address. |
↳ wait | WaitConfig | Holds until the page has settled after the rewind, before the target point's own predicate is checked again. |
↳ selector | string | Waits for a matching element to appear (or disappear, see Absent). |
↳ text | string | Waits for this text to be present on the page. |
↳ url_includes | string | Waits for the page URL to contain this substring. |
↳ absent | bool | Flips the wait to expect the selector to disappear rather than appear. |
↳ delay_ms | int | Is a fixed pause in milliseconds, applied on top of any condition. |
↳ timeout_ms | int | Bounds the wait, in milliseconds. |
↳ always | bool | Runs the step even when re-entry starts the run past it, for work the page cannot report as already done. |
step | Step | What this turn applied, absent when the turn was refused or the model called the goal reached |
↳ id | string | Uniquely names the step within its workflow, echoed on its StepResult. |
↳ type | string | Is the action to perform (click, fill, select, captureUrl, ...). |
↳ label | string | Is a human-readable description shown in run progress UIs. |
↳ selector | Selector | Picks the DOM node(s) the step acts on. |
↳ css | string | Is a CSS selector. |
↳ path | string | Is an explicit DOM path, used when CSS cannot express the target. |
↳ indexes | []int | Narrows a multi-match selector to these positions. |
↳ enter | []string | Names the subtrees CSS has to resolve inside, outermost first. Each entry selects a shadow host or an iframe, resolved in the tree the previous entry opened, so a control inside a web component inside a same-origin frame is reachable by one Selector. Empty runs CSS against the top document. |
↳ index | int | Narrows a multi-match selector to a single position. |
↳ all | bool | Expands the step to act on every match instead of the first. |
↳ value | string | Is the input for value-taking steps such as fill and select. It may contain {{ variable }} placeholders. |
↳ format | string | Names the shape extraction steps return their data in. |
↳ attribute | string | Names the DOM attribute an extraction step reads. |
↳ fields | ExtractField | Are the named columns an extractList step reads out of each item its selector matched. Key order is the column order, and every returned item carries every key. |
↳ limit | int | Caps how many items an extractList step returns. The extension applies its own bound when this is zero, so a listing that lazily grew to ten thousand rows truncates and says so. |
↳ url | string | Is the address for navigation steps. |
↳ fn | string | Names the page function an eval step invokes. |
↳ args | []any | Are the arguments passed to Fn. |
↳ body_params | map[string]string | Overrides or adds form fields on the request a captureDownload re-issues, and carries the parameters a captureUrl sends, in the query for a GET and form-encoded otherwise. |
↳ method | string | Is the HTTP method a captureUrl step fetches its URL with, defaulting to GET. |
↳ filename | string | Renames a captured file (captureDownload, captureUrl and capturePageFile). It may contain {{ variable }} placeholders. |
↳ output | string | Chooses where a captured file goes, and the controlling client is its only destination. |
↳ wait | WaitConfig | Blocks the step until a page condition holds. |
↳ selector | string | Waits for a matching element to appear (or disappear, see Absent). |
↳ text | string | Waits for this text to be present on the page. |
↳ url_includes | string | Waits for the page URL to contain this substring. |
↳ absent | bool | Flips the wait to expect the selector to disappear rather than appear. |
↳ delay_ms | int | Is a fixed pause in milliseconds, applied on top of any condition. |
↳ timeout_ms | int | Bounds the wait, in milliseconds. |
↳ prompt | string | Is the text shown at a user gate asking for manual input. |
↳ continue_on | ContinueCondition | Holds the step until a user gate is cleared or a page condition is met. |
↳ manual | bool | Pauses the run for an explicit user continue. |
↳ selector | string | Resumes when a matching element appears. |
↳ text | string | Resumes when this text is present on the page. |
↳ url_includes | string | Resumes when the page URL contains this substring. |
↳ timeout_ms | int | Bounds the hold, in milliseconds. |
↳ timeout_ms | int | Overrides the workflow's default step timeout, in milliseconds. |
↳ optional | bool | Lets the run continue when this step fails. |
↳ condition | Condition | Condition, when set, gates the step. If it does not hold, the step is skipped rather than failed. |
↳ selector | string | Holds when a matching element exists on the page. |
↳ text | string | Holds when the page (or the selected element) contains this text. |
↳ url_includes | string | Holds when the page URL contains this substring. |
↳ match | string | Decides how the set predicates combine, "any" or "all". |
↳ negate | bool | Inverts the combined result. |
↳ store | string | Names a slot the step's data is saved under, readable by later steps and returned on the WorkflowResult. |
↳ resume | ResumePoint | Resume, when set, marks this step as a point a re-run may start at, and carries the predicate recognizing a browser already standing there. |
↳ id | string | Names this point so a later point's Rewind can reach it. It is unique within one workflow. |
↳ when | Condition | Recognizes the browser standing at this point. |
↳ selector | string | Holds when a matching element exists on the page. |
↳ text | string | Holds when the page (or the selected element) contains this text. |
↳ url_includes | string | Holds when the page URL contains this substring. |
↳ match | string | Decides how the set predicates combine, "any" or "all". |
↳ negate | bool | Inverts the combined result. |
↳ rewind | Rewind | Rewind, when set, is how the browser leaves this point for an earlier one. |
↳ to | string | Names the resume point this reaches, which must sit earlier in the step list than the point declaring it. |
↳ url | string | Navigates the tab back to the earlier point. |
↳ selector | string | Clicks a control on the current page that leads back, for a flow whose way back is a button rather than an address. |
↳ wait | WaitConfig | Holds until the page has settled after the rewind, before the target point's own predicate is checked again. |
↳ selector | string | Waits for a matching element to appear (or disappear, see Absent). |
↳ text | string | Waits for this text to be present on the page. |
↳ url_includes | string | Waits for the page URL to contain this substring. |
↳ absent | bool | Flips the wait to expect the selector to disappear rather than appear. |
↳ delay_ms | int | Is a fixed pause in milliseconds, applied on top of any condition. |
↳ timeout_ms | int | Bounds the wait, in milliseconds. |
↳ always | bool | Runs the step even when re-entry starts the run past it, for work the page cannot report as already done. |
notes | []string | The reasoning behind each accepted step, in order |
trail | []ExtTurn | Every turn taken so far, accepted or not, this one last. It is the only record of why a session went the way it did, since a refused turn leaves no trace in the workflow being written |
↳ turn | int | Is the position of this turn in the session, counting from one. |
↳ reasoning | string | Is the model's one-line justification. |
↳ type | string | Is the step type decided, empty when the model called the goal reached or when the answer could not be read. |
↳ label | string | Is the step's own description. |
↳ ref | string | Is the element the step named. |
↳ value | string | Is what a fill or select step typed, as written, so a token stays a token and a real credential never reaches here. |
↳ ok | bool | Reports whether the turn produced a step the page accepted. |
↳ done | bool | Reports the model calling the goal reached. |
↳ error | string | Is why the turn produced nothing, in the words the next turn read. |
↳ url | string | Is where the page was when the turn was decided. |
count | int | How many steps the session holds |
ok | bool | Whether this turn's step reached the page and succeeded |
done | bool | Whether the model called the goal reached |
error | string | Why this turn was refused or failed, in the words the next turn reads |
data | string | What the applied step returned, which is what an extract step read off the page |
reasoning | string | The model's justification for this turn |
failures | int | How many turns in a row have now failed, reset to zero by a turn that worked |
budget | int | How many turns are left after this one |
match_url | string | The host and path the session's tab is now on, for the next turn to find it by |
url | string | Where the session is after this turn, which is where the page was observed |
continue | bool | Whether another turn is worth taking, meaning the goal is not reached, the budget is not spent and the model is still recovering from its own failures |
Example
Performs one model decision against the live page. It never fails on a step the page refused, so what a workflow reads afterwards is act.ok and act.error rather than a stopped run.
name: Perform one decided browser step
actions:
- name: act
component: machine:extension:apply
vars:
decision: "{{ decide.content }}"
observation: "{{ look.observation }}"
last_observation: "{{ var.last_observation }}"
steps: "{{ var.steps }}"
workflow_id: site-login
budget: 12
outputs:
steps: "{{ act.steps }}"
keep_going: "{{ act.continue }}"
why_not: "{{ act.error }}"