Operate the fleet

Configure sites and accounts, release a spider version, submit jobs, and handle hand-overs and drift.

View as MarkdownAsk Claude about this page

You operate the fleet through its API on the control plane, with a bearer token. The examples use these variables:

export FLEET=http://localhost:7400
export TOKEN=admin-dev-token
alias fleet='curl -sS -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json"'

Sites and accounts

A site is where spiders run, with the limits the fleet keeps for it. Create or replace one with PUT /sites/<id>:

fleet -X PUT $FLEET/sites/portal -d '{
  "config": {
    "origin": "https://portal.example.com",
    "maxConcurrent": 4,
    "minIntervalMs": 2000,
    "windows": [{ "days": [1, 2, 3, 4, 5], "from": "07:00", "to": "20:00", "tz": "Europe/Rome" }],
    "sessions": "exclusive"
  }
}'
Field Meaning
origin The site’s address. A run opens it first unless the version says otherwise.
maxConcurrent Most runs at once on the site. A run paused for a hand-over counts, since its session is still live.
minIntervalMs Least time between two run starts on the site
windows Runs start only inside one of these: days (0 is Sunday), from and to as HH:MM, and tz. None means always.
proxy A host:port every run on the site goes through, for a dedicated egress address
window The browser window size. By default, the size the spider’s sessions were recorded at.
sessions How many sessions one account may have at once: exclusive, parallel or shared. See below.
maxSessions With parallel, the most sessions per account

Sessions per account. Set what the portal allows:

  • exclusive, the default: one live session per account. A second login would end the first, so the fleet never starts a run on an account another run holds, whichever spider it belongs to.
  • parallel: up to maxSessions independent sessions per account, each with its own login.
  • shared: several runs at once on one session. Only safe where the site does not rotate its tokens.

An account is a login on a site. Its password is not in the fleet, only in the vault:

fleet -X PUT $FLEET/accounts/ops@acme.it -d '{ "site": "portal", "username": "ops@acme.it", "props": { "branch": "MI-04" } }'

Release a spider

Define the spider

A spider definition belongs to a site and is shared by all its versions. It may restrict the site’s sessions setting, never widen it. A portal that allows parallel sessions for reading may lock a function such as signing a contract to one session per user:

fleet -X PUT $FLEET/spiders/switch_contract -d '{ "site": "portal", "sessions": "exclusive" }'

Add a version

A version is the spider IR plus how its inputs are filled in the fleet:

fleet -X POST $FLEET/spiders/switch_contract/versions -d @version.json
{
  "spec": {
    "kind": "ir",
    "ir": { "format": "graphdom-ir@1.0", "spider": "switch_contract", "…": "the contents of spider.json" },
    "sessions": ["2026-11-06T09-12-44-101Z-3f9a1c", "2026-11-06T10-02-13-554Z-77b0de"],
    "inputs": { "username": { "account": "username" } },
    "secrets": {
      "password": { "secret": "portal/{account}/password" },
      "otp": { "totp": "portal/{account}/totp" }
    }
  }
}

The answer is the version’s number. sessions lists the task’s recorded sessions, in the workers’ storage; the sandbox tests the version on each of them. inputs and secrets bind the spider’s inputs and secrets to where a run reads them. Inputs left unbound come from the job.

Source A run reads
{ "input": "name" } The job’s input of that name
{ "account": "username" } The job’s account: its username, or one of its props by name
{ "secret": "ref" } The vault entry ref. {account} in the name becomes the job’s account.
{ "totp": "ref" } A one-time code computed from the TOTP seed in the vault, when the step types it
{ "mailbox": "url", "pattern": "regex" } A code mailed to a test inbox after the run started

origins maps the spider’s origin names to other addresses, when the site is not where it was recorded, such as a staging copy. start overrides the first URL the browser opens.

A version that came from another, repaired or rebuilt from it, names it with parent beside spec, for example "parent": 2. The parent must be a version of the same spider. The fleet keeps the link, so the spider’s history shows where each version came from, and you can read what a version changed in IR terms:

fleet $FLEET/spiders/switch_contract/versions/3/diff             # against its parent
fleet $FLEET/spiders/switch_contract/versions/3/diff?against=1   # against another version

The answer lists the states, ways and steps added, removed, moved or changed, each with its reason in words: the same diff Studio shows between two versions.

Test and approve it

fleet -X POST $FLEET/spiders/switch_contract/versions/3/test

A worker runs the version in the sandbox against each recorded session, with the network cut. The report is attached to the version, and a version already tested on the same sessions is not tested again. Then an approver approves it:

fleet -X POST $FLEET/spiders/switch_contract/versions/3/approve -d '{ "note": "Matches the new POD search" }'

Approval is refused without a passing report. Approving a version retires the one approved before it: new jobs run the approved version, unless a job pins another with version.

Submit jobs

fleet -X POST $FLEET/jobs -d '{
  "jobs": [
    { "spider": "switch_contract", "account": "ops@acme.it", "inputs": { "pod": "IT001E12345678", "customer": "C-1042" } }
  ]
}'

The answer lists the new job ids. From a shell or a CI job, graphdom fleet submit does the same:

GRAPHDOM_TOKEN=$TOKEN graphdom fleet submit --control $FLEET --spider switch_contract --account ops@acme.it --inputs job.json --count 10

Follow them with GET /jobs/<id> and GET /runs?job=<id>. A queued job has a why that says what it waits for: an approved version, the site’s window or interval, a free session on the account, or a version’s first run to finish.

A new version’s first real run goes alone. It pauses before each submit step for an operator, and no other job of that version starts until it ends.

Hand-overs

A run pauses and waits for a person when:

  • it reaches a submit step of a version’s first real run (confirm-submit);
  • it meets a page its spider stops on, such as a CAPTCHA or new terms (stop);
  • a submit step failed after its request went out, so it cannot know whether the site received it.

GET /handovers?state=open lists them. For one hand-over, GET /handovers/<id>/frame returns a screenshot of the live browser, /screencast streams it, and POST /handovers/<id>/input sends clicks and keys to it. Then resume or abort:

fleet -X POST $FLEET/handovers/12/resume -d '{ "note": "Checked the POD, confirmed" }'

A paused run keeps its account’s session and its worker slot. A worker draining for a restart waits for it only up to its drain time.

Drift

A run that meets a page its spider does not know ends drift, with the step and the kind of miss. When two runs of the same version drift at the same step, the fleet opens an incident: the version is marked drifted and its spider’s jobs are held.

GET /drift lists incidents. To release the jobs, build a repaired version from new recordings, add it, test it and approve it. Its first successful run resolves the incident. A single drifted run followed by a successful one is treated as a one-off and closes on its own.

Audit

Every change goes to the audit log: who approved what (the approving user’s id, also in GET /approvals), each job’s change of state and of waiting reason, each hand-over decision. Read it with GET /audit, filtered by kind or job. Every run’s session is archived in your storage, redacted.

API

All endpoints take Authorization: Bearer <token>. A token acts as a user, and that user’s roles decide what it can call. An admin can call every endpoint.

Method and path Role Use it to
GET /me any The token’s user: id, name, e-mail, roles
GET /users, PUT /users/<id> operator, admin List users, create one or change its name and roles
GET /sites, PUT /sites/<id> operator, admin List sites, create or replace one
GET /accounts, PUT /accounts/<id> operator, admin List accounts, create or replace one
GET /spiders, GET /spiders/<id> operator List spiders, or one with its versions
PUT /spiders/<id> admin Define a spider
POST /spiders/<id>/versions admin Add a version, with the parent it came from
GET /spiders/<id>/versions/<n>/diff operator What a version changed against its parent, or ?against=<m>
POST /spiders/<id>/versions/<n>/test admin Test a version in the sandbox
POST /spiders/<id>/versions/<n>/approve approver Approve a tested version
GET /approvals operator List approvals, ?spider=
POST /jobs admin Submit jobs, each recording who submitted it (by)
GET /jobs, GET /jobs/<id> operator List jobs, ?state= and ?spider=
GET /runs, GET /runs/<id> operator List runs, ?job=, ?state=, ?spider=
GET /handovers, GET /handovers/<id> operator List hand-overs, ?state=
GET /handovers/<id>/frame, /screencast operator See the paused browser
POST /handovers/<id>/input operator Type and click in it
POST /handovers/<id>/resume, /abort operator Decide
GET /drift operator List drift incidents, ?state=
GET /audit operator Read the audit log, ?kind= and ?job=
GET /healthz, GET /metrics none Health and Prometheus metrics