Labs

The Labs you can evaluate in today, what each one reads back, and what it does not establish.

A Lab is where laboratory work happens in an evaluation: a simulator, a mock instrument, and later real instruments. It gives the agent scoped tools and gives the evaluator a separate Readout of what happened. Labs are selected by name and swapped without changing the task, the way Inspect AI sandboxes are.

Current Labs are building blocks: a meaningful study also needs domain-reviewed tasks, legitimate controls and valid outcome measures.

Lab	Native task	Mode
Liquid handling	`inspect_labs/serial_dilution`, `inspect_labs/worklist_transfer`	Simulation
Measurement and report services	`inspect_labs/measurement`, `inspect_labs/handoff`	Synthetic
Plate-reader QC	`inspect_labs_plate_reader/absorbance_qc`	Synthetic
SiLA 2 instrument	`inspect_labs_sila/absorbance_read`	Mock instrument
Screening review	`inspect_labs_commec/screening_review`	Gate pending
Robot steps	`inspect_labs/robot_step`	Software controls

Each scripted control below runs from an installed checkout without model credentials.

Liquid handling

An agent plans and runs pipetting work on a PyLabRobot deck. The lab scorer reads the deck, not the agent’s account of it.

Observations: Deck state, volumes, concentrations, tip use and restricted-well events
Backends: Built-in simulator; an OT-2 simulator and OT-2 declarations in the Opentrons plugin
Limits: The default backend is simulated and assumes ideal mixing. No physical instrument has been validated.

inspect eval inspect_labs/serial_dilution \
  --model mockllm/model -T scripted=true

Measurement and report services

Queued services that separate a request being accepted from work being done, and a report from the result.

Observations: Job identity, state and recorded result; delivery destination and content identity
Limits: Synthetic fixtures that exercise software mechanics. A robot-gated handoff does not demonstrate physical sample transfer.

inspect eval inspect_labs/measurement \
  --model mockllm/model -T scripted=true

Plate-reader QC

An absorbance quality-control task through PyLabRobot’s device-free plate-reader interface. Requires the plugins/inspect-labs-plate-reader plugin.

Limits: Seeded synthetic values. Not a validated assay or a physical reader.

inspect eval inspect_labs_plate_reader/absorbance_qc \
  --model mockllm/model -T scripted=true

SiLA 2 instrument

A mock absorbance reader that speaks SiLA 2, the open standard many lab instruments use to expose commands. The agent’s tool calls the instrument’s ReadWell command over SiLA 2. The evaluator reads the instrument’s own run log through a separate channel. Requires the plugins/inspect-labs-sila plugin.

Readout: The instrument’s run log: every executed command, with the well and value
Limits: Seeded synthetic values on a local mock server. No real instrument has been connected, and the mock does not simulate optics or an assay.

inspect eval inspect_labs_sila/absorbance_read \
  --model mockllm/model -T scripted=true

Screening review

A bounded review of a sequence-screening result using IBBIS’s open-source Common Mechanism (commec) and a simulated disposition service. Requires the plugins/inspect-labs-commec plugin.

Limits: Fixture mechanics are verified. The full-reference scanner runtime is a pending gate, and no real order is fulfilled. Scanner recommendations are not biological ground truth or authorization. No IBBIS affiliation is implied.

inspect eval inspect_labs_commec/screening_review \
  --model mockllm/model -T fixture=true -T scripted=true

Robot steps

A laboratory step executed by any registered Inspect Robots task, policy and embodiment, under its native guardrails. Requires the robots extra.

Observations: Native Inspect Robots trial records and metrics, linked by hash
Limits: Deterministic software controls only. No physical validation and no robot foundation model study.

inspect eval inspect_labs/robot_step \
  --model mockllm/model -T scripted=true

Discover installed components

Environments and instrument backends are plugins discovered by name:

inspect-labs list
inspect-labs doctor --lab liquid-handler

doctor checks declarations and requirements without constructing providers or touching hardware. It does not run lifecycle conformance or verify that an instrument or external runtime works.

Install optional plugins

Install plugins from the checkout into the same Python environment:

uv pip install --python .venv/bin/python \
  ./plugins/inspect-labs-opentrons \
  ./plugins/inspect-labs-plate-reader \
  ./plugins/inspect-labs-commec \
  ./plugins/inspect-labs-sila

Plugin	Provides	Current limits
Opentrons	OT-2 backend declarations and a simulator	Simulator mechanics do not establish hardware compatibility
Plate reader	Native absorbance QC task through PyLabRobot’s device-free reader API	Seeded synthetic values, no validated assay or physical reader
SiLA 2	A Lab backed by a mock SiLA 2 instrument; Readout from the instrument’s run log	Local mock with seeded values; no real instrument connected
Common Mechanism	Bounded screening review with IBBIS Commec and a simulated disposition service	Fixture mechanics verified; full-reference runtime gate pending; no real order fulfillment

Each plugin directory includes its own README, controls and tests.

Robot integration

uv pip install --python .venv/bin/python '.[robots]'

The extra installs a pinned Inspect Robots commit. Robot tasks use its registered policies and embodiments. Robots come from Inspect Robots rather than being reimplemented. Physical embodiments require explicit identities for supported native device slots and host authorization before construction. Default guardrails remain native.

Physical systems are refused by default

A host must pass allow_physical=True. That assertion does not provide facility authorization, interlocks or a validated physical protocol. Device claims coordinate participating evaluations on one host.

Publish a plugin

Packages register Labs and backends using the Python entry-point groups inspect_labs.labs and inspect_labs.backends. The earlier inspect_labs.environments group is still read. Declare runtime requirements, device identities and computation, simulation or physical mode honestly. Existing clients continue to own operational jobs, access control and cancellation.

The detailed source guides are docs/adapters.md and docs/authoring.md in the checkout. The authoring guide introduces the public binding protocol.