Status and known limitations¶
This page is for anyone who has to decide how far to trust the system: users, operators and contributors. It lists what has been verified and how, and what has not been verified or is not implemented. Every row names the place where you can check it.
The page was last checked against the working tree on 2026-11-10.
In three lines¶
- DiAGRI mode has only ever run on the simulated AC5 (
sim/device.py). In that mode nothing has written to a real device. - GrowBridge on its own (bridge-only mode) has read and controlled one real AC5 from a Mac. That record is older than the rest of the DiAGRI code.
- Nothing has been verified on a Raspberry Pi, with real notification services, or with CSC.
Verified¶
| What | How it was verified | Where to check |
|---|---|---|
| The test suite | 2274 code tests pass on macOS with Python 3.14 (last run 2026-11-14). CI runs the same tests with Python 3.11 and 3.13. The count leaves out tests/docs/, which checks the links of this wiki and grows with it. |
tests/, run with .venv/bin/pytest -q |
| Continuous integration | Passes on ubuntu-latest and macos-latest with Python 3.11 and 3.13. It runs ruff check, ruff format --check, pytest -q, python -m build, a syntax check of the dashboard and console scripts and the console's 142 Node unit tests. On macOS it also compiles the Swift helper, but does not run it. |
ci.yml |
| Coverage | 96 % of the statements in growbridge (396 of 9192 not run), measured on 2026-11-13 on macOS with Python 3.14 and the whole suite. CI does not measure coverage. |
.venv/bin/pytest -q --cov=growbridge; not recorded in the repository |
| A whole research round | From creating the project to delivered, with a manual takeover and a handback in the middle, on the simulated AC5. |
test_e2e_round.py |
| The model roles, end to end | Orchestrator, planner and researcher driven by a scripted model client, with no network and no key, from a delegation to a delivered round. | test_e2e_agents.py |
| A confirmatory conclusion through the workbench | Four rounds on the simulated AC5, every operator step an HTTP request to /wb. The analysis script is frozen before the first stage, freezing is refused once a stage has started, a report is refused until the frozen analysis has run on each snapshot, and the report is then confirmatory and the round delivered. The code sandbox is a fake in this test. |
test_e2e_confirmatory.py, test_workbench_analysis.py |
| The operator console | On 2026-11-13, by hand in a browser on the simulated AC5 (scripts/dev_workbench.py), at widths of 1440, 1024 and 768 pixels, in Chinese and in English, on both entries: sign-in, inbox, projects, creating a project, saving a protocol, setting a call budget, starting a round, submitting a goal, a stage from approval through start, readings in the chart, takeover, a manual light write with its receipt, irrigation on the LAN entry (disabled on the overlay entry), handback and stop, the round choice, alerts, entering an evidence card, the compute page without a compute service, and a change of the safety limits with the re-authentication dialog. No page scrolled sideways, and the focus ring and text contrast were checked. Not exercised: registering a holdout, verifying a card a model proposed, the compute page against a configured service, and anything on a phone-sized screen. |
web/workbench/, tests/web/, Development setup, testing and CI |
| A round through the workbench routes | From the login to delivered on the simulated AC5. Every operator step is an HTTP request to /wb, sent by an in-process client to the app that serve builds; the model roles are a scripted client. A second test stops a stage, chooses redo and aborts the round. No serve process and no real model took part. |
test_e2e_http_round.py, test_workbench_rounds.py |
| A stage runs without a model | An approved stage runs to its end with no model client, and also while every model call fails. tick never calls a model. |
test_e2e_runtime.py, test_e2e_agents.py |
| Restart and recovery | A restart puts a live stage into recovery_check, then paused, and the operator resumes it. A stage that was waiting for review goes back to pending_review by itself. A lost write reply pauses the stage. A recovery number bump voids approvals and sessions. |
test_e2e_runtime.py, test_e2e_reviewed.py |
| Why a stage waits | A stage that the system pauses or puts into the recovery check raises the alert stage_waiting, which names the event and what it carried; the same detail is kept in the stage history. The alert resolves when the stage leaves that state. A wall-clock step pauses a running stage once, and the operator's resume of a stage it paused clears it; starting another stage does not. |
test_e2e_stage_alerts.py, test_executor_clock.py, test_workflow_events.py |
| Running with no notification channel | Nothing is queued, the warning notify_unconfigured stands, and the run health is not affected. Open alerts are sent once a channel is configured. |
test_e2e_notifications.py |
growbridge simulate |
One approved stage runs to ended on the simulated AC5, with three confirmed writes. |
Tutorial, test_e2e_cli.py |
| GrowBridge alone (bridge-only mode) on a real AC5 | By hand, on macOS, with one AC5 (advertised as SF-GGS-PS5, firmware 3.2, plaintext protocol v1): scan, connect, live telemetry, a light write confirmed by read-back, an MCP client, the dashboard. Recorded on 2026-10-29, before the rest of DiAGRI was added. No later check on a real device is recorded. | VALIDATION.md |
Not verified or not implemented¶
"Not verified" means the code exists but has never run under the real condition. "Deployment item" means the design needs it and the code leaves room for it, but nothing in the repository sets it up. "Not implemented" means the code is missing.
| Area | State | Detail | Where to check |
|---|---|---|---|
Multi-user (multi_user) |
verified in tests only | Accounts, greenhouse locks, owners and per-operator visibility are tested on the simulated AC5 and through the workbench routes, not with real users or devices. One process still drives one AC5: the other registered greenhouses can be locked but run no stage until multi-device support exists. Releasing a greenhouse leaves the device at its current settings. The shared-token bridge routes (/status, /history*, /activity), the CLI status/dashboard and the MCP read tools still show every device's data to whoever holds the token. Protocol, run-sheet, stage, experiment, measurement and control request ids are global: guessing another operator's id tells that it exists (nothing more). Open device alerts go to whoever holds the greenhouse now, and may name a reading from before their lock (hiding them would hide live faults). Until the next stage starts, new samples are still filed under the released stage's experiment. Notifications go to the configured recipients, not per operator. |
workflow/access.py, adaptation notes |
| DiAGRI mode on a physical AC5 | not verified | Never run. This includes a Bluetooth interruption in the middle of a stage, power loss of the host or of the AC5, and long unattended runs. | VALIDATION.md |
| Pump outlet read-back | Not verified | The watchdog relies on the pump outlet's on read-back in the device status. It was tested on the simulated AC5 only: the read-back format, when it updates and how soon after a panel switch-on it shows have not been checked on a real unit. When the outlet cannot be read the watchdog only warns and does nothing else. |
executor/watchdog.py, control/device.py |
| Raspberry Pi and Linux Bluetooth | not verified | BleakTransport (Bleak on BlueZ) has only run against a fake client in the tests. No run on a physical Raspberry Pi is recorded. |
transport.py, tests/transport/ |
| The Swift helper's exit when its event channel closes | not verified | On macOS the helper is meant to drop its queued writes and exit when the Python side hangs up (watchChannel). CI only compiles the helper. |
BLEHelper.swift |
| Other controllers, other firmware, encrypted protocol v2 | not verified | Only one AC5 with plaintext protocol v1 has been used. Control requires protocol version 1 and is refused otherwise. | service.py, VALIDATION.md |
The background tasks of serve |
no automated test | The 1 Hz tick loop (_tick_forever), the model-role polling every 30 seconds (_agents_forever) and the second listener for the LAN entry (_serve_listeners) are not covered by any test. The tests call Runtime.tick() and Runtime.run_agents() directly. |
cli.py |
| Real Telegram and email delivery | not verified | TelegramChannel and EmailChannel have only been exercised with fake transports. |
observer/channels.py, test_observer_channels.py |
| External heartbeat | wired into serve, not verified against a real service |
With GROWBRIDGE_HEARTBEAT_HOST_URL and GROWBRIDGE_HEARTBEAT_RUN_URL both set, serve beats once every 60 seconds and at once when the run health turns; one variable alone stops the start; neither raises the standing warning monitor_unconfigured. Tested with a fake HTTP client only: no signal has reached a real monitoring service. |
runtime_heartbeat.py, test_e2e_heartbeat.py |
| Host time synchronisation | not implemented | Nothing checks whether the host clock is synchronised: SystemClock.synced stays true. The only clock check is a step of more than 5 seconds between the wall clock and the monotonic clock, which pauses a running stage until an operator resumes it. A host that sleeps looks like such a step when it wakes. |
archive/clock.py, executor/outage.py |
data and provider of a model role |
not enforced | Both fields are checked on load and read by no code. data does not restrict what a role can read; what a role can reach is set by its tools. |
config.py, agents/tools.py |
| Model prices | not verified | The built-in price of the default model (DEFAULT_PRICE) is a constant in the code and has not been checked against the provider's current price list. Every other model takes its price from the configuration. |
agents/llm.py, Configuration reference |
| OS-account isolation | deployment item | The code is one process. It keeps one SQLite file per service (control, workflow, archive, observer, compute, agents), which is what separate accounts would be built on. |
runtime.py (DATABASES) |
| Overlay network | deployment item | The daemon listens on 127.0.0.1 and expects a proxy in front of it for the overlay entry. The repository contains no proxy or network configuration. |
workbench/auth.py (entry_for) |
| Passkeys | not implemented | An operator signs in with a secret of at least 12 characters, stored as a salted scrypt hash. | workbench/auth.py |
| Entering a research goal | from the goal to a delivered report, on the simulator | An operator creates the project, the research protocol and the round, sets the call budget and states the goal through /wb routes, or in the conversation on the project page. On 2026-11-14, on the simulated AC5 (scripts/dev_workbench.py --claude-code, the roles on claude-opus-5-5 through the Claude Code CLI), a goal went from the orchestrator to the planner, which saved a validated run sheet; the orchestrator was woken by the follow-up and told the user to review it. The same day a whole round ran this way, with --fast-forward: approval, a 14-day stage, measurements, the researcher's inspection, freeze and report, delivered. On 2026-11-15, after the last fixes of pull request 4, a second round ran the same way, with a 21-day stage and the manual-task records (Example: a complete round); the open points it found are listed there. Not run against serve, a real device or a model API key. There is no CLI command. |
workbench/rounds.py, Run a research round |
| Workbench routes that are still missing | not implemented | No route reads the content of a saved research protocol, settles a research task, reads the data snapshot or a report, revokes an approval, or switches the run sheet of a live stage. Projects, rounds, tasks, the stage history and the inbox are listed by the read routes. Nothing records which operator created a project, a protocol or a round, or set a budget. | workbench/api.py, workbench/rounds.py, workflow/service.py |
| The operator console | checked on the simulated AC5 only | The page at /workbench exists (see the Verified table above). It has not been used against a real device, a real notification channel or CSC, in Safari or Firefox (only the browser pane of the development tool was used), or on a phone. Times are shown in the zone of report_timezone. Registering a holdout still means typing the batch ids by hand. The console sends the digest of the run sheet it showed when an operator approves, and the server refuses a sheet that changed since (expected_digest); the console's safety page refuses to approve when the approved limits changed while the operator was reviewing (it reads them again before and after the confirmation), but the server has no such precondition on safety approval yet, which is a follow-up. |
|
| Role tokens on the workbench routes | known limitation | A model role holding a role token can read the runtime of any stage and submit run-sheet drafts to any round it names. This existed before the console. Nothing runs without an operator's approval, which now requires the digest of the reviewed run sheet. Scoping these routes for role tokens, or closing them, is a follow-up. | workbench/api.py |
| Jobs on CSC | wired into serve, not verified against CSC |
When compute_templates_dir, compute_member and the seven FIRECREST_* variables are all set, serve builds the compute service over FirecrestBackend. It follows the CSC and pyfirecrest documentation and has only been tested against a fake client. The API reports no CPU use and no billing balance: each job is charged at its template's full max_cpu_hours (and max_storage_gb), and an operator enters the project balance and each task's budget through the workbench (POST /wb/compute/balance, POST /wb/compute/tasks/{task_id}/budget). The Slurm state names are not documented: a state the code does not recognise (for example SUSPENDED) is an error, never "completed". Nor is it documented whether the job lookup finds jobs that have already finished, so a finished job that FirecREST no longer lists cannot be fetched or cleaned up. The tick reconciles unanswered submits and polls live jobs every minute; where the outcome is unknown (no job found, a cancel whose result cannot be read) the attempt is charged its whole reservation, and an operator can settle an attempt nobody can recover (POST /wb/compute/attempts/{attempt_id}/settle; GET /wb/compute lists the stuck ones). The states DEADLINE, PREEMPTED, BOOT_FAIL and REVOKED end an attempt as failed, REQUEUED and REQUEUE_HOLD keep it queued, SUSPENDED and STOPPED keep it running; this mapping follows the general Slurm manual, not CSC documentation. Without the full configuration the compute tools answer unavailable. |
compute/firecrest.py, runtime_compute.py, workbench/compute.py |
| Literature search | not implemented | There is no search backend. The planner tools lookup_crop_conditions and search_and_verify_sources answer unavailable. |
agents/handlers_planner.py, runtime_agents.py |
| Sandbox for model-written code | macOS only | The sandbox uses sandbox-exec. Everywhere else, Linux and the Raspberry Pi included, it refuses to run anything, with the denial sandbox_unavailable. That stops local analysis and the confirmatory analysis there. |
research/sandbox.py |
| Confirmatory analysis | verified in tests only | The operator freezes the script and starts each run through the workbench; the end-to-end test uses a fake sandbox. Each snapshot runs once, and not one run over all rounds of a series as section 6.5 of the architecture document describes. A process that dies while a run is going leaves that run open until the next start, which settles it as failed and marks it interrupted: the attempt stays used up, and a report that cites the snapshot is then published as exploratory. Holdouts are registered through the workbench, before their outcome data exists; they cannot yet be used to evaluate a model (see below). |
workbench/analysis.py, research/analysis.py, Research methodology |
| Holdout data for model evaluation | not implemented | Holdouts can be registered, and they hide the held-out outcomes from development. Nothing scores a model on them: registering a model and evaluate only check hashes and count reads, and no route registers a model. Scoring needs a design that does not show the held-out outcomes to the model. |
research/holdout.py, workbench/holdouts.py, Research methodology |
| Alert rules | empty by default | DEFAULT_ALERT_RULES is empty and there is no configuration field for alert rules, so the observer raises no threshold alert. |
runtime.py |
| Camera capture | not implemented | There is no camera code. Growth observations are entered as manual measurements. | observer/report.py |
| Backup | SQLite databases only | A backup copies the six DiAGRI databases. It does not copy the bridge history file growbridge.sqlite3, or the files under archive/ in the data directory, such as snapshots. There is no image capture yet, so there are no images to back up. Without backup_dir nothing is backed up, and a standing warning says so. |
runtime_backup.py, archive/backup.py |
| Snapshot freezing on the event loop | known limitation | Freezing a data snapshot (freeze_dataset) and re-checking it before a local analysis read and hash its files while holding a database transaction, on the same event loop that runs the 1 Hz executor tick. For a long stage this can pause the tick for seconds. The sandbox run itself, and the checks before a confirmatory run, are off the loop. |
archive/snapshot.py, agents/handlers_researcher.py |
| The auditor | not verified | The auditor has run only with a scripted model in tests, never with a real model. Its note, its findings and how it uses its 8 calls have not been seen under a real model. | agents/handlers_auditor.py, runtime_agents.py |
| Frozen-reading thresholds | not verified | The default thresholds (temp 120, humi 60, co2 60, tempSoil 240 minutes) come from 64 minutes of idle data. They need to be tuned with real experiment data before the pause on a frozen reading is relied on. | observer/frozen_watch.py |
| Skipped tests | skipped, not passed | Tests that need sandbox-exec are skipped where it is missing, which is the case on Linux CI. Tests that build the Anthropic client are skipped where the anthropic package is missing; CI installs only the dev extra, so they are skipped on both CI systems. |
test_research_sandbox.py, test_e2e_assembly.py, test_agents_llm.py |
How to check again¶
.venv/bin/python -m pip install -c constraints-tested.txt -e '.[dev]'
.venv/bin/pytest -q
.venv/bin/growbridge simulate
The first command installs the test tools. The second runs the whole suite on the simulated AC5; add -rs to list the skipped tests and the reason for each. The third runs one simulated stage, as described in the tutorial.
Checks on real hardware are recorded in VALIDATION.md. A row moves to the first table once its check is recorded there.
See also¶
Pages were last checked against the code on 2026-11-11. DiAGRI mode has been tested on a simulated AC5 only: see Status and known limitations. To change a page, read how this wiki is edited first.