Open
Milestone
Sep 30, 2026–Oct 31, 2026
Bench refinement
🎯 Objective
Fix what the first plesty-bench 0.4 installs on lab benches turned up, so an experiment can be configured, run and inspected from the console, and a bench's fleets are explicit to its agent, its console and remote clients.
Tasks
| id | description | related issues | signature | implementation status |
|---|---|---|---|---|
experiment-manifest |
Configuring an experiment fails with "the module does not publish a manifest. error: unrecognized arguments: --manifest" (plesty-bench 0.4.0, plesty-lib 0.5.0, Windows). | plesty/core/plesty-bench#15 (closed) | Done: plesty-pol-pl 0.1.0 released; plesty/core/plesty-bench!19 (merged) merged (maintain/0.4, exp); shipped in 0.4.2 (2026-10-03) |
|
completed-run-config |
The Configure dialog of a completed experiment can neither scroll the configuration nor open the YAML. The full configuration must be viewable, read-only. | plesty/core/plesty-bench#16 (closed) | Done: plesty/core/plesty-bench!20 (merged), tested on Windows; shipped in 0.4.2 | |
benches-save-edit |
In Server ▸ Benches…, Save writes the old value of a cell edited without pressing Enter. | plesty/core/plesty-bench#17 (closed) | Done: plesty/core/plesty-bench!21 (merged), tested on Windows; shipped in 0.4.2 | |
fleets |
A new bench starts with an empty fleet; a remote client can create fleets and declare into one; the console shows one fleet or all of them, all by default. | plesty/core/plesty-bench#18 (closed) | Done: plesty/core/plesty-bench!45 (merged), !47 (exp), closed 2026-10-20; ships in 0.5.0 |
|
client-replies |
The remote client mis-reads the agent's reply: install crashes with JSONDecodeError, jobs cancel prints [], -- field-test --help is garbled — though each command succeeds on the bench. |
plesty/core/plesty-bench#19 (closed) | Done: plesty/core/plesty-bench!22 (merged), tested on Windows; shipped in 0.4.2 | |
bench-data-path |
Each bench has a data path on the shared disk, kept in fleet.yaml: the installer asks for it (with Skip), Server ▸ Configure… sets it, and every device the bench starts gets DEVICE_DATA_PATH=<data path>/<device>. Today a device started from the bench saves into the server's working directory. An experiment routes a device with a path relative to that data path, so it never needs to know how another bench mounts the disk. |
plesty/core/plesty-bench#20 (closed), plesty/core/plesty-bench#35 (closed), plesty/core/plesty-lib#59 (closed) | Done: plesty/core/plesty-bench!45 (merged), !48, plesty/core/plesty-lib!46 (merged), !47 (exp), closed 2026-10-20; ships in plesty-bench 0.5.0 and plesty-lib 0.6.0 (Breaking: set_data_path is relative to the bench folder) |
|
configure-to-run |
Configure saves an experiment's settings to the run directory, never into the module's shipped config files, and changes only the edited values (comments and layout kept). Found in the 0.4.2 candidate's Windows test. | plesty/core/plesty-bench#21 (closed) | Done: plesty/core/plesty-bench!45 (merged) (exp), closed 2026-10-20; ships in 0.5.0 |
|
bench-app-recovery |
The bench app recovers from what went wrong in a session instead of stalling: a failed call no longer leaves the device panel stuck; Clear log on a running device no longer hangs the app and the agent; a reopened app comes back to the fleet that was open, so running servers are never orphaned; a long Source value no longer widens the window for good. | plesty/core/plesty-bench#22 (closed), plesty/core/plesty-bench#23 (closed), plesty/core/plesty-bench#24 (closed), plesty/core/plesty-bench#25 (closed) | Done: plesty/core/plesty-bench!34 (merged), !35 (maintain/0.4, exp), plesty/core/plesty-lib 0.4.3 / 0.5.2; tested on macOS; ships in plesty-bench 0.4.3 |
|
run-table |
The run table shows what is on disk and what produced it: a created run appears at the top, selected, and a deleted one disappears at once; a run can be renamed before it starts, and Create run says Experiment name and Run name; Started and Ended carry the date; each run's Source is the version that measured it, not the current install. A run keeps its own log beside its data, so it is still readable after an uninstall or from another bench. Found on the Windows bench, 2026-10-16 and 2026-10-19. | plesty/core/plesty-bench#27 (closed), plesty/core/plesty-bench#30 (closed), plesty/core/plesty-bench#31 (closed), plesty/core/plesty-bench#32 (closed), plesty/core/plesty-bench#33 (closed) | #31, #32 merged (plesty/core/plesty-bench!38 (merged), maintain/0.4, exp), shipped in 0.4.4 (2026-10-20); #27, #30 merged (plesty/core/plesty-bench!39 (merged), exp), ship in 0.5.0; all four tested on Windows (TEST 0.4.4.dev16), 2026-10-19, closed. #33 merged (plesty/core/plesty-bench!41 (merged), exp), ships in 0.5.0; tested on Windows (TEST 0.4.4.dev18), closed |
|
one-app-copy |
One copy of the app per bench home: a second launch brings the open window forward instead of starting another that shares the jobs and runs; a finished run's log is read once, not reprinted. Found on the Windows bench, 2026-10-16. | plesty/core/plesty-bench#28 (closed) | Merged (plesty/core/plesty-bench!38 (merged), maintain/0.4, exp), shipped in 0.4.4 (2026-10-20); tested on macOS and on Windows (TEST 0.4.4.dev16), 2026-10-19, closed |
|
device-server-sessions |
A run holds a device's control lease: other clients still read, but only the holder writes or runs operations; the bench control panel opens read-only while a run holds the device. A TimeoutError raised by the device itself is reported with its own message, not as the server's "Operation timed out". |
plesty/core/plesty-lib#57, plesty/core/plesty-lib#60 (closed) | #60 merged (plesty/core/plesty-lib!35 (merged)), shipped in plesty-lib 0.4.2 / 0.5.1. #57 open, priority low: no conflicts seen since build_server serialises calls by default; moves to the next milestone |
|
scaffolds-and-field-tests |
Scaffolds and field tests match the current release: new modules pin the current SDK and the gate guidance points at what exists; a field test can give each operation the arguments it needs, on both tiers, and generated reason comments stay within the line limit; plesty-lib's own checks run the current SDK. | plesty/core/plesty-sdk#44 (closed), plesty/core/plesty-sdk#45 (closed), plesty/core/plesty-lib#58 (closed), plesty/core/plesty-lib#65 (closed) | Done: plesty-sdk#44, #45 shipped in plesty-sdk 0.4.2 / 0.5.2 (2026-10-20); plesty-lib#58, #65 shipped in plesty-lib 0.4.2 / 0.5.1 (2026-10-15) | |
experiment-setups-at-scale |
One experiment module can ship several setups and the run's configuration picks one, so only the devices that configuration needs are served and share-checked. Building a long plan (200,000+ steps) takes seconds, not minutes. | plesty/core/plesty-lib#62 (closed), plesty/core/plesty-lib#63 (closed) | Done: #63 shipped in plesty-lib 0.4.2 / 0.5.1. #62 merged (plesty/core/plesty-lib!48 (merged), plesty/core/plesty-bench!50 (merged), exp), closed 2026-10-21; ships in plesty-lib 0.6.0 and plesty-bench 0.5.0 |
|
reports-and-viewers |
Run reports and viewers stay readable and robust: a figure's legend no longer covers the data; a run written from macOS to the shared disk still gets its report (AppleDouble ._ files are skipped); a long plot caption no longer pins its column in the viewer; a replay can be paused. |
plesty/core/plesty-lib#61 (closed), plesty/core/plesty-bench#26 (closed), plesty/core/plesty-lib#64 (closed), plesty/core/plesty-lib#66 (closed) | Done: plesty-bench#26 shipped in 0.4.3, tested 2026-10-19 with a run written from macOS to the share. plesty-lib#61, #64 shipped in plesty-lib 0.4.2 / 0.5.1. #66 merged (plesty/core/plesty-lib!49 (merged), exp); ships in plesty-lib 0.6.0 |
|
device-module-fixes |
Device modules found broken on real benches: the DMK 37AUX290 camera's identity() crashes with an access violation, so every experiment preflight refuses a rig that has it. APT controllers bound to FTDI's D2XX driver (Kinesis installed, no COM port) cannot be reached. |
plesty/hub/devices/others/cam_dmk_37aux290#1 (closed), plesty/core/plesty-lib#67 (closed) | Done: camera fix shipped in cam_dmk_37aux290 0.1.1 (2026-10-21). plesty-lib#67 merged (plesty/core/plesty-lib!51 (merged), exp); ships in plesty-lib 0.6.0 |
|
install-update-fixes |
An update of a device installed from git reports the version it now runs, and takes a field-test report the module committed instead of refusing to pull. | plesty/core/plesty-bench#36 (closed), plesty/core/plesty-bench#37 (closed), plesty/core/plesty-bench#44 (closed), plesty/core/plesty-bench#45 (closed) | #36, #37 merged (plesty/core/plesty-bench!49 (merged), maintain/0.3, merged up to maintain/0.4 and exp), closed 2026-10-20; unreleased — waits for patch tags 0.3.4 / 0.4.5. Found in the #38 Windows test, both from maintain/0.4: #44 (Load from… on a detached checkout installs with --ref "(no branch)") and #45 (a retried uv sync reuses a half-made .venv and always fails) — open |
|
bench-install-update-ux |
Bench: Installation & Update UX: Add an automatic startup version/update check and a 1-click update mechanism (or clear step-by-step guidance) for the bench and installed modules. Fix the broken menu update script and resolve Windows SmartScreen/Antivirus warnings. One Start menu entry, PLESTY Bench; the update moves into Help ▸ Check for updates…, with an update window and a Help ▸ About that tells the version, install and last update. The uninstaller lists what the bench holds, one item per module, fleet and device configuration, each ticked on its own. | plesty/core/plesty-bench#29 (closed), plesty/core/plesty-bench#34 (closed), plesty/core/plesty-bench#39 (closed), governance#26 | #29 merged (plesty/core/plesty-bench!40 (merged), !42, exp), ships in 0.5.0; tested on Windows (TEST 0.4.4.dev16, and the update 0.4.2 → 0.4.3 with its window, 2026-10-19) and on macOS, closed. #34 merged (plesty/core/plesty-bench!41 (merged), exp), ships in 0.5.0; tested on Windows (TEST 0.4.4.dev18), 2026-10-19, closed. Still open — taking the update check at startup. Not in code: the SmartScreen / antivirus warnings need a code-signing certificate for the installer — can the university buy one? |
|
demo-onboarding-single-pc |
Bench: Single-PC Onboarding & Demo Setup: Streamline the zero-hardware demo to run seamlessly on a single PC without network/cross-bench errors. Add a startup welcome screen (or "Home" tab) guiding new users through installing demo devices and running their first experiment. | plesty/core/plesty-bench#43 (closed) | Taking, after module-lifecycle-catalog: the welcome screen needs installs that work. Experiments already ship a config/demo.yaml |
|
module-lifecycle-catalog |
Bench: Module Lifecycle & Catalog Cleanup: Unify "Hub" naming across the UI, fix dropdown version selection, and prevent cross-type installation bugs (e.g., adding devices as experiments). Provide clear success/failure confirmations in the UI, provide an overview of installed modules and add module uninstallation capabilities. | plesty/core/plesty-bench#38 (closed) | Each Add dialog offers only its own kind: plesty/core/plesty-bench!51 (merged) merged (maintain/0.4, exp), ships in 0.4.5 (0.3 installs devices only and is not affected). Every PyPI release and git tag in the version drop-downs, Hub ▸ Installed modules… with Uninstall (refused while a declaration uses the module), a message after every install, update and uninstall, and Hub throughout: plesty/core/plesty-bench!52 (merged) merged (exp), ships in 0.5.0. Windows test 2026-10-26: items 1, 2, 3 and 5 pass; item 4 (the install and uninstall messages) failed, fixed in plesty/core/plesty-bench!53 (merged) (exp, which also moves Add device… to the Device menu). Re-test on Windows passed; closed 2026-10-26. Untested: the message after an update, while hub.plesty.net is down |
|
gui-layout-data-viz |
Bench: UI Layout, Visualizations & Data Focus: Restructure the GUI to prioritize real-time data visualization (occupying most of the screen during runs) and adopt vertical tabs with distinct background color coding and icons. Add a global status banner showing experiment progress and device states. Replace some text-based buttons with intuitive items (and text shows when hovering with the mouse over it). | plesty/core/plesty-bench#41 (closed) | Taking part: the status banner, and icon buttons that name themselves on hover. Changed: the layout redesign (vertical tabs, colours, live data filling the screen) starts from a mock-up for the PO to approve before it is built | |
journal-and-operator |
Bench: Integrated Lab Journal & Bug Reporting: Add a "Journal" tab that contains the experiment history and possibility for operators to add entries. Operators should be prompted to record notes and confirm operator names before experiment runs. The journal is then stored in the datasets for the experimental runs. | plesty/core/plesty-bench#40 (closed) | Taking part: Create run asks for the operator and a note, saved in the run's folder beside its log. A Journal tab with history comes once those fields are in use | |
dedicated-analyzers |
Bench: Allow for dedicated analyzer installation. Allow installation of analyzers from the HUB. Add an "Analyzers" tab where an analyzer can be opened and configured. E.g. the user needs to provide file location (e.g. from a previous experimental run) and manual configuration options. Then after analysis there should be visualization according to how it is set up in the analyzer. | plesty/core/plesty-bench#42 (closed) | Not taking (plesty/core/plesty-bench#42 (closed) closed). The bench keeps the simple analysis that runs during the measurement; replay and the run report reuse it. Detailed analysis is each user's own, in Python, with their own methods and plots — standardising it would take that freedom away | |
bug-report |
Bench: Integrate a 1-click "Send Bug Report" button directly in the UI and error dialogs that submits logs automatically without requiring a personal GitLab login | plesty/core/plesty-bench#14 (closed) | Not taking. Help ▸ Report a problem… already collects the system state and logs, shows them for review, and opens a filled-in issue. Sending without a login would need a token that can write to GitLab on every lab PC | |
feature-request |
Bench and website: Integrate a public feature voting tool (e.g., via ProductBridge) where users can quickly create and vote for new features, without a login. | Not taking for now: votes on GitLab issues give the same signal at our size, without an outside service and its privacy terms on the website. Revisit when users outside the group arrive | ||
website-navigation-refresh |
Website & Marketing Refresh: Simplify the homepage layout into a single, un-nested navigation bar (Features, Hub, Learn, Blog, Download, Support, GitLab, Join). Merge download and quickstart into a unified "Get Started" path, update feature copy from a researcher's perspective, and add a "Blog" section for dev reports. |
plesty/website/website-content#3 (closed) | Changed after review: the drop-down navigation stays, with Hub and Docs beside the logo. New: a News & Updates page with the latest bench release and the monthly reports; the blog moves to the community site later. The Quickstart starts from the Download page and runs on one PC first, then on two. Done: plesty/website/website-content!15 (merged) merged, with a generator MR on the ops instance that fixes the Download page's dead link and its stale release list; closed 2026-10-26. Live once the plesty.net servers are back | |
docs-language-audit |
Documentation & AI Language Audit: Audit and simplify documentation, website copy, and AI-generated text to ensure concise, human-friendly language (e.g., replacing terms like "cross-bench" or "Runs card"). Fix cross-site inconsistencies, dead links, and point module links directly to docs instead of raw GitLab repos. | plesty/website/website-content#4 (closed) | Taking: dead links, cross-site inconsistencies, and module links to docs. Changed: plain language is checked on every docs change from now on, instead of a one-off audit that goes stale. Done: plesty/website/website-content!13 (merged) and !16 merged, closed 2026-10-26. Module links go to docs.plesty.net, writing rules in the README, no dead internal links in a local build; the experiment tutorial builds a simpler pow-pl from the instruments' demo/mock-replay branches, walked end to end. Unchecked while the plesty.net hosts are down: links to them. Follow-ups: plesty/core/plesty-lib#69 (closed), plesty/hub/experiments/plesty-pol-pl#2 (closed) |
|
branding-and--onboarding |
Project Branding & Onboarding: Standardize the logo and visual identity across the website (remove the "plesty" text from its right side which doesn't look good now), docs, and GitLab. Add root READMEs and subgroup descriptions across all GitLab repos, and add clear UI guidance on how users can contribute modules back to the Hub. | plesty/website/website-content#5 (closed), governance#25, plesty/core/plesty-sdk!36 (merged), plesty/website/contributor-guide!1 (merged), plesty/website/reports!1 (merged) | Taking. Logo: the mark without the wordmark in module docs (plesty/core/plesty-sdk!36 (merged), merged to exp, ships in plesty-sdk 0.6.0) and on plesty.net, the Hub and the Manager (merged on the ops instance 2026-10-26; live once the plesty.net servers are back up). Descriptions set on 31 projects and 16 subgroups; 19 subgroups have an avatar of their own (the X in the group's colour with its letter; a vendor group shows the vendor's short name). Owner needed (governance#25, assigned to the PO): the descriptions of 7 groups, the avatars of plesty and 10 subgroups, and archiving two empty Thorlabs repositories. READMEs: contributor-guide and reports merged; the device modules' READMEs show on main with their first release. Open: the group page README (plesty/gitlab-profile) and the bench's Hub menu link to the new contribute page. How to contribute a module: plesty/website/website-content!14 (merged) merged (Learn ▸ Contribute a Module to the Hub), closed 2026-10-26 |
Loading
Loading
Loading
Loading