Skip to content

fix(calibration): avoid concurrent motor bus reads during status polling - #132

Merged
nicolas-rabault merged 1 commit into
huggingface:mainfrom
Sjeongheon:fix/calibration-status-polling-race
Oct 9, 2026
Merged

nicolas-rabault merged 1 commit into
huggingface:mainfrom
Sjeongheon:fix/calibration-status-polling-race

Conversation

@Sjeongheon

Copy link
Copy Markdown
Contributor

Fixes #121

Problem

During range-of-motion recording, LeLab could access the same motor bus from two different threads at the same time.

The calibration worker continuously calls:

sync_read("Present_Position")

to record the minimum and maximum position of each joint.

At the same time, the calibration page polls /calibration-status every 200 ms. Before this change, get_status() also called sync_read("Present_Position").

This meant that the recording loop and the HTTP status request could both try to use the same serial bus at the same time.

On an SO-101, the unpatched code reproduced errors from both paths during calibration:

Failed to read positions: ... [TxRxResult] Incorrect status packet!
Error reading positions during recording: ... [TxRxResult] There is no status packet!

The errors were intermittent, but they occurred repeatedly during calibration, including when the motors were moving very little or not moving.

This matters for calibration because if the recording loop fails to read a position at the moment a joint reaches its physical minimum or maximum, that sample cannot be included in the recorded range.

Root cause

The calibration worker and get_status() were both reading Present_Position from the same motor bus.

The frontend status request runs separately from the calibration worker, so the two sync_read() calls could overlap.

This matches the failure described in #121, where valid servo replies were observed on the wire but the host-side packet handling failed when two threads used the same serial connection concurrently.

Fix

This change makes the calibration worker the only code path that reads motor positions during range recording.

The recording loop still reads Present_Position and updates _mins and _maxes exactly as before. After a successful read, it also stores the latest min, max, and current values in status.recorded_ranges.

get_status() no longer accesses the motor bus. It only returns the values already recorded by the calibration worker.

The cached range data is updated while holding _status_lock. get_status() also returns a copied status object while holding that lock, so the calibration worker cannot change the nested recorded_ranges dictionary while FastAPI is serializing the response.

If a motor read started while recording was active but finishes after calibration has already moved to another state, its result is not written back into the status cache.

The frontend API and response format are unchanged.

Calibration finalization

Before this change, the calibration status could still be "recording" while _complete_calibration() was writing calibration registers. A simultaneous /calibration-status request could therefore start another motor read while write_calibration() was using the same bus.

Because get_status() no longer performs motor I/O, that overlap is also removed.

This PR does not change write_calibration() itself. It does not add register read-back, retries for lost replies, or recovery from partially written calibration values. Those are separate motor-communication concerns in LeRobot rather than part of the LeLab status-polling race fixed here.

Related LeRobot work

This PR changes how LeLab uses the motor bus; it does not replace or modify the underlying motor communication implementation.

Those PRs work at different layers. This change only ensures that LeLab's calibration page does not create a second motor-bus reader through status polling.

Testing

Automated tests

tests/test_calibrate.py now checks that:

  • get_status() does not call the motor bus while calibration is recording;
  • returned status data is independent from later updates made by the recording worker;
  • status polling can return even while the recording worker is blocked waiting for a motor read;
  • a read that finishes after recording stops cannot publish stale range data;
  • status polling does not start a motor read while calibration registers are being written.

Local checks:

pytest tests/test_calibrate.py
20 passed

ruff check
passed

pre-commit run -a
passed

Hardware

Tested with SO-101 leader and follower arms.

With the unpatched version, the packet errors described above were repeatedly reproduced during calibration.

With this change, five consecutive calibration runs completed without observing the corresponding Incorrect status packet! or There is no status packet! warnings. The calibration page continued to update the current position and recorded minimum/maximum values normally.

I also tested repeated status requests while calibration was recording.

The browser was left open, so the normal UI polling of /calibration-status continued every 200 ms. At the same time, a separate script sent another 200 requests to the same endpoint at approximately the same interval.

All 200 additional requests returned successfully with:

status = recording

This intentionally created duplicate status polling: the normal UI requests and a second stream of requests from the test script were hitting the same endpoint at the same time.

The timing numbers from this script are not intended to measure the execution time of get_status() alone, because they also include the local HTTP request and response, FastAPI request handling, JSON serialization, and JSON parsing.

The useful result here is that additional status requests continued to return normally while calibration was recording, without causing the extra motor-bus reads and packet warnings seen with the previous implementation.

@nicolas-rabault nicolas-rabault left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfectly executed PR.
Thank you @Sjeongheon .

@nicolas-rabault
nicolas-rabault merged commit 9b5afae into huggingface:main Oct 9, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Calibration: concurrent sync_read from get_status() and recording loop causes "[TxRxResult] Incorrect status packet!"

2 participants