fix(loading errors): improving dataset loading errors handling and logging

docs/source/_toctree.yml

@@ -134,7 +134,7 @@
   - local: notebooks
     title: Notebooks
   - local: feetech
-    title: Feetech Troubleshooting and Firmware Update
+    title: Updating Feetech Firmware
   - local: damiao
     title: Damiao Motors and CAN Bus
   title: "Resources"

docs/source/feetech.mdx

@@ -1,60 +1,27 @@
-# Feetech Troubleshooting and Motor Firmware Update
-
-## Troubleshooting
-
-### Position Overflow
-
-If during calibration you encounter an error like this:
-
-```bash
-ValueError: Magnitude 2816 exceeds 2047 (max for sign_bit_index=11)
-```
-
-Or
-
-```bash
-RuntimeError: Some motors have invalid position readings {'wrist_roll': 6015}, which can lead to incorrect homing offsets.
-```
-
-The firmware may be overflowing and returning incorrect position readings (usually they should sit within [0, 4095]).
-
-**Quick fix:** Try to disconnect the robot's AC power and USB cable, move it to the middle of its range of motion, then reconnect and rerun the calibration script. This should give you correct position readings again.
-
-If the issue persists, you can try to reset the positions of the motors:
-
-1. Complete the first 4 steps of the motor firmware update process
-2. Select the _Programming_ tab
-3. Move all joints to the middle of their range
-4. Click _Offset_
-   <img
-     src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/lerobot/feetech-reset-offset.png"
-     alt="Feetech Offset Position"
-   />
-
-## Feetech Motor Firmware Update
+# Feetech Motor Firmware Update
 
 This tutorial guides you through updating the firmware of Feetech motors using the official Feetech software.
 
-### Prerequisites
+## Prerequisites
 
 - Windows computer (Feetech software is only available for Windows)
 - Feetech motor control board
 - USB cable to connect the control board to your computer
 - Feetech motors connected to the control board
 
-### Step 1: Download Feetech Software
+## Step 1: Download Feetech Software
 
 1. Visit the official Feetech software download page: [https://www.feetechrc.com/software.html](https://www.feetechrc.com/software.html)
 2. Download the latest version of the Feetech debugging software (FD)
 3. Install the software on your Windows computer
 
-### Step 2: Hardware Setup
+## Step 2: Hardware Setup
 
 1. Connect your Feetech motors to the motor control board
 2. Connect the motor control board to your Windows computer via USB cable
 3. Ensure power is supplied to the motors
 
-### Step 3: Configure Connection
+## Step 3: Configure Connection
 
 1. Launch the Feetech debugging software
 2. Select the correct COM port from the port dropdown menu
@@ -62,13 +29,13 @@ This tutorial guides you through updating the firmware of Feetech motors using t
 3. Set the appropriate baud rate (typically 1000000 for most Feetech motors)
 4. Click "Open" to establish communication with the control board
 
-### Step 4: Scan for Motors
+## Step 4: Scan for Motors
 
 1. Once connected, click the "Search" button to detect all connected motors
 2. The software will automatically discover and list all motors on the bus
 3. Each motor will appear with its ID number
 
-### Step 5: Update Firmware
+## Step 5: Update Firmware
 
 For each motor you want to update:
 
@@ -79,12 +46,12 @@ For each motor you want to update:
 4. **Click on Upgrade button**:
    - The update progress will be displayed
 
-### Step 6: Verify Update
+## Step 6: Verify Update
 
 1. After the update completes, the software should automatically refresh the motor information
 2. Verify that the firmware version has been updated to the expected version
 
-### Important Notes
+## Important Notes
 
 ⚠️ **Warning**: Do not disconnect power or USB during firmware updates, it will potentially brick the motor.
 
@@ -94,7 +61,7 @@ For debugging purposes only, you can use the open-source Feetech Debug Tool:
 
 - **Repository**: [FT_SCServo_Debug_Qt](https://github.com/CarolinePascal/FT_SCServo_Debug_Qt/tree/fix/port-search-timer)
 
-#### Installation Instructions
+### Installation Instructions
 
 Follow the instructions in the repository to install the tool, for Ubuntu you can directly install it, for MacOS you need to build it from source.
 

src/lerobot/datasets/dataset_metadata.py

@@ -180,16 +180,6 @@ class LeRobotDatasetMetadata:
         self.episodes = load_episodes(self.root)
         self.stats = load_stats(self.root)
 
-    def ensure_readable(self) -> None:
-        """Guarantee metadata is fully loaded for read operations.
-
-        Idempotent — when metadata is already in memory this is a single
-        ``is None`` check.  Call this before transitioning from write to
-        read mode on the same instance.
-        """
-        if self.episodes is None:
-            self._load_metadata()
-
     def _pull_from_repo(
         self,
         allow_patterns: list[str] | str | None = None,

src/lerobot/datasets/dataset_reader.py

@@ -87,7 +87,7 @@ class DatasetReader:
         """Attempt to load from local cache. Returns True if data is sufficient."""
         try:
             self.hf_dataset = self._load_hf_dataset()
-        except (FileNotFoundError, NotADirectoryError):
+        except (FileNotFoundError, NotADirectoryError, ValueError):
             self.hf_dataset = None
             return False
         if not self._check_cached_episodes_sufficient():

src/lerobot/datasets/io_utils.py

@@ -78,7 +78,10 @@ def load_nested_dataset(
     with SuppressProgressBars():
         # We use .from_parquet() memory-mapped loading for efficiency
         filters = pa_ds.field("episode_index").isin(episodes) if episodes is not None else None
-        return Dataset.from_parquet([str(path) for path in paths], filters=filters, features=features)
+        try:
+            return Dataset.from_parquet([str(path) for path in paths], filters=filters, features=features)
+        except ValueError:
+            raise ValueError(f"Failed to load parquet files in {pq_dir}, make sure the dataset is valid and is not missing any files.")
 
 
 def get_parquet_num_frames(parquet_path: str | Path) -> int:

src/lerobot/datasets/lerobot_dataset.py

@@ -278,7 +278,6 @@ class LeRobotDataset(torch.utils.data.Dataset):
     def _ensure_reader(self) -> DatasetReader:
         """Lazily create the reader on first access."""
         if self.reader is None:
-            self.meta.ensure_readable()
             self.reader = DatasetReader(
                 meta=self.meta,
                 root=self.root,

src/lerobot/motors/motors_bus.py

@@ -777,16 +777,6 @@ class SerialMotorsBus(MotorsBusBase):
 
         self.reset_calibration(motor_names)
         actual_positions = self.sync_read("Present_Position", motor_names, normalize=False)
-
-        if any(pos < 0 or pos > 4095 for pos in actual_positions.values()):
-            invalid_positions = {m: p for m, p in actual_positions.items() if p < 0 or p > 4095}
-
-            raise RuntimeError(
-                f"Some motors have invalid position readings {invalid_positions}, which can lead to incorrect homing offsets.\n"
-                "Try to disconnect the robot's AC power and USB cable, move it to the middle of its range of motion, then reconnect.\n"
-                "If the problem persists, check the documentation: https://huggingface.co/docs/lerobot/feetech"
-            )
-
         homing_offsets = self._get_half_turn_homings(actual_positions)
         for motor, offset in homing_offsets.items():
             self.write("Homing_Offset", motor, offset)

tests/datasets/test_lerobot_dataset.py

@@ -535,31 +535,6 @@ def test_getitem_works_after_finalize(tmp_path):
     assert "task" in item
 
 
-def test_getitem_after_finalize_with_delta_timestamps(tmp_path):
-    """After finalize(), dataset[0] works when delta_timestamps require episode metadata.
-
-    Regression test for https://github.com/huggingface/lerobot/pull/3305.
-    The create -> write -> finalize -> read path left meta.episodes as None
-    because the write path flushes episodes to disk without updating them
-    in memory.  Features that access meta.episodes (video decoding,
-    delta_timestamps) would crash with a TypeError.
-    """
-    dataset = LeRobotDataset.create(
-        repo_id=DUMMY_REPO_ID, fps=DEFAULT_FPS, features=SIMPLE_FEATURES, root=tmp_path / "ds"
-    )
-    for _ in range(5):
-        dataset.add_frame(_make_frame())
-    dataset.save_episode()
-    dataset.finalize()
-
-    # Set delta_timestamps so get_item() accesses meta.episodes via _get_query_indices
-    dataset.delta_timestamps = {"state": [0.0]}
-
-    item = dataset[0]
-    assert "state" in item
-    assert "state_is_pad" in item
-
-
 # ── Property delegation ──────────────────────────────────────────────