Commit Graph

  • 3424644ecd Fix init temp Co-authored-by: s1lent4gnt <kmeftah.khalil@gmail.com> AdilZouitine 2025-04-16 14:43:47 +00:00
  • c37936f2c9 Update log_std_min type to float in PolicyConfig for consistency AdilZouitine 2025-04-15 14:02:24 +00:00
  • c5382a450c fix caching AdilZouitine 2025-04-15 13:16:22 +00:00
  • 2f7339b410 Handle caching AdilZouitine 2025-04-15 13:02:31 +00:00
  • 9e5f254db0 change the tanh distribution to match hil serl Co-authored-by: s1lent4gnt <kmeftah.khalil@gmail.com> AdilZouitine 2025-04-15 08:31:14 +00:00
  • 8122721f6d match target entropy hil serl AdilZouitine 2025-04-15 08:00:38 +00:00
  • 5c352ae558 stick to hil serl nn architecture AdilZouitine 2025-04-15 07:44:32 +00:00
  • 9386892f8e Refactor modeling_sac and parameter handling for clarity and reusability. AdilZouitine 2025-04-14 14:00:57 +00:00
  • 267a837a2c fix encoder training AdilZouitine 2025-04-11 11:50:46 +00:00
  • 28b595c651 [pre-commit.ci] auto fixes from pre-commit.com hooks pre-commit-ci[bot] 2025-04-09 15:05:17 +00:00
  • 9fd4c21d4d General fixes in code, removed delta action, fixed grasp penalty, added logic to put gripper reward in info Michel Aractingi 2025-04-09 17:04:43 +02:00
  • 02e1ed0bfb [pre-commit.ci] auto fixes from pre-commit.com hooks pre-commit-ci[bot] 2025-04-09 13:51:31 +00:00
  • e18274bc9a fix caching and dataset stats is optional AdilZouitine 2025-04-09 13:20:51 +00:00
  • 68c271ad25 Add rounding for safety AdilZouitine 2025-04-08 08:50:02 +00:00
  • a3ada81816 [pre-commit.ci] auto fixes from pre-commit.com hooks pre-commit-ci[bot] 2025-04-07 15:48:39 +00:00
  • 203315d378 fix sign issue AdilZouitine 2025-04-07 15:44:06 +00:00
  • 78c640b6d8 Refactor complementary_info handling in ReplayBuffer AdilZouitine 2025-04-07 14:48:42 +00:00
  • d5a87f67cf Handle gripper penalty AdilZouitine 2025-04-07 08:23:49 +00:00
  • 8bcf41761d fix caching AdilZouitine 2025-04-04 14:29:38 +00:00
  • 1efaf02df9 [pre-commit.ci] auto fixes from pre-commit.com hooks pre-commit-ci[bot] 2025-04-04 07:59:22 +00:00
  • cf58890bb0 fix indentation issue AdilZouitine 2025-04-03 16:05:29 +00:00
  • 7c2c67fc3c Enhance SAC configuration and replay buffer with asynchronous prefetching support AdilZouitine 2025-04-03 14:23:50 +00:00
  • 70130b9841 Enhance SACPolicy to support shared encoder and optimize action selection AdilZouitine 2025-04-03 07:44:46 +00:00
  • 6167886472 Enhance SACPolicy and learner server for improved grasp critic integration AdilZouitine 2025-04-02 15:50:39 +00:00
  • f9fb9d4594 Refactor SACPolicy for improved readability and action dimension handling AdilZouitine 2025-04-01 15:43:29 +00:00
  • d86d29fe21 Add mock gripper support and enhance SAC policy action handling AdilZouitine 2025-04-01 14:22:08 +00:00
  • f83d215e7a Refactor SAC policy and training loop to enhance discrete action support AdilZouitine 2025-04-01 11:42:28 +00:00
  • 7361a11a4d Refactor SAC configuration and policy to support discrete actions AdilZouitine 2025-04-01 09:30:32 +00:00
  • 0cce2fe0fa Added Gripper quantization wrapper and grasp penalty removed complementary info from buffer and learner server removed get_gripper_action function added gripper parameters to common/envs/configs.py Michel Aractingi 2025-04-01 11:08:15 +02:00
  • 88d26ae976 [pre-commit.ci] auto fixes from pre-commit.com hooks pre-commit-ci[bot] 2025-03-31 16:10:00 +00:00
  • 3a2308d86f Add grasp critic to the training loop s1lent4gnt 2025-03-31 18:06:21 +02:00
  • fdd04efdb7 Add get_gripper_action method to GamepadController s1lent4gnt 2025-03-31 17:40:00 +02:00
  • ff18be18ad Add gripper penalty wrapper s1lent4gnt 2025-03-31 17:38:16 +02:00
  • 427720426b Add complementary info in the replay buffer s1lent4gnt 2025-03-31 17:36:35 +02:00
  • 66693965c0 Add grasp critic s1lent4gnt 2025-03-31 17:35:59 +02:00
  • 334cf8143e [pre-commit.ci] auto fixes from pre-commit.com hooks pre-commit-ci[bot] 2025-03-31 13:59:31 +00:00
  • 5b49601072 Fix convergence of sac, multiple torch compile on the same model caused divergence AdilZouitine 2025-03-31 13:54:21 +00:00
  • 0185a0b6fd Fix cuda graph break AdilZouitine 2025-03-31 07:59:56 +00:00
  • 70d418935d Fix: Prevent Invalid next_state References When optimize_memory=True (#918) s1lent4gnt 2025-03-31 09:43:40 +02:00
  • eb44a06a9b [pre-commit.ci] auto fixes from pre-commit.com hooks pre-commit-ci[bot] 2025-03-28 17:20:38 +00:00
  • 8eb3c1510c Added support for controlling the gripper with the pygame interface of gamepad Minor modifications in gym_manipulator to quantize the gripper actions clamped the observations after F.resize in ConvertToLeRobotObservation wrapper due to a bug in F.resize, images were returned exceeding the maximum value of 1.0 Michel Aractingi 2025-03-28 18:12:03 +01:00
  • 4d5ecb082e Refactor SACPolicy for improved type annotations and readability AdilZouitine 2025-03-28 16:46:21 +00:00
  • 6e687e2910 Refactor SACPolicy and learner_server for improved clarity and functionality AdilZouitine 2025-03-28 16:40:45 +00:00
  • eb710647bf Refactor actor_server.py for improved structure and logging AdilZouitine 2025-03-28 14:17:16 +00:00
  • 176557d770 Refactor learner_server.py for improved structure and clarity AdilZouitine 2025-03-28 13:28:47 +00:00
  • 3beab33fac Refactor imports in modeling_sac.py for improved organization AdilZouitine 2025-03-28 13:26:47 +00:00
  • c0ba4b4954 Refactor SACConfig properties for improved readability AdilZouitine 2025-03-28 13:26:31 +00:00
  • 8fb373aeb2 fix AdilZouitine 2025-03-28 10:51:32 +00:00
  • 5a0ee06651 Enhance logging for actor and learner servers AdilZouitine 2025-03-28 10:43:03 +00:00
  • 05a237ce10 Added gripper control mechanism to gym_manipulator Moved HilSerl env config to configs/env/configs.py fixes in actor_server and modeling_sac and configuration_sac added the possibility of ignoring missing keys in env_cfg in get_features_from_env_config function Michel Aractingi 2025-03-28 08:21:36 +01:00
  • 88cc2b8fc8 Add WrapperConfig for environment wrappers and update SACConfig properties AdilZouitine 2025-03-27 17:07:06 +00:00
  • b69132c79d Change HILSerlRobotEnvConfig to inherit from EnvConfig Added support for hil_serl classifier to be trained with train.py run classifier training by python lerobot/scripts/train.py --policy.type=hilserl_classifier fixes in find_joint_limits, control_robot, end_effector_control_utils Michel Aractingi 2025-03-27 10:23:14 +01:00
  • db897a1619 [WIP] Update SAC configuration and environment settings AdilZouitine 2025-03-27 08:13:20 +00:00
  • 0b5b62c8fb Add wandb run id in config AdilZouitine 2025-03-27 08:11:56 +00:00
  • 056f79d358 [WIP] Non functional yet Add ManiSkill environment configuration and wrappers AdilZouitine 2025-03-26 08:15:05 +00:00
  • 114ec644d0 Change config logic in: - gym_manipulator - find_joint_limits - end_effector_utils Michel Aractingi 2025-03-25 14:24:46 +01:00
  • 26ee8b6ae5 Add .devcontainer to .gitignore for improved development environment management AdilZouitine 2025-03-25 08:21:07 +00:00
  • 38e8864284 Add task field to frame_dict in ReplayBuffer and simplify save_episode calls AdilZouitine 2025-03-24 20:28:14 +00:00
  • 80d566eb56 Handle new config with sac AdilZouitine 2025-03-24 20:19:28 +00:00
  • bb5a95889f Handle multi optimizers AdilZouitine 2025-03-24 15:34:30 +00:00
  • 0ea27704f6 [pre-commit.ci] auto fixes from pre-commit.com hooks pre-commit-ci[bot] 2025-03-24 13:41:27 +00:00
  • 2abbd60a0d Removed depleted files and scripts Michel Aractingi 2025-03-24 14:41:01 +01:00
  • 1c8daf11fd [pre-commit.ci] auto fixes from pre-commit.com hooks pre-commit-ci[bot] 2025-03-24 13:16:38 +00:00
  • cdcf346061 Update tensor device assignment in ReplayBuffer class AdilZouitine 2025-03-21 14:21:31 +00:00
  • 42f95e827d [pre-commit.ci] auto fixes from pre-commit.com hooks pre-commit-ci[bot] 2025-03-20 12:58:43 +00:00
  • 618ed00d45 Initialize log_alpha with the logarithm of temperature_init in SACPolicy AdilZouitine 2025-03-20 12:55:22 +00:00
  • 50d8db481e [pre-commit.ci] auto fixes from pre-commit.com hooks pre-commit-ci[bot] 2025-03-19 18:53:26 +00:00
  • e4a5971ffd Remove unused functions and imports from modeling_sac.py AdilZouitine 2025-03-19 18:53:01 +00:00
  • 36f9ccd851 Add intervention rate tracking in act_with_policy function AdilZouitine 2025-03-19 18:37:50 +00:00
  • 787aee0e60 - Updated the logging condition to use log_freq directly instead of accessing it through cfg.training.log_freq for improved readability and speed. AdilZouitine 2025-03-19 13:40:23 +00:00
  • 0341a38fdd [PORT HIL-SERL] Optimize training loop, extract config usage (#855) Eugene Mironov 2025-03-19 20:27:32 +07:00
  • ffbed4a141 Enhance training information logging in learner server AdilZouitine 2025-03-19 13:16:31 +00:00
  • 03fe0f054b Update configuration files for improved performance and flexibility AdilZouitine 2025-03-19 09:54:46 +00:00
  • fd74c194b6 [pre-commit.ci] auto fixes from pre-commit.com hooks pre-commit-ci[bot] 2025-03-18 14:57:57 +00:00
  • 0959694bab Refactor SACPolicy and learner server for improved replay buffer management AdilZouitine 2025-03-18 14:57:15 +00:00
  • 7b01e16439 Add end effector action space to hil-serl (#861) Michel Aractingi 2025-03-17 14:22:33 +01:00
  • 66816fd871 Enhance SAC configuration and policy with gradient clipping and temperature management AdilZouitine 2025-03-17 10:50:28 +00:00
  • 599326508f [pre-commit.ci] auto fixes from pre-commit.com hooks pre-commit-ci[bot] 2025-03-12 10:16:54 +00:00
  • 2f04d0d2b9 Add custom save and load methods for SAC policy AdilZouitine 2025-03-12 10:15:37 +00:00
  • e002c5ec56 Remove torch.no_grad decorator and optimize next action prediction in SAC policy AdilZouitine 2025-03-10 10:31:38 +00:00
  • 3dfb37e976 [Port HIL-SERL] Balanced sampler function speed up and refactor to align with train.py (#715) s1lent4gnt 2025-03-12 10:35:30 +01:00
  • b6a2200983 [HIL-SERL] Migrate threading to multiprocessing (#759) Eugene Mironov 2025-03-05 17:19:31 +07:00
  • 85fe8a3f4e [pre-commit.ci] auto fixes from pre-commit.com hooks pre-commit-ci[bot] 2025-03-04 13:38:47 +00:00
  • bb69cb3c8c Add storage device configuration for SAC policy and replay buffer AdilZouitine 2025-03-04 13:22:35 +00:00
  • ae51c19b3c Add memory optimization option to ReplayBuffer AdilZouitine 2025-02-25 19:04:58 +00:00
  • 9ea79f8a76 Add storage device parameter to replay buffer initialization AdilZouitine 2025-02-25 15:30:39 +00:00
  • 1d4ec50a58 Refactor ReplayBuffer with tensor-based storage and improved sampling efficiency AdilZouitine 2025-02-25 14:26:44 +00:00
  • 4c73891575 Update ManiSkill configuration and replay buffer to support truncation and dataset handling AdilZouitine 2025-02-24 16:53:37 +00:00
  • d3b84ecd6f Added caching function in the learner_server and modeling sac in order to limit the number of forward passes through the pretrained encoder when its frozen. Added tensordict dependencies Updated the version of torch and torchvision Michel Aractingi 2025-02-21 10:13:43 +00:00
  • e1d55c7a44 [Port HIL-SERL] Adjust Actor-Learner architecture & clean up dependency management for HIL-SERL (#722) Eugene Mironov 2025-02-21 16:29:00 +07:00
  • 85242cac67 Refactor SAC policy with performance optimizations and multi-camera support AdilZouitine 2025-02-20 17:14:27 +00:00
  • 0d88a5ee09 - Fixed big issue in the loading of the policy parameters sent by the learner to the actor -- pass only the actor to the update_policy_parameters and remove strict=False - Fixed big issue in the normalization of the actions in the forward function of the critic -- remove the torch.no_grad decorator in normalize.py in the normalization function - Fixed performance issue to boost the optimization frequency by setting the storage device to be the same as the device of learning. Michel Aractingi 2025-02-19 16:22:51 +00:00
  • 62e237bdee Re-enable parameter push thread in learner server AdilZouitine 2025-02-17 10:26:33 +00:00
  • c85f88fb62 Improve wandb logging and custom step tracking in logger AdilZouitine 2025-02-17 10:08:49 +00:00
  • a90f4872f2 Add maniskill support. Co-authored-by: Michel Aractingi <michel.aractingi@gmail.com> AdilZouitine 2025-02-14 19:53:29 +00:00
  • a16ea283f5 Fixed bug in the action scale of the intervention actions and offline dataset actions. (scale by inverse delta) Michel Aractingi 2025-02-14 15:17:16 +01:00
  • 8209a6dfb7 Modified crop_dataset_roi interface to automatically write the cropped parameters to a json file in the meta of the dataset Michel Aractingi 2025-02-14 12:32:45 +01:00
  • b5fbeb7401 Optimized the replay buffer from the memory side to store data on cpu instead of a gpu device and send the batches to the gpu. Michel Aractingi 2025-02-13 18:03:57 +01:00
  • 2ac25b02e2 nit Michel Aractingi 2025-02-13 17:12:57 +01:00
  • 39fe4b1301 removed uncomment in actor server Michel Aractingi 2025-02-13 16:53:33 +01:00