Training Environment

For policy training, we use NVIDIA Isaac Sim and Isaac Lab. The code containing Berkeley Humanoid Lite training environment can be installed as an extension to the Isaac Lab.

Install Isaac Sim and Isaac Lab

Please refer to Issac Lab Documentation for more detailed guidance on installing Isaac Sim and Isaac Lab.

Here, we provide the procedure we followed to install on our machines. We have verified this flow on a fresh install of Ubuntu 24.04.

Some system packages are required by Isaac Lab. We install these first.

sudo apt install cmake build-essential

Then, create a conda environment to contain all the Python packages.

conda create -yn berkeley-humanoid-lite python=3.10
conda activate berkeley-humanoid-lite

Use this command to install Isaac Sim 4.5.0.

pip install isaacsim[all,extscache]==4.5.0 --extra-index-url https://pypi.nvidia.com

Clone Issac Lab and switch to the stable release.

git clone [e-mail]:isaac-sim/IsaacLab.git
cd IsaacLab
git checkout v2.0.2

Finally, install Isaac Lab and the training frameworks.

./isaaclab.sh -i

Setting up Berkeley-Humanoid-Lite extension

After installing the Isaac Lab environment, we can proceed to install the Berkeley Humanoid Lite Extension.

We start by cloning the project repository.

git clone [e-mail]:HybridRobotics/Berkeley-Humanoid-Lite.git

The project repository contains submodules for robot description and low-level code. The submodules can be initialize with this command.

cd Berkeley-Humanoid-Lite
git submodule update --init

Then, install the modules to IsaacLab environment. Don't forget to check that you are in the correct conda environment (berkeley-humanoid-lite)!

python -m pip install -e ./source/berkeley_humanoid_lite/
python -m pip install -e ./source/berkeley_humanoid_lite_assets/
python -m pip install -e ./source/berkeley_humanoid_lite_lowlevel/

There are some additional packages required by our project. Run the following script to install them.

pip install -r requirements.txt

Training the policy

Two tasks are defined in the codebase. Velocity-Berkeley-Humanoid-Lite-v0 trains a policy that actuates all 22 degrees of freedom on the robot, and Velocity-Berkeley-Humanoid-Lite-Biped-v0 trains a policy that only controls the 12 degrees of freedom on the leg.

To train the policy, run this following command. By default, we train for 6000 iterations, which should take around two hours.

python ./scripts/rsl_rl/train.py --task Velocity-Berkeley-Humanoid-Lite-v0 --headless

To view the trained result, run the following command.

python ./scripts/rsl_rl/play.py --task Velocity-Berkeley-Humanoid-Lite-v0 --num_envs 16

In addition to get a visualization of the policy, the play.py script will also export the trained policy into an ONNX file for deployment. It will also create or update the configs/policy_latest.yaml configuration file, which records the parameters that will be used during sim2sim and sim2real deployment.

We will see how to use this exported model checkpoint and configuration file in the next section.