Training Environment

For policy training, we use NVIDIA Isaac Sim and Isaac Lab. The code containing Berkeley Humanoid Lite training environment can be installed as an extension to the Isaac Lab.

Some system packages are required by Isaac Lab. We shall install these first.

sudo apt install cmake build-essential

For managing the Python environment, we support using either uv or conda. uv is recommended for its faster speed, modern dependency resolution, and strict reproducibility.

Setting up environment with uv

uv is an extremely fast Python package and project manager.

To set up uv, run this command:

wget -qO- https://astral.sh/uv/install.sh | sh

Then, simply run this command to install everything

uv sync

Finally, to activate the environment, do

source ./.venv/bin/activate

Setting up environment with conda

Alternatively, conda can be used to manage Python packages.

First, create a conda environment to contain all the Python packages.

conda create -yn berkeley-humanoid-lite python=3.10
conda activate berkeley-humanoid-lite

Installing Isaac Sim and Isaac Lab

Please refer to Issac Lab Documentation for more detailed guidance on installing Isaac Sim and Isaac Lab.

Here, we provide the procedure we followed to install on our machines. We have verified this flow on a fresh install of Ubuntu 24.04.

Use this command to install Isaac Sim 4.5.0 and Isaac Lab 2.1.0.

pip install torch==2.5.1 torchvision==0.20.1 --index-url https://download.pytorch.org/whl/cu121
pip install isaacsim[all,extscache]==4.5.0 --extra-index-url https://pypi.nvidia.com
pip install isaaclab[isaacsim,all]==2.1.0 --extra-index-url https://pypi.nvidia.com

Setting up Berkeley-Humanoid-Lite extension

After installing the Isaac Lab environment, we can proceed to install the Berkeley Humanoid Lite Extension.

We start by cloning the project repository.

git clone https://github.com/HybridRobotics/Berkeley-Humanoid-Lite.git

The project repository contains submodules for robot description and low-level code. The submodules can be initialize with this command.

cd Berkeley-Humanoid-Lite
git submodule update --init

Then, install the modules to IsaacLab environment.

pip install -e ./source/berkeley_humanoid_lite/
pip install -e ./source/berkeley_humanoid_lite_assets/
pip install -e ./source/berkeley_humanoid_lite_lowlevel/

There are some additional packages required by our project. Run the following script to install them.

pip install -r requirements.txt

Training the policy

Two tasks are defined in the codebase. Velocity-Berkeley-Humanoid-Lite-v0 trains a policy that actuates all 22 degrees of freedom on the robot, and Velocity-Berkeley-Humanoid-Lite-Biped-v0 trains a policy that only controls the 12 degrees of freedom on the leg.

To train the policy, run this following command. By default, we train for 6000 iterations, which should take around two hours.

uv

uv run ./scripts/rsl_rl/train.py --task Velocity-Berkeley-Humanoid-Lite-v0 --headless

conda

python ./scripts/rsl_rl/train.py --task Velocity-Berkeley-Humanoid-Lite-v0 --headless

To view the trained result, run the following command.

uv

uv run ./scripts/rsl_rl/play.py --task Velocity-Berkeley-Humanoid-Lite-v0 --num_envs 16

conda

python ./scripts/rsl_rl/play.py --task Velocity-Berkeley-Humanoid-Lite-v0 --num_envs 16

In addition to get a visualization of the policy, the play.py script will also export the trained policy into an ONNX file for deployment. It will also create or update the configs/policy_latest.yaml configuration file, which records the parameters that will be used during sim2sim and sim2real deployment.

We will see how to use this exported model checkpoint and configuration file in the next section.