Training Environment
For policy training, we use NVIDIA Isaac Sim and Isaac Lab. The code containing Berkeley Humanoid Lite training environment can be installed as an extension to the Isaac Lab.
Some system packages are required by Isaac Lab. We shall install these first.
sudo apt install cmake build-essential
For managing the Python environment, we support using either uv or conda. uv is recommended for its faster speed, modern dependency resolution, and strict reproducibility.
Setting up environment with uv
uv is an extremely fast Python package and project manager.
To set up uv, run this command:
wget -qO- https://astral.sh/uv/install.sh | sh
Then, simply run this command to install everything
uv sync
Finally, to activate the environment, do
source ./.venv/bin/activate
Setting up environment with conda
Alternatively, conda can be used to manage Python packages.
First, create a conda environment to contain all the Python packages.
conda create -yn berkeley-humanoid-lite python=3.10
conda activate berkeley-humanoid-lite
Installing Isaac Sim and Isaac Lab
Please refer to Issac Lab Documentation for more detailed guidance on installing Isaac Sim and Isaac Lab.
Here, we provide the procedure we followed to install on our machines. We have verified this flow on a fresh install of Ubuntu 24.04.
Use this command to install Isaac Sim 4.5.0 and Isaac Lab 2.1.0.
pip install torch==2.5.1 torchvision==0.20.1 --index-url https://download.pytorch.org/whl/cu121
pip install isaacsim[all,extscache]==4.5.0 --extra-index-url https://pypi.nvidia.com
pip install isaaclab[isaacsim,all]==2.1.0 --extra-index-url https://pypi.nvidia.com
Setting up Berkeley-Humanoid-Lite extension
After installing the Isaac Lab environment, we can proceed to install the Berkeley Humanoid Lite Extension.
We start by cloning the project repository.
git clone https://github.com/HybridRobotics/Berkeley-Humanoid-Lite.git
The project repository contains submodules for robot description and low-level code. The submodules can be initialize with this command.
cd Berkeley-Humanoid-Lite
git submodule update --init
Then, install the modules to IsaacLab environment.
pip install -e ./source/berkeley_humanoid_lite/
pip install -e ./source/berkeley_humanoid_lite_assets/
pip install -e ./source/berkeley_humanoid_lite_lowlevel/
There are some additional packages required by our project. Run the following script to install them.
pip install -r requirements.txt
Training the policy
Two tasks are defined in the codebase. Velocity-Berkeley-Humanoid-Lite-v0
trains a policy that actuates all 22 degrees of freedom on the robot, and Velocity-Berkeley-Humanoid-Lite-Biped-v0
trains a policy that only controls the 12 degrees of freedom on the leg.
To train the policy, run this following command. By default, we train for 6000 iterations, which should take around two hours.
uv run ./scripts/rsl_rl/train.py --task Velocity-Berkeley-Humanoid-Lite-v0 --headless
To view the trained result, run the following command.
uv run ./scripts/rsl_rl/play.py --task Velocity-Berkeley-Humanoid-Lite-v0 --num_envs 16
In addition to get a visualization of the policy, the play.py
script will also export the trained policy into an ONNX file for deployment. It will also create or update the configs/policy_latest.yaml
configuration file, which records the parameters that will be used during sim2sim and sim2real deployment.
We will see how to use this exported model checkpoint and configuration file in the next section.
Last updated