On May 30th, 2023, Brian Plancher, assistant professor of computer science, co-presented new research, titled “Just Round: Quantized Observation Spaces Enable Memory Efficient Learning of Dynamic Locomotion,” at the IEEE International Conference on Robotics and Automation (ICRA) in London. The paper explores the potential of observation space quantization to reduce the memory requirements of reinforcement learning. This will enable the next generation of field robots to adapt to their environments through learning at the edge. The researchers explored this topic through the use of four simulated robot locomotion tasks and two state-of-the-art deep reinforcement learning (DRL) algorithms, the on-policy Proximal Policy Optimization (PPO) and off-policy Soft Actor-Critic (SAC). The results indicate that observation space quantization reduces overall memory costs by as much as a factor of 4.2x without impacting a robot’s learning performance.
Deep reinforcement learning is one of today’s most powerful tools for synthesizing complex robotic behaviors, but training DRL models is incredibly compute and memory intensive. Professor Plancher’s results suggest that observing space quantization holds promise as a simple solution to help address this challenge.