Robot learning
Reinforcement and imitation learning, behavior cloning, action chunking, multimodal demonstrations, and policy evaluation.
Robotics · Machine Learning
Himanshu
M.Sc. candidate in Robotic Systems Engineering at RWTH Aachen University.
I work on robot learning, manipulation, and autonomous systems. My current research focuses on learning force aware policies for contact rich assembly, with an emphasis on controlled evaluation, robustness, and transfer from simulation to physical systems.
At ZEISS Corporate Research & Technology, I have developed reinforcement learning policies for robotic manipulation and deployment workflows for Franka hardware. Previously, I worked on sensor fusion and localization at RWTH Aachen and on aerial robotics at IIIT Hyderabad.
News
Completed master’s thesis research on force aware reinforcement learning for contact rich assembly at ZEISS Corporate Research & Technology.
Won the Reply × BEST Aachen “AI Agents in Action” coding challenge.
Completed a robot learning internship at ZEISS, working on PPO, domain randomization, and transferring policies from simulation to a physical robot.
Achieved the best score in RWTH Aachen’s reinforcement learning car racing competition.
Research interests
I am interested in learning systems that remain reliable when they leave simulation and encounter uncertain physical environments.
Manipulation and simulation transfer
Contact rich assembly, force feedback, domain randomization, Cartesian impedance control, and deployment on physical robots.
State estimation
Sensor fusion, Kalman filtering, localization, uncertainty analysis, and systematic experimental evaluation.
Experience
May to Oct 2026
Master’s Thesis Researcher, Robot Learning
Carl Zeiss AG · Corporate Research & Technology
Developed force aware reinforcement learning policies for contact rich assembly in Isaac Lab. Designed controlled robustness experiments and evaluated how the robot configuration affects force and torque bias on Franka hardware.
View assembly experiments →Sep 2025 to Mar 2026
Robot Learning Intern
Carl Zeiss AG · Corporate Research & Technology
Trained PPO manipulation policies with PyTorch, Isaac Gym, and Isaac Lab, then built a deployment workflow with ROS and FrankaPy that included runtime logging and safety checks.
View simulation transfer results →Aug 2024 to Aug 2025
Student Assistant, Sensor Fusion and Localization
Institute of Automatic Control · RWTH Aachen University
Evaluated Kalman and unscented Kalman filters through reproducible localization experiments in Python and MATLAB.
Jun 2022 to May 2023
Project Assistant, Autonomous Systems and Robotics
Robotics Research Center · IIIT Hyderabad
Developed TD3 quadrotor control policies, evaluated UWB localization in DJI flight tests, and contributed to research on waypoint navigation and collision avoidance for multiple UAVs.
Selected projects
Recent work in physical robot learning and imitation learning.
Learning policies for contact rich assembly
Reinforcement learning policies, controlled robustness experiments, and transfer from simulation to a Franka Panda.
Transformer Based Imitation Learning
A Behavior Transformer using visual encoders, action clustering, and temporal sequence modeling.
Reinforcement Learning Car Racing
A TD3 agent for track navigation and cone avoidance; best score in the course competition.
Selected publications
All publications →IEEE IECON 2023
AGVO: Adaptive Geometry Based Velocity Obstacle for Heterogenous UAVs Collision Avoidance in UTM
H. Kumar, D. Datta K., J. V. Pushpangathan, H. Kandath, and A. Dhabale.
DOI →IFAC PapersOnLine 2022 · Best Presentation Award
Waypoint Navigation of Quadrotor using Deep Reinforcement Learning
Himanshu, H. Kandath, and J. V. Pushpangathan.
DOI →Springer 2022 · Best Paper Award, ICICV 2021
Path Planning of Mobile Robot Using Adaptive Particle Swarm Optimization
Himanshu, A. Singha, A. Kumar, and A. K. Ray.
DOI →Education
M.Sc. Robotic Systems Engineering
RWTH Aachen University · Germany
Thesis: Learning Force Aware Policies for Contact Rich Assembly Tasks Using Reinforcement Learning.
B.Tech. Electrical and Electronics Engineering
National Institute of Technology Sikkim · India
Thesis: Visual Scene Understanding Using Deep Learning.
Methods & tools
Learning: PyTorch, TensorFlow, PPO, TD3, SAC, behavior cloning, and ACT style action chunking
Robotics: ROS/ROS2, Isaac Lab, Isaac Sim, Isaac Gym, MuJoCo, PyBullet, Webots, FrankaPy, LeRobot
Engineering: Python, MATLAB/Simulink, C++, Git, Linux, Docker, LaTeX