Instance Segmentation :

• Worked on training instance segmentation models spanning multiple architectures such as SOLOV2, YOLOx and Yolact.

Synthetic Dataset Generation :

• Created synthetic dataset generation pipelines in Blender for training instance segmentation models. The pipeline uses pybullet to generate valid collision free scene configurations that are later imported by blender scripts which internally apply procedural materials and conditions for domain randomization.

Robotics Simulation/Development :

• Implemented core automation logic for the dishwashing robot .
• Created a simulation system for robotic manipulation tasks, with abstractions / interfaces to load detected objects, handling collision groups, and motion planning for pick and place operations.
• Built a tele-operation system to remotely control the robot with a joystick or keyboard over gRPC
• Created GUI that integrates with the remote application, that can display video streams, and
• Change the control flow of application, this was developed to provide human input while ML
• Models were under development. Later on this was repurposed to record sequences for future
• Reinforcement Learning needs.
• Created GUI tools to debug and inspect model inference by varying post processing parameters.
• Implemented analytical as well as ML methods for post processing noisy point cloud data to create parametric models of detected objects with 6D orientations and grasp poses.
• Created a library of utilities and tools to represent hierarchical transformation relationships, that greatly simplified representation and extrapolation of 6D grasp poses and controlling the robot to move in different frames of reference such as camera space, world space and joint space.
• Implemented methods for localizing camera, end effector of robots, and key objects in scene using a combination of aruco based pose estimation and ICP based registration from depth streams

Synthetic data Generation:

• Automated Synthetic Data Generation using NVidia omniverse ecosystem ( Isaac Sim and Replicator ).

Unreal Engine development C++ :

• Created a plugin to generate synthetic data, supporting annotations such as Instance Segmentation, Semantic Segmentation, normal and depth maps for rendered images.
• Implemented prompt to 3d world generation using LLMs.
• Created a plugin for Domain Randomization supporting procedural textures-geometry, randomized scene configurations with distribution control.
• Wrote Custom Shaders for Domain Randomization and Post Processing for visualization purposes.
• Built Unreal Editor Utilities to streamline setup for new data generation environments

• Was primarily involved in creating tools for processing SQLs, and automating parts of CICD pipeline.
• Developed jobs using Jenkins for Deployment with integrations to Gitea , and version control of blobs and artifacts.
• Created command line utilities to process SQLs and create Liquibase changelogs.

• Created interactive applications to teach physics concepts as a Freelancer using Unity and Blender.
• Designed and implemented interactive examples describing physics phenomena.
• Implemented ui to embed multiple interactive scenes within chapters.