I build what a humanoid sees and how it moves — perception, RL policies, and the road from sim → sim → hardware.
Humanoid Perception & RL Engineer · Robot Vision · Locomotion Policies · Sim-to-Real · Industrial CV · Edge Inference
Open to roles in humanoid / robot perception, robot learning (RL, sim-to-real) and industrial computer vision.
graph LR
subgraph See["👁 Perception"]
A["Multi-camera RGB-D<br/>detection · segmentation<br/>pose · depth"] --> B["VLA / scene<br/>understanding"]
end
subgraph Move["🦿 Policy"]
C["Task & reward design<br/>curricula · randomisation"] --> D["PPO in MuJoCo Warp<br/>(mjlab · RSL-RL)"]
end
subgraph Ship["🔩 Sim → Sim → Hardware"]
E["Second-simulator<br/>validation"] --> F["ONNX export<br/>C++ runner · HAL"] --> G["Robot"]
end
B --> C
D --> E
B -. "INT8 / INT4 on edge" .-> G
| Perception | the vision stack a robot actually runs: multi-task models (detection, instance segmentation, pose, depth) quantised FP16 → INT4 for edge targets, plus vision-language-action models in the control loop |
| Policies | whole-body locomotion and motion tracking trained from scratch with PPO: reward design, gait mechanics, push/drag curricula, domain randomisation |
| Sim → sim → hardware | model verification against CAD, actuator limits and latency modelling, cross-simulator checks, ONNX → C++ deployment behind a hardware-abstraction layer |
I build the path a humanoid policy takes from a training simulator to the robot:
- The robot model is checked, not trusted — URDF → MJCF conversion with an automated verifier that compares the simulated model against the CAD before any training starts.
- Training — velocity-tracking and motion-tracking tasks with PPO in MuJoCo Warp / mjlab, with mass randomisation and datasheet torque limits.
- Then the reality gap, on purpose — an actuation-latency model, a second-simulator pass, and ONNX export into a C++ state-machine runner behind a hardware-abstraction layer.
- One measured lesson: a policy with 0 % falls in an idealised simulator fell 100 % of the time once a 45 ms actuation delay was modelled. Retrained with the delay in the loop, it went back to 0 %. That is the gap that breaks robots on day one, and it was caught before any hardware was at risk.
I also build the perception layer for multi-camera robots (detection, instance segmentation, pose, depth, quantised FP16 → INT4) and perception for a medical screening robot. Robot, customer and architecture specifics stay with my employer; everything below is work I own outright.
🦿 g1-robust-locomotion — whole-body RL for the Unitree G1 (23-DoF)
Trained from scratch, with no pretrained policy and no imitation data, in MuJoCo Warp with RSL-RL PPO on free-tier T4s:
- Running that is actually running — a speed-adaptive gait clock shortens the cycle from 0.6 s to 0.4 s with commanded speed and pushes stance below 0.5, opening a true flight phase. A fixed clock turns "running" into a fast walk.
- Take hits from any direction — velocity impulses mixed with sustained horizontal drags from random headings, so balance recovery is learned as a skill rather than memorised against one shove.
- Reverse locomotion is a trained command (
lin_vel_x ∈ [−1.2, 2.0]), on a three-stage command curriculum. - Anti-cheat reward shaping — standing-still and feet-air terms stop the classic marching-in-place reward farm.
- Built to survive session limits — every run is checkpointed and resumable across 9–12 h GPU session kills.
Base stacks I build on: unitree_rl_mjlab (fork) · τ0-VLA (fork, tracked for VLA work).
Humanoid_Perception_for_Edge — vision-language-action for a humanoid, on a CPU
- Tiangong 2 Pro in MuJoCo with a head-mounted RGB-D camera (real intrinsics, metric depth), and a 17-joint upper-body action contract resolved by joint name, so a model can't silently drive the wrong joint.
- A safety filter (±0.30 rad envelope plus rate limiting) between the model and the actuators, and SmolVLA in a 50 Hz control loop.
- Every number is measured on a laptop: SmolVLA-base, fp32 on an i5-10210U, takes p50 9.6 s / p95 11.5 s per 50-action chunk at 6.6 GB peak RSS. That gap to real time is exactly what TernaVLA attacks. This milestone is plumbing; the motion is not trained yet, and the README says so.
TernaVLA · ternary-vlm — 1.58-bit models for robots without a GPU
- TernaVLA benchmarks ternary VLA backbones on a laptop CPU (real layer shapes): the 0.5B tier runs 0.44 s per action chunk at 8 threads, which is real time, against 1.48 s for 1B and 2.42 s for 2B. bitnet.cpp runs the real 2B model 21 % faster than ONNX Runtime.
- ternary-vlm: a SigLIP2 vision encoder on BitNet b1.58 2B, with LoRA that is quantised jointly with the ternary weights and merges bit-exactly (tested) for llama.cpp export. Research in progress; paper in preparation.
EasyAnnotate — annotation with nothing to set up
pip install easyannotateBoxes, polygons, oriented boxes and keypoints, with SAM 2.1 and YOLO-World assist, a fast review loop for model predictions, dataset operations, and export to YOLO / COCO / VOC / LabelMe / CSV. Local-first, with AI tools sized to your machine. Measured: the editor opens a 20k-image project with the first image in 1.6 s and page loads of 5–7 ms; SAM takes 0.044 s per click on an RTX 2050 and 0.17 s on CPU. CI, tests, desktop releases.
data-doctor — diagnose vision datasets before they lie to you
pip install vision-data-doctorA delivered production train/val split contained 81 groups of byte-identical images spanning both sides, so 40 % of validation was leaked. The model's 0.75 recall did not survive an honest split, and nothing in the training stack warned. data-doctor makes that a 10-second CI check:
- Hashes nominate; only pixels convict. SHA-256 catches exact copies, and two perceptual hash families across all 8 dihedral orientations nominate near-duplicates. Every candidate is then verified on decoded pixels, so a reported pair reads
99.4% similar, rot90, never a hash coincidence. - It also checks for train/val leakage, COCO structural defects and corrupt files. It exits 1 on failure, and Pillow is its only dependency.
Hobby project: conveyor-belt rock segmentation and sizing (Mask R-CNN, GigE camera, operator dashboard).
| Project | What it is |
|---|---|
| Synthetic_Data_Factory | Synthetic images for data-scarce vision (SD 1.5 + LoRA, from-scratch DDPM), screened for memorisation and tested with paired multi-seed A/B runs. Reported as measured: macro-F1 +0.005 ± 0.019, consistent with zero. The writeup says so instead of selling a headline. |
| KrishiDisha.ai | First-author IEEE ICoEIT 2025 platform: field-photo crop-disease detection, tool-grounded multilingual assistant, crop/fertiliser/yield models, live weather and market prices. Field-validated with 150+ farmers. |
| rsna-knee-abnormality-detection | 12 findings from knee MRI, with weak labels mined from multilingual radiology reports (regex 0.738 → LLM 0.816 → blend 0.846 macro AUC). |
| edgefront_JEV | Hosted model or small local model? Measures accuracy, latency, calibration and cost on your own task (ONNX Runtime, int8). |
| Robot learning | MuJoCo · MuJoCo Warp · mjlab · RSL-RL (PPO) · reward & curriculum design · domain randomisation · actuator/latency modelling · URDF/MJCF |
| Deployment | ONNX · C++ control runners · TensorRT · ONNX Runtime · OpenVINO · TFLite · bitnet.cpp / llama.cpp · FP16 / INT8 / INT4 |
| Edge targets | NVIDIA Jetson · Android robots · embedded boards · laptop-class CPUs · GPU servers |
| Perception | YOLO family · RF-DETR · D-FINE · Mask/Faster R-CNN · SAM 1/2/3 · pose · depth · RGB-D · VLA (SmolVLA, LeRobot) · classical CV & projective geometry |
| Core | Python · C++ · PyTorch · OpenCV · MediaPipe · FastAPI · GitHub Actions |
- KrishiDisha: Revolutionizing Agriculture with Intelligent Recommendations, Disease Detection, and Yield Prediction — first author · IEEE ICoEIT 2025, pp. 1028–1040 · doi:10.1109/ICoEIT63558.2025.11211713
- Web-BCD: A Machine and Deep Learning Approach for Breast Cancer Detection — IBM Technical Report, Dec 2024 · ROC-AUC 0.96, deployed for clinician use
- A Review on Understanding the Patterns of Student Dropout — first author · Studies in Smart Technologies, Springer, 2025 · doi:10.1007/978-981-97-9006-7_20
ML Engineer · Kody Technolab — humanoid perception & RL (Nov 2025 – present; intern Apr–Oct 2025) · DL Research Mentee · IBM India (2024) B.Tech CSE (AI/ML) · Medi-Caps University · CGPA 8.67
I read the papers properly, design for the compute the robot actually has, and don't ship a number that hasn't earned it.



