Precision visual inspection underpins quality assurance across aerospace, semiconductor, and medical manufacturing, where undetected surface anomalies on high-value parts translate directly into scrap, rework, and field failures. Robotic visual inspection requires precise alignment between the end-effector and local surface geometry in the presence of perception noise and surface irregularities. In industrial settings a human operator is often kept in the loop via teleoperation or shared autonomy, introducing real-time adjustments that render purely offline motion planning inadequate.
We present a novel, real‐time, closed-loop robotic orientation control pipeline for precision visual inspection, representing the first‐of‐its‐kind application of perception‐ driven orientation regulation within a human‐in‐the‐loop inspection framework. Built on an admittance‐based architecture that unifies operator input and perception‐driven surface alignment, the approach models the end‐effector as a virtual sphere moving through a viscous medium. The resulting physically interpretable mass‐damper system generates synchronized, compliant motion from orientation error and operator commands. We validate the framework on a 6‐DOF UR5e manipulator with eye‐in‐hand depth sensing, demonstrating stable surface‐normal tracking and a final mean orientation error of 0.4°.
A new perception-driven, closed-loop orientation regulator that couples real-time surface-normal estimation with a virtual mass–damper outer-loop compliance framework, such that the end-effector continuously realigns to local surface geometry without requiring pre-computed inspection trajectories.
An admittance-based control formulation that maps orientation error, teleoperation inputs, and human-in-the-loop corrections into a single compliant motion stream. This enables operator commands and perception-driven corrections to coexist while respecting actuator torque and velocity constraints.
We design the controller as a modular outer-loop archi- tecture issuing task-space velocity commands through standard servo interfaces, such that the framework de- ploys on industrial position/velocity-controlled manipu- lators without modifying their internal control architecture.
We validate the framework experimentally on a UR5e platform with eye-in-hand depth sensing, demonstrating stable convergence, robustness to perception noise, and a final mean orientation error of 0.4° that matches prior infrared-based methods while additionally supporting real-time, human-in-the-loop operation.
System architecture. Perception-driven orientation regulation runs as a modular outer loop over the manufacturer's existing position/velocity controller, executed through the ROS 2 Servo node — no modification to the underlying servo architecture.
Perception-driven, human-compliant orientation control for inspection assistance: from depth-based normal estimation through the virtual mass–damper admittance layer to the velocity commands executed by the manipulator.
An eye-in-hand Intel RealSense D405 streams depth of the local surface. Point clouds are filtered with RANSAC plane fitting to reject sensor outliers, then PCA recovers the surface normal as the eigenvector of the smallest eigenvalue of the point covariance. This normal defines the desired end-effector orientation.
The end-effector is modeled as a virtual sphere of mass $m$ and radius $R$ moving through a viscous medium. Force and torque inputs are mapped to smooth velocity commands through this physically interpretable admittance model, letting perception corrections and operator commands coexist in a single compliant motion stream.
A PD law on the orientation error generates the virtual control torque, tuned for a critically damped ($\zeta = 1$) response. A torque-saturation bound derived from actuator limits caps control effort and constrains the achievable natural frequency, keeping motion within safe velocity limits.
System architecture. Perception-driven orientation regulation runs as a modular outer loop over the manufacturer's existing position/velocity controller, executed through the ROS 2 Servo node — no modification to the underlying servo architecture.
Closed-loop orientation dynamics. With PD gains chosen to match a standard second-order form, the regulated error obeys
$$I_A\,\ddot{\theta} + (b + k_2)\,\dot{\theta} + k_1\,\theta = 0, \qquad k_1 = I_A\,\omega_n^2,\quad k_2 = 2\zeta\omega_n I_A - b.$$
Here $I_A$ is the virtual inertia about the pivot and $b = 6\pi\mu R d^2$ is the viscous damping. Choosing $\zeta = 1$ yields a monotonic, overshoot-free convergence to the surface normal.
Validated on a UR5e with eye-in-hand depth sensing under two regimes. The 15° case stays entirely in the unsaturated regime; the 40° case (with a lowered saturation threshold of 20°) drives the controller into torque saturation before settling into nominal admittance behavior. Hardware closely tracks the MATLAB admittance simulation in both cases.
Orientation error converges monotonically; torque stays well within actuator limits.
Linear and angular velocities follow the admittance prediction.
Torque clips at the actuator limit during the initial phase, then releases as the error shrinks.
Velocity plateaus at the imposed $v_{\max}$ during saturation before tapering off.
Across both cases the controller repeatably reaches a final mean orientation error of 0.4° (10 samples at trial end).
Against a manual-teleoperation baseline (10 trials from a 30° offset), the controller converges in a repeatable ~6 s versus a human mean of 8 s ± 1.5 s. It matches the 0.4° final error of prior infrared-based iterative adaptive control — while adding real-time, force-based, human-in-the-loop compliance.
| Method | Sensing | Loop | Human-in-the-loop | Final mean error | Reorientation time |
|---|---|---|---|---|---|
| Manual teleoperation (baseline) | Operator vision | — | Full manual | — | 8 s ± 1.5 s |
| Nakhaeinia 2018 | Infrared / proximity | Iterative, position-based | No | 0.4° | — |
| This work | Eye-in-hand depth (D405) | Real-time closed loop | Yes — compliant | 0.4° | ~6 s (repeatable) |
The full ROS 2 implementation is open source: github.com/macs-lab/Inspection_Control.
@inproceedings{banerjee2026admittance,
title = {Admittance-Based Surface Alignment for Human-in-the-Loop Robotic Visual Inspection},
author = {Banerjee, Antara and Acton, Colin and Chen, Xu},
booktitle = {Under Review},
year = {2026},
note = {Under Review}
}