Sensors in Robots — Robotics Tutorials

Complete Sensor Ecosystems in Modern Robots

A real robot is not equipped with a single sensor — it deploys a carefully designed sensor ecosystem where multiple sensing modalities cooperate to provide the state information needed for safe, effective operation. The sensor suite depends strongly on the robot's application domain.

Industrial manipulator sensor suite (e.g., KUKA KR 10 R1100):

6× absolute multi-turn encoders (one per joint) — for position/velocity feedback to joint controllers
Motor current sensors on each drive — for torque estimation and overload detection
Temperature sensors on motors and drives — for thermal protection
Optional wrist F/T sensor (e.g., ATI Gamma) — for force-controlled assembly tasks
Optional vision system (2D/3D camera on workcell) — for part localization and inspection
Safety encoder (redundant second encoder per joint) — for functional safety monitoring (PL d/e, SIL 2/3)

Mobile robot sensor suite (e.g., ROS-based autonomous mobile robot):

2D LIDAR (360°) — primary obstacle detection and map-based localization
Wheel encoders (2×) — odometry for dead reckoning between LIDAR updates
IMU — attitude estimation, enhancing odometry on slopes and during dynamic maneuvers
Ultrasonic sensors (8×) — close-range proximity for low objects missed by LIDAR scan height
RGB-D camera — 3D environment perception, person detection
Bump sensors / cliff sensors — physical contact and drop detection
Battery monitor — state-of-charge estimation for return-to-charger planning

Humanoid robot sensor suite (e.g., Boston Dynamics Atlas, Agility Robotics Digit):

Joint encoders + joint torque sensors at every degree of freedom (28–32 joints)
High-rate IMU on pelvis/torso — central to whole-body balance estimation
Stereo camera pair + depth camera — manipulation and navigation
3D LIDAR — environment mapping and obstacle avoidance
Foot force/torque sensors — ground contact estimation for gait control
Distributed tactile sensing on hands (research systems) — manipulation feedback

Industrial Robot Sensor Suite Details

Sensor	Location	Purpose	Typical Specs
Absolute multi-turn encoder	Each joint axis (6 total)	Joint position and velocity feedback for servo loop	23-bit resolution, SSI or EnDat interface, 4 kHz update
Motor current sensor	Each servo drive board	Torque estimation, overload detection, commutation	±15 A range, 0.01 A resolution, 16 kHz sampling
Temperature sensor (NTC)	Motor winding, drive heatsink	Thermal protection and derating	−20 to +150°C, 1°C accuracy
Safety encoder (redundant)	Each joint (SIL 2 systems)	Independent position monitoring for safety stop	Dual-channel output, Cat 3 / PL d conformance
6-axis F/T sensor (optional)	Robot flange (wrist)	Force-controlled assembly, collision detection	±500 N / ±20 Nm, 1 kHz output, EtherCAT interface
2D machine vision camera	Workcell frame or robot-mounted	Part localization, presence detection, quality inspection	5 MP, 30 fps, GigE Vision, with calibrated lens
3D vision system (optional)	Workcell frame	Bin picking, 3D pose estimation of parts	0.1 mm point accuracy, 1–5 fps for scan acquisition

Sensor Fusion: Combining Imperfect Information

No single sensor is perfect — every sensor has noise, bias, limited range, and gaps in coverage. Sensor fusion combines measurements from multiple sensors to produce estimates that are more accurate, robust, and complete than any individual sensor could provide.

Complementary fusion: Each sensor covers the weaknesses of another. The IMU provides high-rate (200 Hz+) motion estimates, but drifts. The LIDAR provides accurate absolute position, but at only 10 Hz. A complementary filter fuses them: the IMU fills in the gaps between LIDAR scans with high-rate pose propagation, while LIDAR corrects accumulated IMU drift.

Competitive fusion (redundant sensors): Multiple sensors of the same type provide independent measurements of the same quantity. Their outputs are averaged (or weighted by inverse variance) to reduce noise. If one sensor fails or gives an outlier, consistency checks can detect and exclude the faulty reading.

The Kalman Filter is the optimal linear estimator for Gaussian noise models and is ubiquitous in robot state estimation. It operates in two phases:

Prediction: Propagate the state estimate forward using the dynamic model
Update: Correct the prediction using new sensor measurements, weighted by relative uncertainties

For nonlinear systems (almost all robots), the Extended Kalman Filter (EKF) linearizes about the current estimate, and the Unscented Kalman Filter (UKF) uses sigma points for more accurate propagation through nonlinear functions.

Kalman Filter Equations

The discrete-time Kalman Filter optimal estimation cycle:

--- PREDICTION ---
x̂_k⁻ = A × x̂_{k-1} + B × u_k
P_k⁻ = A × P_{k-1} × Aᵀ + Q

--- UPDATE ---
K_k = P_k⁻ × Hᵀ × (H × P_k⁻ × Hᵀ + R)⁻¹
x̂_k = x̂_k⁻ + K_k × (z_k − H × x̂_k⁻)
P_k = (I − K_k × H) × P_k⁻

where:
  x̂_k = state estimate at time k
  A = state transition matrix (system dynamics)
  B = control input matrix
  u_k = control input
  P_k = state covariance matrix (estimate uncertainty)
  Q = process noise covariance (model uncertainty)
  K_k = Kalman gain (optimal weighting)
  H = observation matrix (maps state to measurement)
  z_k = actual measurement
  R = measurement noise covariance
  (z_k − H × x̂_k⁻) = innovation (measurement residual)

The Kalman gain K automatically balances trust between the prediction and measurement: when R is small (accurate sensor), K is large and the update trusts the measurement; when Q is small (accurate model), K is small and the prediction is trusted more.

Mobile Robot Sensor Suite — Turtlebot-class AMR

Sensor Type	Quantity	Purpose	Example Sensor / Topic
2D LIDAR	1	Obstacle detection, SLAM (GMapping/Cartographer), AMCL localization	SICK TiM571 / /scan
Wheel encoder (quadrature)	2	Differential-drive odometry for pose propagation	Built-in motor encoders / /odom
IMU (6-DoF MEMS)	1	Pitch/roll compensation for odometry on slopes, heading reference	Bosch BMI088 / /imu/data
RGB-D camera	1	Person detection, 3D obstacle avoidance, object manipulation	Intel RealSense D435i / /camera/depth/image_rect_raw
Ultrasonic sensors	4–8	Close-range proximity (undercarriage objects below LIDAR height)	HC-SR04 / /sonar/range
Cliff sensors (IR)	4	Detect staircase edges to prevent falls	Sharp GP2Y0A21 / /cliff_detected
Battery monitor	1	State-of-charge, voltage monitoring for return-to-dock planning	INA226 shunt monitor / /battery_state
WiFi / UWB	1	Communication and optionally UWB-based indoor positioning	UWB anchor system / /pose_uwb

ROS Sensor Topics and the Sensor Abstraction Layer

ROS (Robot Operating System) provides a standardized middleware for publishing and subscribing to sensor data across distributed robot software. Each sensor is wrapped in a driver node that publishes data on typed topics:

/scan (sensor_msgs/LaserScan) — 2D LIDAR range data; consumed by SLAM and navigation
/imu/data (sensor_msgs/Imu) — IMU linear acceleration, angular velocity, orientation
/joint_states (sensor_msgs/JointState) — joint name, position, velocity, and effort for all joints
/camera/image_raw (sensor_msgs/Image) — raw camera frames; processed by vision pipelines
/camera/depth/image_rect_raw (sensor_msgs/Image with 32FC1 encoding) — depth frames
/odom (nav_msgs/Odometry) — computed odometry pose and twist with covariance
/tf (tf2_msgs/TFMessage) — coordinate frame transformations between all robot links

The ROS tf tree maintains the spatial relationship between all coordinate frames in the system (base_link, camera_link, lidar_link, map, odom, etc.), allowing any node to transform data from any frame to any other using a consistent API. This abstraction is central to sensor fusion in ROS-based robots.

Robot_localization and SLAM toolbox are the standard ROS 2 packages for EKF-based sensor fusion and simultaneous localization and mapping respectively.

Sensor Calibration: Intrinsic and Extrinsic

Sensor calibration is the process of determining the mathematical relationship between raw sensor output and the physical quantity being measured. Without proper calibration, sensor data is unreliable regardless of sensor quality.

Intrinsic calibration characterizes the internal parameters of a sensor:

Camera intrinsic calibration: Determines focal length (f_x, f_y), principal point (c_x, c_y), and lens distortion coefficients (k_1, k_2, p_1, p_2) using a checkerboard target (Zhang's method). Implemented in OpenCV, MATLAB Camera Calibrator, and ROS camera_calibration.
IMU intrinsic calibration: Determines gyroscope and accelerometer scale factors, misalignment angles, and bias using a multi-position or multi-rate tumbling procedure (Allan Variance analysis for noise characterization).
LIDAR intrinsic calibration: Determines the angular offset and range bias of each laser channel using a reference target.

Extrinsic calibration determines the spatial transformation (rotation + translation) between two sensors or between a sensor and the robot body:

Camera-LIDAR extrinsic calibration: Finds the 6-DoF rigid body transform T_CL between camera and LIDAR frames, enabling 3D point clouds to be projected onto images for sensor fusion.
Hand-eye calibration: For robot-mounted cameras, finds the transform between the robot flange (hand) and the camera (eye), T_CE. Two methods: Eye-in-Hand (camera moves with robot) and Eye-to-Hand (camera is fixed in world). Solved using the AX=XB formulation (Tsai-Lenz method or Park-Martin method).

Industrial Robot vs. Collaborative Robot Sensor Design

Traditional Industrial Robot (e.g., KUKA KR, FANUC)

Sensors focus on position accuracy and repeatability
Absolute encoders at every joint, redundant for safety category
No intrinsic force sensing — environment is fixed, fenced
External safety devices: light curtains, area scanners, physical guards
F/T sensing only added for specific force-controlled process steps
Safety achieved through separation: no humans in robot workspace
Sensor data primarily used internally by robot controller
Typical sensor cost: $5,000 – $15,000 per robot

Collaborative Robot / Cobot (e.g., UR, Franka, KUKA iiwa)

Joint torque sensors in every joint — fundamental to the design
Continuous collision detection from torque monitoring
Power-and-force limiting (PFL) per ISO/TS 15066 — stops on unexpected contact
External sensors: safety-rated 2D LIDAR for zone monitoring
Integrated F/T sensor at wrist in some models
Sensor data shared with user application via open APIs (URScript, FCI, etc.)
Safety achieved through sensing and control — humans can share workspace
Higher sensor density increases base cost but enables flexible deployment

Key Principles of Robot Sensor System Design

01
Redundancy is critical for safety — critical measurements (joint position for industrial robots, contact force for cobots) should have independent backup sensors or channels.
02
Sensor placement is a design-time decision with lasting consequences: field-of-view, occlusion patterns, vibration exposure, and heat proximity all affect long-term performance.
03
Noise filtering must be matched to the control bandwidth — a low-pass filter improves noise rejection but adds phase lag; the cutoff frequency must be above the control loop bandwidth.
04
Real-time requirements: joint control loops run at 1–4 kHz; safety monitoring at 1 kHz+; SLAM at 10–25 Hz; high-level planning can tolerate 1–10 Hz. Each layer must use sensors at appropriate rates.
05
Extrinsic calibration between sensors (camera-to-LIDAR, camera-to-robot-flange) must be maintained; mechanical changes (gripper swap, camera remounting) invalidate previous calibrations.
06
SLAM (Simultaneous Localization and Mapping) requires tight integration of exteroceptive sensors (LIDAR/camera) with interoceptive odometry (encoders, IMU) — it is a canonical sensor fusion application.
07
ROS sensor topic conventions and the tf coordinate frame system provide a standardized architecture for multi-sensor robot systems, dramatically reducing integration effort.
08
Functional safety standards (ISO 13849, IEC 62061) impose specific requirements on sensor reliability and redundancy for robots operating near humans — understanding these is essential for industrial deployment.

Key terms

Sensor fusion: The combination of data from two or more sensors to produce a state estimate that is more accurate, robust, or complete than any single sensor alone.
Kalman filter: An optimal recursive Bayesian estimator for linear systems with Gaussian noise; combines a dynamic model prediction with sensor measurements, weighted by their relative uncertainties.
ROS (Robot Operating System): An open-source middleware framework providing standardized communication (topics, services, actions), coordinate frame management (tf), and tool ecosystem for robot software development.
Hand-eye calibration: The process of determining the rigid body transformation between a robot's end-effector (hand) and a mounted camera (eye), enabling the robot to use camera observations to guide manipulation.
Sensor redundancy: The use of multiple independent sensors measuring the same quantity to improve reliability, allow fault detection, and maintain operation if one sensor fails.
Noise: Random, unwanted variation in sensor output that does not represent the true value of the measured quantity; characterized by its statistical distribution (variance, power spectral density).
Calibration: The process of determining and correcting systematic differences between sensor readings and true values; includes intrinsic (internal parameters) and extrinsic (spatial relationship to other sensors/robot).
SLAM (Simultaneous Localization and Mapping): The computational problem of constructing or updating a map of an unknown environment while simultaneously tracking the robot's pose within that map, using sensors such as LIDAR and cameras.
Odometry: Estimation of a mobile robot's pose change from wheel encoder counts and kinematics; subject to cumulative drift but provides high-rate, low-latency pose estimates.
Extrinsic calibration: Calibration of the spatial transformation (position and orientation) between two sensors or between a sensor and the robot body frame.

← Sensing & Perception