See, Spawn, Synchronize: A Digital Twin Pipeline for Smart Production Cells

Abstract

With Industry 5.0, human-robot collaboration has become the center of attention. This has introduced new challenges, where workspaces become highly dynamic, leading to safety concerns for robots and especially for humans. This makes it important to have a realistic and accurate digital representation of a production cell and its components in environments for movement planning and remote supervision.

This paper presents a digital twin of a smart production cell, synchronizing objects bidirectionally between a real and virtual workspace with minimal effort using only a single RGB-D camera. A fine-tuned YOLO-based detector identifies tools and items in the scene, estimates their spatial position, and spawns them in Unity relative to the robot via coordinate transformation. Experiments with different scanning velocities demonstrate a mean planar spawn deviation of 5.82 mm (standard deviation 2.24 mm) at 0.1 m/s at 0.7 m height, while maintaining a constant depth bias of -0.54 mm.

Once spawned, objects can be manipulated freely via drag-and-drop within the simulation. Upon confirmation, a motion planning module calculates trajectories to execute these changes physically. Across 95 trials and 1805 object placements, the system achieves 100% success within working bounds, successfully executing complex tasks such as repositioning objects and stacking them into pyramid structures. The presented system provides a framework to see, spawn, and synchronize industrial workspaces, enabling rapid setup and safe remote supervision of smart production cells in highly dynamic industrial environments.

Details