ROSETTA Final Summary Report

ROSETTA develops human-centric technology for robots that will not only appear more human-like, but also cooperate with workers in ways that are safe and perceived as natural. Such robots will be programmed in an intuitive and efficient manner, making it easier to adopt them to new tasks when a production line is changed to manufacture a new product. The ROSETTA project is a four year research project started in March 2009. The research leading to these results has received funding from the European Community’s Seventh Framework Programme FP7/2007-2013 – Challenge 2 – Cognitive Systems, Interaction, Robotics – under grant agreement No 230902 - ROSETTA.

The project aims at developing industrially relevant technologies that simplify the use and integration of industrial robots into otherwise manual assembly lines. Today’s market place is characterized by products that come in many variants and have short lifetimes. This calls for flexible manufacturing systems that allow for frequent product changes. Industrial robot automation is the automation method of choice to meet with those demands, however, the application requires the ability to adapt even more quickly to new tasks. Further, it is desirable to integrate the robots with humans in the assembly lines in order to achieve the highest possible level of flexibility while utilizing the individual strengths of both human worker and robot. In the ROSETTA concept the robot is not necessarily separated from humans by any physical safety fences or barriers. This implies that the robot needs to be safe towards the human either by being intrinsically safe or by employing active safety systems.

The ROSETTA project ended during the spring of 2013 and the results show great progress towards the vision of having an industrial robot that is flexible, easy to instruct and can safely co-operate with human workers. Some of the significant achievements of the project include:

  • Task-level programming environment where the user can instruct the robot without textual robot programming. This programming environment is included in state-of-the-art simulation software which allows for offline verification of robot programs.
  • An overall system structure that, together with the programming approach, allows for robot program re-use through centralized knowledge repositories. This architecture also enables automatic optimization and machine learning by which robots can automatically increase their efficiency and robustness.
  • Development of a safety approach in which the robot can perceive humans in its workspace and can change its behaviour based on the danger potential of the human. For this to work, new sensor systems have been developed for workspace supervision and a framework for injury criteria has been developed. These activities have also provided input for the work on new international standards for human-robot safety in collaborative environments.

The project’s results have been disseminated at public events such as scientific conferences and journals, to the industry at the AUTOMATICA trade fair 2012 in Munich, and in joint industrial and academic forums such as the European Robotics Forum 2013 in Lyon.

The ROSETTA Approach
The envisioned resulting assembly system contains robots working next to human workers in an assembly line, in a mixed human-robot environment. The approach comprises three main areas of development; (1) safe human-robot collaboration, (2) simplified robot programming, and (3) advanced robot control techniques that enable automatic learning/optimizing of robot program execution.

The approach is embodied through a tool-chain that consists of;

  • An Engineering System by which a user can build a virtual workstation, instruct the robot on the task to accomplish, and simulate program execution.
  • A Knowledge Integration Framework that works as a knowledge repository where shared knowledge between robot installations and engineering systems can be stored and re-used.
  • A new controller architecture (called the ROSETTA Controller) that contains a generic task execution engine, interfaces towards different controller platforms, and a safety controller that can integrate new safety functions into the task execution.
  • An industrial robot controller that is a standard robot controller, specific to the type/model of robot required for the particular task.

The approach is based on several key-technologies; a new type of flexible concept robot that can be integrated close to humans, new task-level programming techniques, re-usable and platform-independent task descriptions, new types of safety sensors used to make the robot aware of the presence of humans in the workspace, and more efficient sensor integration in the robot controller.

Robot task programming

In the ROSETTA project a task-level programming approach has been implemented by which the user instructs the robot using instructions such as “Pick this workpiece” and “Insert Object A into Object B”, instead of traditional motion-level programming, in which the user teaches the robot explicit motions. The programming system is developed as part of the ROSETTA Engineering System. In the task-level approach the actual robot motions are then automatically generated based on the high-level task description. The Engineering System is connected to the Knowledge Integration Framework and can make use of services that are implemented on the server, for example a system for natural language processing. In the ROSETTA approach the user has been given two options as to how to instruct the robot;
  • via application-specific high level task descriptions such as assembly trees that describe an assembly operation using the CAD-structure of the final product, or
  • by using natural language to tell the robot what actions to perform.

Independently of which approach the user has to creating the program, a task-level program will be generated that is readable to humans without robot programming knowledge or experience. The task-level program is readable as a sequence of steps and each step can be visualized in the 3D simulation environment. The Engineering System can then automatically generate the robot program from the task-level instructions and deploy it to the robot controller. Both a virtual controller in the simulation environment and a real robot can execute the final program.

Assembly graph

Assembly tree defining a set of four assembly operations

Task execution
During task execution the ROSETTA controller will execute together with the industrial controller and provide functionality such as advanced sensor integration. These sensor-integrated robot skills allow the robot to perform advanced assembly operations as well as connect to the Knowledge Integration Framework, so that machine learning approaches can be used to feed real-world sensor data into the Knowledge Integration Framework and to automatically improve robot task execution.

Human-robot collaboration
The robot is expected to work next to human workers while carrying out its assembly task. During the ROSETTA project two sensor systems have been developed that will allow the robot to perceive humans in its working area;

  • A camera-based surveillance system that can track humans moving around in the scene, and estimate their intentions, so that the robot can adapt its behaviour based on the human worker’s action. A human-centric redundancy resolution scheme has been developed that makes the robot move in a way that is similar to how a human moves his arm. Once a human is in the vicinity of the robot, the robot will start to move using this human-centric motion scheme and also to slow down its motion.
  • An arm-mounted proximity sensor has also been developed that detects humans and other objects unknown to the robot. If a human is moving into the workspace of the robot, the robot arm will automatically withdraw its elbow or go into a protective stop, if required.

These new systems enable the robot to more naturally integrate into an otherwise human workspace. The ROSETTA control system differentiates between three distinct ways of human-robot collaboration:

  • Co-existence, when the human resides in or moves through the robot workspace but does not interact directly with the robot.
  • Co-operation, when the human will somehow interact with the robot in a productive way.
  • Interference, when the human will reach into the robot working area or otherwise obstruct the robot workplace. In this case, the robot must suspend production and then may resume once the human has left the working area.

Injury criteria
A framework for evaluation of injury criteria has been developed in order to evaluate the injury potential for the human standing next to a robot in an assembly line, both during setup of the station and during runtime. A methodology for deriving injury potential via FEM analysis has been developed and the robot used has been experimentally characterized in a test bench developed specifically for this purpose. The injury criteria models are based on simulations over a variable set of parameters; human body part (head, arm, thorax), type of impact (free, clamping), radius of the impactor, translational mass, velocity, force, type of impactor (padded, deformable, rigid), and the stopping time of the robot. Over 20.000 simulations have been done in the project in order to setup a comprehensive test case in the project.

We provide the injury risk tables from simulations in offline work. These are then used by an online lookup mechanism to determine current injury risk in a running application. Based on this online assessment, an appropriate reaction for the robot is selected from a number of prepared schemes.