Helping robots ‘see’ by using advanced AI and video in manufacturing

Not long ago, a world inhabited by robots was merely a sci-fi storyline.

Today, robots have become part and parcel of many manufacturing lines. We’ve become used to seeing them assemble, pick and pack, and zip down warehouses looking for the latest item for delivery.

They have become indispensable to many organizations and individuals, where many work hand-in-hand with human colleagues. Now, they are being given the ability to see and are using video technology to be even more efficient.

Rise of the cobot

Most of the robotic technology seen to date in manufacturing are termed “cobots”, or collaborative robots. These are collaborative robots that require human-to-robot interaction in order to work effectively.

They can (quite literally) pick up in areas where humans cannot, lifting and assembling heavy objects.

Simultaneously, they have helped to improve the safety and efficiency of assembly lines, with studies showing that human-cobot teams are 85 percent more productive than a human or robot operating alone.

Plus, their human colleagues report feeling more satisfied in their jobs when sharing their workload with a cobot partner. Indeed, 77 percent of employees would be happy to work alongside cobots if it meant fewer manual processes.

Greater potential

This potential can extend even further with advances in machine learning and artificial intelligence (AI) that are improving all robots’ capabilities. AI vision is a type of computer vision that enables robots (and cobots) to “understand” their surroundings and human co-workers in greater detail.

There are two types: 2D and 3D vision. With 3D vision, robots can understand an object in all three dimensions (its size, shape, and orientation) including depth.

Giving these robots the power of vision helps them see everything around them and everything they touch with a high degree of accuracy.

This will have huge ramifications for the industry. To start, the machine vision market is growing rapidly. It is projected to reach $2.4 billion (US) by 2025, from $1.5 billion in 2020. As the sector grows, so too will its applications for cobots.

More uses

Until this development, cobots have essentially acted as strong and efficient extra limbs for operators. Now, with machine vision, they are developing situational awareness to complement their mobility and dexterity.

They will be able to undertake more challenging activities, such as picking and packing fragile or difficult-to-hold items. Supported by video, cobots can orientate and perform different actions on the same object based on its positioning. They can also be programmed for quality control.

Moreover, AI vision will allow cobots to work in less structured environments. Because of the situational awareness created, cobots will be able to function effectively in agricultural and food processing plants where not everything on a conveyor belt may be the same size and structure.

Cobots will also be able to work with random parts placed on a belt or racking automobile components.

Improved safety

Cobots with AI vision will also be able to operate more safely. They can automatically detect when they are close to a human colleague and either slow down or stop completely based on set distance parameters.

This reduces the likelihood of a robot-related accident on the plant floor and can reassure human colleagues working alongside them. Hanwha Techwin has been driving progress in this area, working with the robotics arm of the Hanwha Corporation to augment its equipment with deep learning and AI vision.

For example, Hanwha’s HCR Advanced robots are configurable with a “Robot Monitoring Service”, which uses camera technology from Hanwha Techwin to keep a close eye over the immediate workspace, sounding warnings in the event of imminent danger.

Other video-based solutions for Hanwha’s robots include “Robot AI 3D Vision”, which adds a 3D camera so the robot can scan its surroundings and use deep learning technology to optimize its movements.

A separate “Robot Visual Safety” feature tracks the location of the human operators with whom the robot shares its space, to avoid accidents.

Should someone step into the robot’s “yellow zone”, it will automatically slow its movements to lower the risk of collision. And if a human steps into the cobot’s “red zone”, it will stop completely.

Optimising performance

At the same time, AI vision can use cameras placed across a manufacturing plant to supervise the performance of the cobots themselves. Again, this is an area where Hanwha Corporation is innovating by calling on the video technology expertise of Hanwha Techwin.

The AI system can proactively alert plant operators of potential issues, such as the risk of a cobot breaking down. If a cobot stops working, the AI system can detect this quickly and flag it to engineering for a quick response. I

t could also tell maintenance when to service a particular cobot based on its state when operating. Visual data from the plant floor could tell supervisors how well the cobots are performing and could inform future plant layouts.

Autonomous mobile robots moving around a plant floor can use visual data to learn the layout of a warehouse. They can navigate safely around obstacles in real-time and find the most efficient route through a building. This will save humans from doing time-consuming and laborious manual transport.

Constantly improving

Insights from cameras placed inside and alongside robots can be combined with deep learning to optimise their movements. A cobot can automatically assess how well it has assembled or packed an object based on the visual data being sent back to it.

It can then try different combinations or movements until it reaches the one that achieves the best results in the most efficient time.

Part of a digital evolution

It’s worth mentioning the wider scale of digitisation happening in manufacturing today. So-called Industry 4.0 relies on AI vision and deep learning, advances in the Internet of Things (IoT) Machine to Machine (M2M) communications, 5G, cloud technology and more.

These will ultimately all work together to make warehouses and production lines smarter. Creating safer and more productive relationships between robots and humans.

Major manufacturing issues could become a thing of the past, as proactive maintenance and intelligent shipping are made possible, while workplace safety is maintained using advanced video analytics from Hanwha Techwin.