Machines always have been inherently capable of auditory communication — particularly those with electrical and moving parts. The whirring of an otherwise silent laptop fan would signal a system working on full load, an unusual buzz in a car could denote a faulty engine. In factories, heavy machines groan all the time, except it’s tough to hear them amid the din.

A Bengaluru company has now developed artificial intelligence algorithms and analytics tools to monitor factory floor operations and diagnose the health of industrial machines merely by the sounds they make. The data points derived from the sounds help generate factory-level visualisations and diagnostics on the entire operations.

“We have developed deep learning-based AI models that use sound as the input and then categorise the sound into various instances or categories,” says Anand Deshpande, cofounder and chief executive of Asquared IoT. “Machines make sounds that carry and convey a lot of information, be it the status of a machine, the stage of a particular process, or whether a process is being done correctly.”

Asquared IoT was founded in 2017 by Deshpande and Aniruddha Pant — both PhD holders in mechanical engineering who previously worked in areas including  machine learning, mathematical modeling and data analytics — along with Kanchan Pant, who has been operating a automobile component manufacturing company that doubles as Asquared’s testing  facility.

Globally, a few other companies including Amsterdam-based OneWatt and Helsinki-based Noiseless Acoustics operate in a similar space. In India, according to Deshpande, Asquared is the only company offering an industrial sound-based diagnostics and analytics solution.

AI listening in on the machines

If you’ve been to a factory or a machining shop floor, you know it can get really loud and noisy. One of the challenges Asquared had to tackle was to be able to distinctly identify the various sound sources. “On the factory floor, there are so many sounds mixed together that it is difficult to identify one source. The background noises are many. (But) our system can filter out and identify the single source being monitored. We have tackled that problem and our trained machine learning algorithms can single out and identify the source of the sound,” says Deshpande.

Another challenge the team faced was that of the size of the data generated by the system. Most industrial data analytics systems usually process data remotely on the cloud. But sound-based data is heavy and require large data bandwidth, making it cumbersome and impractical to stream over the internet.

To circumvent this problem, Asquared developed an edge computing system made up of processing units picked off the shelf. Edge computing is a method of processing and analysing data near the edge of a network, or locally, where the data is generated, instead of transmitting the data to a centralised data-processing server for analysis. Asquared’s devices are installed in factories near the machines to be analysed. The company then runs its machine learning algorithms to process and analyse the sounds picked from the machines.

The algorithms developed by the company are then customised to run on specific computing platforms. “We don’t design the edge device but we buy them off the shelf based on our requirements. For instance, right now, we are using Intel boards but there are also other companies like Nvidia that develop these boards. We then do some optimisation on our algorithms to run on a particular processing platform before deploying our system,” says Deshpande.

What sounds can tell about a machine

Machines today come with an array of sensors that collects and shares various data points. But a lot many more processes and systems still do not have the capability to do this. A unique selling point of Asquared’s solution is that it can extract data points out of systems that previously did not generate any — such as old welding machines and lathes from the pre-sensor era.

Some of these include operations and machines found in companies that have manufacturing shop floors with processes such as cutting, welding and drilling.

“Our USP is that our solution helps derive data from machines and processes that previously did not give data points, and using our analytics platform we derive insights from these data points for our customers,” says Deshpande.

The device, called Equilips, listens to the sounds of machines and categorises them into events such as whether or not a machine or a process is working well. The system generates a log of this data by the second and the software uses the collected data points to calculate multiple parameters and generate operational information and analytics.

Although the primary focus is on such systems, the company’s solution will also function on machines already equipped with sensors that generate similar data points. In these cases, Asquared’s internet of things (IoT) solutions act as an additional data point source for analytics.

The insights that factories are provided with through the analytics platform include operational data points such as uptime, downtime, live status, process quality, and historical data on operations. On top of this, the platform also provides information on machine health, warning signs and predictive maintenance alerts for the various machines.

The company’s solutions, primarily designed for the manufacturing industry, and more specifically machining operations, are installed for varying use cases across India. Asquared declined to disclose the names of its clients, instead describing some of its factory installations.

In Japan, Asquared IoT has deployed its solution in a factory in the Kobe region for a company that handles metal cutting and sawing.

“The factory owner earlier had an idea about overall production but there was no data on how much each machine was producing or how the machines were functioning. For instance, there are multiple machines and it is possible that one machine was doing a lot more work and some other machines were underutilised,” says Deshpande. “Also, earlier, just to know how many machines were working and how many weren’t, they had to manually call and check. But now, they can view that on the dashboard and get a real-time snapshot of what is happening. They can also figure out inefficiencies and operational details.”

Another of Asquared’s deployment is in a Chennai factory for monitoring its ultrasonic welding process for joining plastic parts. “It is a fairly complex process to monitor. There will be a lot of data in the ultrasound spectrum than the sonic spectrum, and the processes take only a few seconds to complete. Our system is being deployed to analyse the quality of the ultrasonic welding process,” says Deshpande.

For now, the obvious limitation that the system has is that it will only work on processes that give a sound signature. Machines that do not generate unique sound signatures relating to their process also pose a challenge for obvious reasons.

Globally, sound analytics has found some other interesting applications as well. It is deployed by PepsiCo in the quality check process for its Frito-Lay’s chips, which involves hitting the chips with lasers and then listening to the sounds to determine the chips’ texture. Anheuser-Busch InBev SA, the makers of Budweiser beer, has deployed a sound-based machine learning system for predictive maintenance of their machines.

The challenge of noise and tech

Deshpande insists Asquared’s system is not a solution to everything but works on processes and machines that make varying sounds depending on the status and processes of the machines. “Thankfully for us, that covers most of the manufacturing machines and processes.”

Even so, industry awareness and adoption of new technologies such as sound-based analytics remain low in India, says Nihal Kashinath, founder and CEO of Applied Singularity, a platform for IoT and AI companies. “Today, a lot of the machining and fabrication industry is not aware of the availability of such technologies and systems and what these can do for them. Once the hurdle of (a lack of) awareness is taken care of then you can have more such instances (of such technology) being deployed in the industry.”

The changing soundscape and growing number of sensors in today’s factory floors are the other challenges Kashinath sees for sound-based systems.

“Today, the factory floor is changing constantly and this will lead to a dynamic and constantly changing soundscape. The success of such sound-based tech will depend on how well they are able to filter out the background noise from the sound that is required to be monitored or analysed,” he says. “Also, data points from sound could work better with other supporting sensor data to give better and more accurate data points. These machine learning models will get better as they spend more time on the factory floor training themselves.”

---