AI technology is becoming an underlying framework for many automation systems today, and one of the latest applications for employing AI is in vision inspection systems. In fact, the key component of a vision inspection system is the camera, and suppliers like Cognex, Keyence and SICK AG are now routinely building AI into their cameras.
One system integrator reports using Cognex and Keyence cameras frequently. Actemium USA (formerly known as Outbound Technologies Inc.) and a CSIA (Control System Integrators Association) certified member, offers a broad range of advanced services and solutions, leveraging AI to enhance quality decision making in food and beverage applications. According to TJ March, senior project engineer, AI-based inspection technologies that his firm has deployed include detecting foreign objects, identifying defects and anomalies, ensuring precise label identification and placement and sorting products efficiently.
March has employed cameras with assisted AI, including the following models: Cognex In-Sight 2800 for product sorting; In-Sight D900 for OCR and label imperfections; In-Sight 3800 for label identification, complex OCR for inkjet printing color ID and label positioning; VisionPro Vidi for defect detection from various angles; and Keyence’s IV3 for product sorting.
“Currently, we are working with a multinational food processing company to develop a high-speed inspection system that detects surface and geometry defects on packaging,” says Sean Dugan, mechanical engineer at Huffman Engineering, Inc., a CSIA member.
“Our past projects included OCR systems to inspect package and carton labeling, as well as systems to inspect package integrity in the pharmaceutical industry,” adds Dugan.
Fortress Technology builds X-ray, metal detection, checkweighing and vision systems. Often these systems are used standalone, but can be combined together with a common data reporting system, says Matthew Gidman, product manager. “It can be more complex to have all these individual systems in one area of a production line—mainly due to physical space and cost constraints. However, AI now makes it feasible to collect data and monitor processes across all four systems.”
Having all this data from these systems and using AI can provide valuable information on production lines. However, a little work is required before setting up any AI-based inspection systems, says Gidman. “Before an organization can begin augmenting data analysis of systems with AI, it must assess its data quality. Accessible, high-quality data is essential for AI effectiveness, as well as relevance of the data for the specific problems being solved.
“Given the volume of legacy data, merging this together without complications, intensive data management programs, costly setup and maintenance programs can be challenging. To remedy this, collected data and repositories must be set up with AI in mind from the outset. One of the most significant benefits of combination units is that the data center is integrated, rather than trying to tie multiple disparate database formats together.”
Vision systems inspect products and give robotics sight. Gray Solutions, a CSIA member, began with vision systems for grading vegetables and gradually expanded into inspecting for defects and color consistency, says Gregory Powers, vice president of digital transformation. “Given our extensive work with robotics, we have integrated vision systems with robots to perform quality inspections. Additionally, we use vision systems on high-speed lines to capture packaging defects, such as issues with cans and bottles, including proper labeling.”
The main promise of AI systems in machine vision applications is that they are easier to ‘train’ to do the job than it would be to define some combination of acceptance metrics that would be amenable to traditional vision algorithms.
Moving AI-Vision Applications from Industrial Parts to Food
“The main promise of AI systems in machine vision applications is that they are easier to ‘train’ to do the job than it would be to define some combination of acceptance metrics that would be amenable to traditional vision algorithms,” says Huffman’s Dugan. “Measuring a dimension on a machined part and comparing it to a given tolerance is straightforward, but assessing the texture and shape of a chocolate chip cookie, for example, so that it can be judged good/bad/OK is a challenge.”
Dugan notes that recent projects didn’t require AI algorithms because pass/fail criteria were relatively straightforward, but in the cases of surface inspection where the surfaces are complex or have randomized features, AI algorithms might offer an effective solution.
“Integrating vision systems with AI in the food industry is an effective solution for quality inspection,” says Aaron Burke, an engineer at Concept Systems, a CSIA member. The setup is similar to being in a machining environment. Providing a variety of images with clear pass or fail conditions and correct programming will result in a successful inspection system. Some examples currently utilizing AI are position checks, measurement data of key features, label applications, contamination checks and defect detection. Keyence and Cognex products have been utilized by Concept Systems in food quality applications, says Burke.
Training AI is Getting Easier
Due to the high variability in food products, deep learning models excel when trained with a comprehensive, representative sample set (i.e., a collection of data samples that accurately reflects the full range of variations and conditions present in the target application), says Actemium’s Marsh. “Given the complexity and variability of these applications, we recommend leveraging the expertise of a system integrator. A system integrator can effectively manage all the requirements and nuances involved in deploying these advanced AI systems, especially when upgrading or adding to existing, potentially outdated inspection lines.”
“With regard to training an AI vision system, either system integrators or end users can train AI vision systems, however, training these systems for consistence and accuracy can be tricky, says Caleb Feagans, a chemical engineer at Huffman Engineering. Understanding the necessary variations and defects required to train and program a robust vision inspection system can be challenging, which is where an experienced system integrator can be beneficial. (See the box, “Training Considerations for AI/Vision Systems.”)
Training Considerations for AI/Vision Systems
Training vision systems with built-in AI for food quality applications can present some challenges compared to a more structured environment, such as manufacturing machined parts. Here are some considerations for training AI vision systems:
- Variability: Food surfaces exhibit a wide range of variations of color, textures and sizes. Machined parts often have consistent shapes and surfaces while food items do not and can vary significantly, even in the same batch.
- Diverse training data: Training AI models for food quality inspections (or any AI training for that matter) requires a diverse dataset that captures variability in appearance and defects. This training set must cover different lighting conditions, angles and variations in order to ensure quality results.
- Integration challenges: Upgrading an existing inspection system to include AI capabilities often requires integration with the current production line and existing infrastructure. This might involve adapting software, examining compatibility with existing hardware and possibly retraining the workforce.
— Caleb Feagans, Chemical Engineer, Huffman Engineering, Inc.
Gray Solutions has long utilized vision systems for various food quality applications, says Powers. Initially, setup requires input from a solution integrator, but with new AI tools, these systems are now more user friendly and can be trained by operators. Gray Solutions began by using vision systems to grade products and expanded their use to meat processing, ensuring meat products are free from contaminants and meet quality standards for texture and appearance. As camera technology improves and becomes more affordable, the use cases continue to grow, including ensuring date codes, bar codes and general packaging applications.
It’s important to remember that machine vision is typically part of a larger inspection system that can include x-ray systems, says Fortress Technology’s Gidman. The goal is to strengthen quality control by capturing an image and processing it against a set quality control parameters. In inspection technology, vision is commonly deployed for food pack label verification, and sometimes for food surface defects. Working in tandem and using a common AI system, these technologies can be used to strengthen food safety:
- Vision can improve X-ray performance by providing contextual information about each pack, including the label placement and presence of legally required information, such as dates and allergens.
- Vision cameras can be used for pack separation to ensure products are correctly spaced out as they are fed into inspection apertures or onto sensitive weighing conveyors.
- Metal detectors can be combined with X-ray to ensure high performance across the full contaminant spectrum. This combination could also be applied for dual-density products, e.g., inspecting baked bread with a harder surface crust and softer, dough middle.
AI Cameras a Drop-In Replacement for Older Vision Inspection Cameras?
Like any other equipment purchase, the decision to upgrade or purchase new equipment depends on the age and design of the current system, says Gidman. “This decision should be based on a thorough risk and cost analysis conducted by the purchaser. This ensures that the chosen option aligns with the company’s operational needs and budget constraints.”
“AI camera systems are not necessarily drop-in replacements or simple add-ons,” says Actemium’s Marsh. Due to the hardware requirements of deep learning or AI, it is rare for an AI-based camera to be a direct drop-in replacement for an older camera. They use their own specialized software for configuring and programming the cameras, which often involves additional programming to ensure proper communication with PLC or SCADA/MES systems. This integration process requires a detailed understanding of both the AI systems and the existing industrial control architecture to ensure seamless operation and data flow.
“It is important to consider that the new cameras may need adjustments for distance and lighting, likely requiring a systems integrator,” says Gray’s Powers. “Additionally, we need to assess how the new AI can utilize the training data from the old system, as retraining the new system may be necessary. Some new AI software solutions are compatible with various cameras, allowing for seamless camera switching.”
Training the camera is only one part of implementing a new AI-based machine vision system, says Huffman’s Dugan. For example, many lines run different variations of the same product or different products altogether. In these cases, it is necessary to load the cameras with different programs or configuration/job files for the different products. Today, this is usually done over Ethernet with either a PLC or PC-based workstation sending control sequences to the cameras to load different programs based on user input. Most older machine vision systems use proprietary software to perform this task, so just dropping in a new camera is not enough.
“Also, it is necessary to understand how the old cameras sent results back to the system, for example to a reject station downstream. Coordinating communication back and forth between the camera system and the rest of the line requires understanding how it all works together,” adds Dugan.
Getting the Lighting Right
When transitioning from a manual inspection process to an automated system, the existing lighting setup is often inadequate for the precision needed in machine vision systems with assisted AI, says Actemium’s Marsh. “Therefore, it is crucial for the system integrator to assess and specify the appropriate lighting environment and camera hardware for optimal performance of the new system.”
“Lighting provides contrast and enhancement of part features for cameras to easily find patterns, features, edges or defects,” says Concept’s Burke. For best results, the end user should incorporate controlled external lighting. Some examples include a light bar, spotlight or a lighting attachment fixed to a camera. Lighting also provides consistent and repeatable conditions for the engineer to effectively program a vision solution. But, reliance on ambient lighting can result in unpredicted results if the ambient lighting changes from when the camera was programmed.
While camera vendors will always be a good source of lighting information, it’s still up to the original inspection system integrator to scope out the full requirements and oversee the setup, says Gidman. “Misuse or misunderstanding of equipment can frequently cause operational and performance issues. If this causes machine performance tests to fail, this may even compromise food safety audits.”
What Software is Required to Drop-In new AI-based Cameras?
“The level of software involvement required to integrate new AI-based cameras depends significantly on the complexity of the inspection tasks,” says Savannah Toombs, staff development manager at Actemium. While simpler applications may require less sophisticated AI systems and minimal programming, there is always some degree of configuration needed. These systems are not purely plug-and-play.
Even for basic applications, configuring the AI-based cameras involves setting up the software to recognize specific inspection criteria, adjusting parameters and ensuring accurate data communication with existing systems, adds Toombs. More complex applications necessitate detailed programming, including developing custom algorithms, configuring advanced settings and integrating with PLCs or SCADA/MES systems. Additionally, PLC programming tags will need to be updated based on the specific camera model and manufacturer to ensure seamless operation.
“AI-based cameras will always involve some type of programming,” says Concept’s Burke. The hardware that is selected and the intended purpose of the vision system will dictate the complexity of the AI setup. Simple AI cameras can be set up with minimal vision experience. General engineering experience is always preferred for simple vision systems. Other AI cameras involve significant time and experience to effectively integrate the AI solution.
Issues in Upgrading Older Equipment
Data disparity is one of the biggest issues that processors face in upgrading to AI-powered inspection equipment, says Fortress Technology’s Gidman. The ability to tether multiple front-end machines to back-end reporting software in real time is a key step toward AI-driven systems. Fortunately for food processors, machine builders like Fortress have done their homework. “By creating their own software, Fortress has attempted to support processors and help them avoid unexpected issues,” says Gidman.
AI-based systems have the potential to solve challenging problems in machine vision applications, says Dugan. However, it is important not to get swept away by all of the excitement surrounding “AI.” Dugan suggests a couple of issues to overcome are loss of insight and vendor lock-in.
“An AI-based system might at first generate the results the engineers or managers are looking for, but dependence on them necessarily means that an understanding of how or why the camera is rejecting/accepting a part is lost,” says Dugan. “Applying conventional inspection systems requires knowing exactly what you are looking for. If we depend on AI to judge good cookies from bad cookies, will we actually know why a good cookie looks like a good cookie? If this insight is lost, how will this affect the rest of production? Will we miss opportunities for improving production because we no longer have a concrete, measurable definition of what a good cookie looks like?”
The second issue, Dugan notes, “At some point the new AI-based machine vision system will become the old, obsolete, ‘AI’-based machine system. When the time comes to upgrade, will we be able to convert or translate the programs or configuration files to a new system from a different vendor? The ideas behind the conventional machine vision tools such as edge detection, OCR, histograms, etc. are common across vendors. It is possible to manually convert a program from one vendor’s cameras into a program for another vendor’s cameras because the discrete tools are understood. With an AI-based program, the question becomes: How is this possible?”
“If you are considering implementing a new AI solution for capturing images, it may be worthwhile to assess the cost of replacing the cameras,” says Powers. “You might be surprised at how much these new cameras have improved, similar to the advancements seen in smartphone cameras with each model upgrade.” But Powers notes that while some AI tools claim to be compatible with any camera system, this is not always the case.
For a discussion of more issue when upgrading to AI-based vision systems, see the sidebar, “AI-Vision System Upgrade Gotchas.”
One final thought, AI tools do require extra hardware. A good example: An AI based graphics sharpening tool I purchased two years ago requires 5 GB of disk space for its modeling engine where a plain vanilla sharpening plugin tool from years ago needs under 1 MB. There are similar needs for RAM and processing power: A high-end graphics card and/or CPU is needed to run the model versus a simple processor for the plugin. Is the AI sharpening tool worth it? A resounding “Yes!” Why? It sharpens the image while simultaneously decreasing noise and artifacts. Old tools just worked on sharpening alone.
AI-Vision System Upgrade Gotchas
When replacing older vision/camera equipment with new AI-based systems, several “gotchas” can arise, along with important considerations for a successful upgrade:
- Speed and performance issues: AI programs often require more computational power and memory compared to traditional machine vision algorithms. This can result in slower performance and larger program sizes, which might necessitate upgrading to high-end, PC-based deep learning systems to maintain the desired speed and efficiency.
- Integration complexity: Integrating AI-based systems with existing infrastructure can be challenging. These systems often require different software and communication protocols, which means reprogramming and reconfiguring existing PLCs or SCADA/MES systems.
- Data handling and storage: AI systems generate and process large volumes of data, which can strain existing data handling and storage solutions. Ensuring adequate infrastructure for data management is crucial.
- Training and fine-tuning: Unlike traditional systems, AI-based systems require extensive training with representative sample sets to achieve accuracy. This process can be tedious and time-consuming, involving continuous refinement and adaptation to new data and conditions.
- Environmental and operational adjustments: AI systems may necessitate adjustments in the operational environment when tasked with inspecting a broader range of complexities. This could include optimizing lighting and configuring specific lenses to ensure effective performance. As a result, revisiting and adjusting the physical setup is often required to accommodate these needs.
- Technical expertise: Deploying AI systems necessitates a higher level of technical expertise. Staff may need additional training to handle the new technology, or it may be necessary to bring in external specialists or system integrators for the deployment and maintenance.
- Switching to AI-based vision systems offers substantial benefits, including the ability to perform tasks that traditional machine vision systems cannot handle reliably. However, the transition involves navigating these complexities and investing in the necessary resources to fully realize the advantages of AI technology.
— TJ Marsh, Senior Project Engineer II, Actemium USA