Pharma 4.0: powered by AI

Cognex’s vision technology is used widely throughout the pharmaceutical industry for pill sorting (Image: Cognex)

Benjamin Skuse discovers how cutting-edge vision solutions could make future pharma production more efficient, accurate and compliant with the latest regulations

Producing life-saving drugs in vast quantities, the pharmaceutical industry has long required safety and security standards above and beyond those of many other industries. In short, customer safety is paramount – at its most extreme, conforming to these standards can be the difference between life and death.

This is why pharma companies were some of the earliest adopters of machine vision solutions, though the first vision systems that came out in the 1980s were understandably primitive. For example, in 1981 lecturer Robert Shillman, and two graduate students Marilyn Matz and Bill Silver, spun out a company called Cognex from the Massachusetts Institute of Technology. The company’s first product was DataMan, the world’s only industrial optical character recognition (OCR) system at the time, which was first applied to inspect typewriter keys to ensure that they were located in the correct position.

“You can imagine, the first computers doing vision were racks taking up half a room, and the vision systems used to be $50-60,000, so inflation-adjusted super, super expensive,” says Joshua Deats, Cognex Global Packaging Manager. “But in terms of getting in with all the big pharmaceutical brands, we were one of the first who really pioneered OCR, and barcode reading as well.”

Cognex's cameras are also used in the pharmaceutical industry to inspect bottles and syringe seals (Image: Cognex, click here for larger view)

Fast forward to today and OCR – used in packing facilities to read and verify the correctness and legibility of important information on pill bottle labels, for example – is just the tip of the iceberg of what vision systems can do, not only helping companies comply with strict standards, but also delivering significant productivity gains.

Applications that have become run-of-the-mill include everything from PPE quality control inspection to vial and bottle packaging inspection, and counterfeit identification. Steve Zhu, Director of Sales for Asia at Teledyne Dalsa – a Canadian leading machine vision and imaging technology innovator – provides another common application. “When I joined the company almost 20 years ago, the first inspection application in the pharmaceutical industry was blister pack inspection for one of the largest pharmaceutical manufacturers in Asia,” he recalls. “Basically, we needed to detect missing capsules in the blister board, if there was an empty capsule, if there was a stain on the surface of the blister board, or if there was a hair or other foreign matter outside or inside.”

Teledyne Dalsa’s iNspect software being used to check for empty capsules and leaked powder in blister packs (Image: Teledyne Dalsa, click here for larger view)

At the time, this called for an analogue camera coupled to an infrared backlight in order to see inside the blister pack. Today, peering inside blister packs is done using one of Teledyne Dalsa’s X-ray cameras alongside Sherlock vision software.

With stiff competition in this space, machine vision solution providers must have a unique selling point. For Cognex, it is offering a popular high-performance product portfolio ideal for most applications that has seen the company become one of the world’s leading providers of vision systems, software, sensors, and industrial barcode readers. For Teledyne Dalsa, it is providing products that allow customers to zoom in on fine detail. And for German camera technology and image capture solutions specialist Allied Vision, it is providing the precise solution that fits customers’ needs, either through Allied Vision’s vast product range or via abespoke solution.

Thanh Luu, Junior Business Development Manager at Allied Vision, elaborates: “We are currently driving a lot for applications such as vial/ampoule inspection,” Luu details. “While the vials are being rotated, up to 20 cameras (depending on the machine) inspect them for cracks and scratches.” Luu says that for applications like this, as well as label inspection and code reading, the company’s Ethernet-based products such as the 1GigE and 5GigE Alvium cameras are popular. Offering standardised interfaces that are user-friendly and easy to install, these cameras boast high framerates, meaning fast inspection with high throughput. Allied Vision also provides useful firmware features like custom convolution filters or a colour transformation matrix. “These enable high image contrast for edge detection, which again is crucial for reliable inspection of pharma products,” adds Luu.

With these and many more vision systems embedded in pharmaceutical operations around the world, they are now providing an automated, reliable and objective means to inspect and analyse a wide range of pharmaceutical products, while also offering data analysis capabilities to expedite drug discovery, a relatively cheap and less error-prone way of optimising manufacturing processes, end-to-end visibility through the supply chain, and rapid verification of drug authenticity.

This has meant that tasks traditionally only entrusted to humans are being increasingly taken up by machine vision systems. A good example comes from Zhu: contact lens inspection, required to verify that there are no embedded particles, inclusions or contamination on the contact lens that could present a hazard to the wearer. “Before, they just used human inspection so for sure it affected consistency and had some human error,” he reveals. Until recently, the transparency of contact lenses made implementing machine vision-based detection a challenge in terms of accuracy and consistency. “But Teledyne Dalsa’s machine vision solutions for this purpose are based on user-friendly, low-cost iNspect embedded vision software and hi-resolution Genie Nano cameras.” These high-speed, low-noise global shutter CMOS cameras are ideal for fast and fine-detail inspection tasks, even when dealing with transparent objects like contact lenses.

Machine vision can also be used to detect embedded particles, inclusions or contamination on contact lenses that could present a hazard to the wearer (Image: Teledyne Dalsa)

Another example is pill inspection. Straight after manufacture, pills are fed onto a conveyor that gently rotates the tablets as they pass to confirm their features are correct in terms of dimensions, surface texture, shape, colour and labelling, and to ensure there are no surface defects. Until recently, this was left to humans, as machine vision solutions could not cope with the visual complexity of the scene, often flagging false positives and missing damaged pills.

But AI-powered systems are now taking over this laborious and tedious work. Cognex’s In-Sight D900 vision system is a CMOS camera embedded with a full suite of deep learning tools ideal for this application. Trained on images of defect-free tablets taken from various angles, it detects any pills with anomalies, while leaving therest to continue through the production process.

In-Sight D900 is part of a growing trend towards sophisticated AI solutions. “Human-like AI judgement is becoming the modus operandi,” summarises Deats. “Traditional human judgements like ‘is it a bubble or is it a blister that makes it a non-conforming product?’ or ‘is it a scratch or actually a hole in the package that compromises what’s inside of it?’ are very tricky for conventional vision systems, and even humans, but not for AI machine vision systems.”

Zhu agrees, offering his own example. “Feature analysis techniques can be used to qualify the concentration of a colour for a pharmaceutical ingredient, or the texture – for example, if the surface is smoother or rougher,” he explains. “Before, conventional vision algorithms have had difficulty tackling something with complicated textures, but with AI technology and deep learning, which we have been working on lately, this becomes very possible.”

Despite these benefits, take-up for advanced AI machine vision systems has been slow in the pharmaceutical industry. One reason is that the very standards that make machine vision such a good fit with the pharmaceutical industry also drag out the testing and verification processes for new products. Validating automation products is also particularly time-consuming and expensive due to the raft of documentation needed and the involvement of various stakeholders. As a result, if a pharma company finally manages to install an automated vision system that works, the company tends to stick with it.

Inspecting syringe seals is another modern pharamaceutical application of Cognex's cameras (Image: Cognex)

Another reason has been the effort, investment and know-how required in labelling images, training the AI system to recognise passable or faulty products, and customising it to a particular company’s needs. For this, Deats is excited by Cognex’s latest product, In-Sight 3800. “In-Sight 3800 has a new thing called edge learning on it, which is basically a subset of deep learning, but it’s really easy to set up and it’s really powerful,” he explains. “So getting pass and fail results on things that used to take days or weeks to program using traditional vision, you can do with five or six images now. And you can categorise 300 different failure modes on the fly in the production environment, without any fancy licensing, as everything’s done straight on the camera.”

Deeper into production

With serious improvements expected from AI, deep learning, as well as more advanced 2D and 3D imaging technologies, where else will machine vision penetrate the pharmaceutical industry?

“When you have a large manufacturing line, every time you change products, five or six individuals go around with a notepad and physically have to check every single pinch point [places on a production line representing a potential hazard to humans] and sign it off to the manufacturing process standard,” Deats says. “That costs some companies £1 billion plus a year.” Cognex’s solution, in development with partners, is to position 60 to 70 small cameras around the manufacturing line with a view of anticipated pinch points. These cameras produce a series of images from which a deep learning library flags anything abnormal. “With this, one person can actually check if it’s normal or not on an iPad and digitally sign that off and send it to the cloud without any human intervention.”

Zhu also has specific applications in mind. “Crystallisation is a kind of purification process during drug manufacture,” he outlines. “Before, you had to put everything under a microscope in offline analysis because conventional vision technology finds it very hard to separate crystals that are overlapping, but with AI this becomes feasible.”

What all three agree on is that AI-enabled machine vision, coupled with the Internet of Things (IoT) and robotics, will be a key technology in driving the next technological revolution in pharmaceutical manufacture, Pharma 4.0. “Powerful vision systems will enable customers to deliver smart production where they become even more efficient, accurate and better compliant with the given regulations and requirements,” says Luu. But it won’t happen overnight. “Systems like these are cost-intensive to develop but also for customers to integrate with their existing infrastructure,” Luu concludes. “It will take some time to get to the point these systems are reliable and qualified in a pharmaceutical manufacturing setting.”

Learn how Edmund Optics’ 120i objectives are enhancing quality control in pharmaceuticals