Optimizing Pharma Production with AI Recognition
Posted on 14 October 2020



by Alexander Demmer

Download as PDF



  • 68% of defects in industrial production are caused by humans. Even in interconnected and smart Industry 4.0 environments, human activities are a blind spot
  • There is a next-gen technology entering the market: Video-based AI error recognition technology makes the production process more reliable: It supports individuals in complying with hygiene, safety, and quality requirements by spotting irregularities in real time
  • In a highly regulated industry where minute errors can lead to high follow-up costs, video-AI error recognition, operating as invisible Cobots (electronical partners) prevents production delays, downtimes, recalls, recourse claims, material waste and accidents, this way significantly reducing error-related costs
  • Video-based AI error recognition will considerably facilitate compliance with GMP requirements and improve margins


“Image recognition is about pixel by pixel analysis in order to recognize a particular object. A new technology is entering the market: Video-based AI Error Recognition recognizes anomalies and patterns in video data and interprets them just like a human eye and brain. This new technology has the potential to disrupt how errors on the shop floor are prevented, recognized and corrected.”

Marc Vorderman, Director Europe, Novatek International



There has been an escalating need for an uninterrupted supply of drugs globally with new discoveries of diseases and persisting chronic illnesses on the rise. These emerging global needs will require optimized efficiencies within each function in the pharmaceutical industry, especially within manufacturing.

Clearly, the time is right and the industry is well equipped to face the upcoming challenges. A multitude of technological innovations is currently changing the nature of manufacturing across the globe and in all sectors. The promise of Industry 4.0 in pharma – Pharma 4.0, describes a smart factory: highly automated, adaptive devices and facilities, governed by control systems that integrate, monitor and analyse all parts of the manufacturing process with sensors acting as “organs of perception”.



Pharma manufacturing is a highly regulated, demanding environment with zero error tolerance. Minute production irregularities, non-compliance to safety and hygiene requirements can have serious consequences. In order to ensure the strength of the active ingredients as well as quality and purity of the final products, high standards need to be maintained throughout the entire production process. Flaws and irregularities can lead to substantial production losses and severe planning and logistic issues, even resulting in reputational damage.

It is obvious that increasing automation will support pharma´s constant search for perfection. Sensors measure parameters like position, velocity, pressure, vibration, temperature. They will become increasingly important to streamline and monitor the production process and provide real-time data for further processing and analysis. In this whole framework, optical sensors play an important role. These expert cameras typically analyse and verify objects that have a constant shape, texture and form, by comparing objects in a 2D setting; pixel by pixel.



While Industry 4.0 has its clear focus on machines, humans still perform nearly three in four tasks on the production floor-while remaining almost invisible to analytics. Therefore, what´s missing from the automation conversation – though not from the production floor – is humans.

Despite their clear benefit – humans bring reason, creativity and problem-solving skills and continue to add the majority of the value in manufacturing operations. Their contribution however has one significant disadvantage: they introduce variability, which opens the door to errors and defects. According to the AT Kearney study: “The state of human factory analytics” 1, 68 percent of defects are caused by humans.

Traditionally, error mitigation strategies have been training and supervision. However, training cannot fully eliminate human errors. Supervision by humans is error-intensive, biased and insufficient. Minute safety and compliance errors can cause huge follow-up costs: material waste, production delays, recalls, reputational damage as well as health risks.

What brings human intelligence and the required scrutiny together? Cobots might be the answer to the high standards that pharma manufacturing sets forth: rather than replacing, Cobots augment the human skillset and take over the repetitive, accuracy, availability part in the equation.



According to Athina Kanioura, Chief Analytics Officer and Global Lead – Accenture Applied Intelligence in her research report: Human AI is here,

“There is tremendous potential in humans working collaboratively with machines to develop new forms of intelligence and apply them to business … problems.” 2

Human motions, gestures, characteristics are 3D, their activities are clearly too complex to be recognized, let alone traced, analysed by traditional optical sensors. It clearly needs a more sophisticated AI technology: a smart, pattern recognizing Artificial Intelligence System that understands anomalies and outliers against a constantly changing backdrop and signals non-compliance activities and threats.

Video-based AI error recognition is a technology that operates in the background and does not require any operation training or specific skills. Standard video cameras equipped with video-based AI error recognition detect narrowly defined anomalies, e.g.:

  • Safety or production risks arising from unusual or unwanted movements or gestures
  • Scratches, deformations, faulty packaging, faulty assembly, number of objects, filling levels, anomalies in shape or color caused by human error
  • Leaks in containers or anomalies in materials like fluids, compounds, powder that would otherwise need closer inspection by humans

Once the technology has detected an anomaly, the system triggers an alert. This can be audible, visible or an automated action like stopping the process and allowing to correct the error.



Pharma manufacturers leverage video-based AI error correction to mitigate error-related cost:

  • An operator has forgotten to put on his head cap, goggles, gown or gloves, thus violating hygiene requirements
  • An operator is injured while trying to remove an obstructing object in the production line, instead of calling the service team
  • A bottle on the packaging line is without a label, not correctly closed or broken
  • Safety equipment like a fire extinguisher or a safety shower is blocked by an object, not a human
  • Leaks in connectors to a bioreactor or other vessel are detected that would open an ingress route for contaminants

In many more situations, video-based AI error-recognition can drastically enable error correction “on the spot”. Coming back to the examples, a camera at the entrance of a hygiene area would “see” that the cap, goggles, gown or gloves are missing and an audible alarm hint at the fact that safety instructions are not observed. The technology spots the “anomaly” bottle and ejects it or halts the process, so it can be removed. It will instantly recognize that it is an object – not an individual – which obstructs free access to the life-saving equipment. A set of cameras will detect liquid on the bioreactor surface as well as spillage on the floor. Video based AI error-recognition works like a manager in the background who warns of dangers and enables quick corrections. AI error-recognition automates tedious monitoring tasks that otherwise humans would perform. In fact, the technology “sees” like a human, but does not get tired or distracted, it does not need to be organized in shifts, it does not ask for holidays or sick leave. It operates reliably 24/7.



The pharmaceutical industry is already experiencing regulatory scrutiny in a number of areas, including quality control activities around manufacturing operations. For example, the European Commission requires that:

“…records are made, manually and/or by recording instruments, during manufacture which demonstrate that all the steps required by the defined procedures and instructions were in fact taken and that the quantity and quality of the product were as expected. Any significant deviations are fully recorded and investigated” 3

Video-based AI error-recognition makes it easy to trace back errors during production and packaging. The solutions record and filter all deviations from the normal process and make it easy to spot the moment where the anomaly happened. Administrative staff working on investigation reports addressing complaints will never again need to crawl through many, many hours of video footage in order to find that one irregularity that spoiled that specific batch.



Video-based AI error-recognition offers valuable insights and reduces error-related cost. In order to make projects successful, companies however need competent partners and a clear strategy:

Business View: Video-based AI error recognition projects need a mentor not only from a technical but very much from the business perspective: Where do errors happen frequently causing severe financial impact? It is crucial to clearly identify the pain and risk and from there build the business case. Expert partners will discuss and assess in detail how video-based AI error recognition can make a difference.

Fit for Purpose Data: As AI Systems need to be trained, relevant data is crucial for the adaptation and implementation period. Expert partners will give advice on what content is relevant for reaching the defined goal and accuracy. With the right data, they will adapt the standard video-based AI Solutions to match customers´ specific requirements.

Scale: Innovation projects need a clear business perspective. What next steps are required to make the technology available to a larger audience? How can the solution be implemented across business lines, geographies and production sites? AI projects need a strategic scaling plan in order to yield business value.

Cultural Fit and Maturity to Transform: It is obvious that in a company with a strong DNA in Lean Management (a culture of Kaizen / Continual Improvement) and intensive experience with various transformational waves the success of projects of this kind will be accelerated. Technology is improving processes creating new roles and responsibilities by unburdening highly qualified employees of standard activities. The way those will be upgraded to execute more sophisticated tasks will increase the acceptance of the organization to support this kind of innovations.

Economic Value: When it comes down to cost and project duration, standard video-based AI error recognition solutions are by no means comparable with typical IT implementations or Industry 4.0 projects. Implementation and deployment projects usually take just a few months, mainly due to the learning capabilities of AI Systems. The total costs of ownership, depending on the number of potential anomalies that need to be monitored, are easy to calculate similar to the return on investment compared to expenses caused by inadequate human behaviour. ROIs below six months are not uncommon.



Pharma companies continue to face challenges of globalization, complex supply chains and hyper-competition – all while demand for treatments continues to increase. As a result, the need for greater throughput, higher quality and reduced costs has become a top priority.

Video-based error-recognition for pharmaceutical development and production is a smart new technology that will replace tedious, error-intensive human supervision thus avoiding production delays, accidents, material waste, recalls, compliance fulfilment and reputation damage. In the pharma world, it will make a real difference both in competitiveness and cost reduction.



Advanced AI algorithms are screening the hidden relationships between events and behaviour building evolved models to predict results with high accuracy. High complex calculations can be executed with limited computing power with technology working on small devices.

Start-up companies are driving the race with fresh ideas and innovative test models. Among those DeepEyes is a good example of how technology can be made available to any kind of manufacturer by offering solutions working on any data base, neural networks, internet data sources as well as standalone without internet access. The algorithms operating in real-time are edge-capable, they don’t need cloud access for the analysis of video streams and are running on standard hardware and standard cameras.

The combination of IIoT (Sensors) with AI is ringing a new era in which again the humans make the difference!


1 A.T. Kearney: The State of Human Factory Analytics https://www.kearney.com/digital-transformation/the-state-of-human-factory-analytics

2 Accenture, February 2019: Human AI is here https://www.accenture.com/au-en/insights/digital/human-ai-here

3 https://ec.europa.eu/health/sites/health/files/files/eudralex/vol-4/pdfs-en/2005_10_gmp_part1_chap1_en.pdf

Category: Reports

For news and updates about GBR please follow us on LinkedIn

Francois Germain

Francois Germain holds a Masters’ degree in General Engineering. His career spans power generation, chemicals, refining and offshore. In 2014 he joined Total Refining and Chemicals as VP Health & Safety where he leads global Safety Culture transformation programs for several Business Unit’s within Total. Francois is now leading Digital Innovation within Total Refining and Chemicals

Ron Wissenburg

Ron his main experience is redesigning and implementation of the changes in supply chain processes and organization by leveraging IT solutions. He started his career in consulting at Deloitte where he supported business transformations with SAP in various industries. He Joined Alstom Power in March 2010 which in 2016 Alstom became General E. At Alstom/GE he had various IT leadership roles and the past 3 years he has been working on driving Digital Technology programs within Quality and EHS for Gas Power.

Pierluigi Petrali

Pierluigi Petrali has been working Whirlpool Europe since 1989. He started his career in Information Technology, working at introduction in Whirlpool of CIM, MES and Fault Tolerant Networks. He has then worked in innovation projects, contributing to the spread and growth of TRIZ methodology in Italy and Europe and to the development of product/process design support approaches. Presently he coordinates manufacturing research and development activities for EMEA region. Directly involved as technical and main partner in more than 10 funded research projects under FP7 and H2020 program in the last five years, he’s currently working in definition and implementation of Whirlpool Industry 4.0 strategy. He’s the inventor of five patents and author several technical papers and public speech on TRIZ, Technology Forecasting, Manufacturing Quality, Internet of Things and Industry 4.0.

Henrik Fransson

Henrik Fransson is a Swedish manager with an aeronautics and computer science background working for Airbus in Hamburg, where he has held various management positions. He is passionate about finding new ways of working and influencing the lives of others by making their daily life easier.

“Use my data to serve me, do it wisely and with care”

Bjoern Neal Kirchner

Bjoern Neal Kirchner, a graduated economist from Munich and London universities, has almost 20 years’ experience in logistics and supply chain management at senior level. Shifting twice between China and Europe, he has worked in 5 countries moving along the entire supply chain. He gained experience in emerging markets and led teams through challenging SAP-implementations and post-merger integration projects. With the focus on building high performance teams, Bjoern has been engaged in various business process optimization and organizational transformation projects. Currently, he is leading the Global Supply Chain team in Amsterdam for Adhesive Technologies of HENKEL.

Stefan Schrauf

Dynamic, self motivated and results-oriented cross-cultural team player with a proven track record of digital operations and supply chain strategy, excellence and transformations across several industries including industrial products, automotive, consumer goods and chemicals.

Dr. Stefan Kluge

• Education: Chemistry, Information Technology, General and Environmental Law, Toxicology

• Companies: DASA, General Electric Capital/ BMW, Siemens, Infineon, Intel

• Functions: R&D, Systems Engineering, Factory Automation, Strategic Planning, Manufacturing Strategy, Business Planning, Outsourcing Strategy, Supplier Development, External Manufacturing Management

• Currently: Sr. Director Operations & Systems Enabling, Corporate Planning Group, Intel

Toni Sirviö

Mr. Toni Sirviö has been working at Konecranes Plc. since 2005 with various responsibilities within Sales development, Commercial Excellence and Supply Chain Management including Project Management, Global Order Management, Pricing, Planning and ERP Tool process and conceptual development/deployment. Mr. Sirviö has also been working in the Commercial Excellence program as a project director for the sales lead
development process.

Håvard Jörgensen

Håvard Jörgensen is in his role as VP Group Supply chain in Electrolux, leading the work on the strategic direction and plans for the Electrolux supply chain. Specific focus on SC Resilience, digital development, E2E SC integration, SC Sustainability, processes and capabilities. He has +20 years of experience within Supply Chain and Operations from various leadership roles in Procter & Gamble, Arla Foods and Coca-Cola European Partners.

Sean Garoghan

An experienced and results orientated Sustainability and Compliance senior manager with a background of 10 years working within the service and energy sector with strong leadership and people management skills and specialising in Sustainability and Compliance. A professional buyer with a wealth of experience in stakeholder management and developing supplier compliance frameworks. Pro-efficiently experienced in various procurement and supplier systems such as; ORACLE 11i & R12, SAP, Proactis and Avetta. Experienced within all Microsoft and Google packages, highly proficient in Excel and highly experienced in data analysis and commercial intelligence with extensive knowledge within OHSAS and ISO audits and accreditation's and CDM regulations.

Vera Bartsch

Vera Bartsch is the Sustainability Manager for UPM Raflatac’s EMEIA region. She leads the self-adhesive label company’s sustainability efforts in the market. Vera’s activities include providing expert sustainability consultancy for sales, stakeholder and partnership development, management of eco-design, life cycle assessment with UPM Raflatac’s Label Life tool, recycling of label waste through the company’s RafCycle program as well as sustainability marketing and communications.

Vera has a background in sales of label material with regards to sustainability in Central Europe, working with both label printers and brand owners. She coordinates the cooperation with other UPM business units for UPM Raflatac and has worked on several international projects for the development of UPM Raflatac.

Rebekah Bell

Rebekah currently serves as the Work Management & Planning Manager for Southern Company’s Fleet Performance & Reliability organization. Rebekah was the leader in the integration and implementation of CiM Visual Planner for daily work scheduling and has spent the last seven years of her career as a subject matter expert on work management, maintenance strategy, and process improvement across the Southern Company fossil, hydro, and renewables fleet.
Rebekah began her career with Southern Company in 2006, and over the last 16 years Rebekah has served in a variety of maintenance roles, including experience additionally in contractor relations, fleet outage optimization, and reporting and analytics.

Rebekah holds a Bachelor of Science in Chemistry from the University of South Alabama and a Master of Engineering in Construction Engineering Management from the University of Alabama at Birmingham. She is the proud mother of Kimberly Grace, 17.

Harald Kirsch

Dynamic, entrepreneurial business strategist with over 30 years of professional experience in organizational and business development in the segments of Digital Transformation, IIoT, Artificial Intelligence, Cyber Security, automation of enterprise processes, sales of innovative SW solutions and services, portfolio management, consulting, system integration services, outsourcing as well as business transformation, change management, product management and product development.

Jean Charbonneau

Mr. Jean Charbonneau is a renowned worldwide speaker with over 30 years of experience in asset reliability and a leading influencer on the best methods for asset intensive organizations to achieve maintenance excellence. He founded the Maintenance Excellence Group in 1997, putting together a team of asset reliability experts to assist organizations with maximizing their production output while minimizing asset maintenance costs. MEG’s flagship solution, the Visual Planner Digital Platform, is used by over 10,000 maintenance professionals from all industrial sectors worldwide, allowing them to execute the right maintenance, at the right time, at the lowest costs.

Rachel Cooke

I am a technical leader with over 15 years of experience in global operations in food and drink and logistics. I have held a range of supply chain roles including engineering, manufacturing, purchasing and project management.

I currently lead Reliability Engineering for Amazon Europe, which includes robotics, reliability programs, learning and information services, predictive analytics (machine learning), computerized maintenance management systems, new technology introduction and new site launches. Previously, I was Head of Central Programs and Capability for Amazon Engineering Services Europe, where I led several teams that supported the growth of Amazon's warehousing capacity. Prior to this I was responsible for manufacturing performance measurement and operational excellence for SABMiller (FTSE-10), who were the world’s second largest brewer by volume with over 100 operations worldwide in Africa, Asia Pacific, Europe, Latin America and the USA. Before this I led a team delivering a portfolio of product change initiatives from project initiation through to implementation and launch at Cadbury. I have spent three years living in Poland, where I worked on the design, construction and commissioning of two new-build confectionery factories.