Textbook in PDF format

This book presents the concept of Informed Machine Learning and demonstrates its practical use with a compelling collection of applications of this paradigm in industrial and business use cases. These range from health care over manufacturing and material science to more advanced combinations with Deep Learning, say, in the form of physical informed neural networks. The book is intended for those interested in modern Informed Machine Learning for a wide range of practical applications where the aspect of small data sets is a challenge.
Machine Learning with small amounts of data? After the recent success of Artificial Intelligence based on training with massive amounts of data, this idea may sound exotic. However, it addresses crucial needs of practitioners in industry. While many industrial applications stand to benefit from the use of AI, the amounts of data needed by current learning paradigms are often hard to come by in industrial settings. As an alternative, learning methods and models are called for which integrate other sources of knowledge in order to compensate for the lack of data. This is where the principle of “Informed Machine Learning” comes into play.
Informed Machine Learning combines purely data driven learning and knowledge-based techniques to learn from both data and knowledge. This has several advantages. It reduces the need for data, it often results in smaller, less complex and more robust models, and even makes Machine Learning applicable in settings where data is scarce. The kind of knowledge to be incorporated into learning processes can take many different forms, for example, differential equations, analytical models, simulation results, logical rules, knowledge graphs, or human feedback which makes the approach overall very powerful and widely applicable.
The past decade has seen substantial progress in the field of Artificial Intelligence (AI). This has primarily been due to the increasingly rapid developments in the field of machine learning (ML) which, in turn, benefited from the confluence of four technological trends: (1) availability of ever-increasing training data sets, (2) comparatively cheap high-performance computing hardware, (3) open source code sharing and access to software for model training or to pre-trained models, and (4) theoretical and practical progress in deep learning and artificial neural networks. As a consequence, there have been significant advancements, say, in natural language processing, image/speech recognition, or autonomous systems. As a result of these developments, AI has now made its way out of academic research into companies and our daily lives.
A key feature of today’s cutting-edge AI technologies is their hunger for resources. This is because modern ML models (deep neural networks) have become incredibly large and complex and involve millions if not billions of adjustable parameters. Their training therefore requires enormous amounts of data and considerable computing infrastructures and therefore energy. Alas, in many industries and application domains, data is still scarce or incomplete and there often is limited access to large-scale high performance computing facilities.
But even if data availability, compute resources, and energy costs are not an issue, model complexity may still pose challenges with respect to explainability, accountability, or trustworthiness of AI solutions which can be dire in settings where regulatory guidelines have to be met or safety guarantees must be ensured. This is where the paradigm of Informed Machine Learning (Informed ML) comes into play. In a nutshell, the idea of Informed ML is to systematically leverage additional prior knowledge for the design and training of data-driven AI models. The overall goal is to use reliable background knowledge in order to, on the one hand, reduce model complexity and the need for extensive training data and, on the other hand, increase interpretability and explainability of the decisions made by trained models.
Introduction and Overview
Part I Digital Twins
Optimizing Cooling System Operations with Informed ML and a Digital Twin
AITwin: A Uniform Digital Twin Interface for Artificial Intelligence Applications
Part II Optimization
A Regression-Based Predictive Model Hierarchy for Nonwoven Tensile Strength Inference
Machine Learning for Optimizing the Homogeneity of Spunbond Nonwovens
Bayesian Inference for Fatigue Strength Estimation
Incorporating Shape Knowledge into Regression Models
Part III Neural Networks
Predicting Properties of Oxide Glasses Using Informed Neural Networks
Graph Neural Networks for Predicting Side Effects and New Indications of Drugs Using Electronic Health Records
On the Interplay of Subset Selection and Informed Graph Neural Networks
Informed Machine Learning Aspects for the Multi-Agent Neural Rewriter
Part IV Hybrid Methods
Training Support Vector Machines by Solving Differential Equations
Informed Machine Learning to Maximize Robustness and Computational Performance of Linear Solvers
Anomaly Detection in Multivariate Time Series Using Uncertainty Estimation