Methodological Guideline for Data Engineering

In AI (Artificial Intelligence) and ML (Machine Learning), data is the complementary part of algorithms. Algorithm performance depends on data quality. Mastering the data life cycle in an End-to-End process, to provide trustworthy data, is fundamental in the development of critical/complex systems. This methodological guide explains how to perform data engineering activities for this purpose. The structure of this guide is as follows: The Overview section introduces the reason to stress on the data lifecycle, presents an overall view of the data activities described in this guide, and the position of this data lifecycle in the context of the End-to-End process Next sections detail each activity organized by phase. The last section is exploratory and indicates how to combine data and ML techniques.