Використання LLM для автоматизованого аналізу якості табличних даних

Abstract

This paper introduces AEDA (Automated Exploratory Data Analysis), a framework for automated data-quality assessment in tabular machine-learning datasets. AEDA integrates diverse analytical modules, including statistical analysis, missing-value and outlier detection, and rule mining, to evaluate data integrity. Uniquely, it employs a lightweight local large language model to generate explanations for identified issues and computes an overall quality score in accordance with ISO/IEC 25012 [1]. Experimental evaluations on real-world datasets confirm the framework's scalability and practical effectiveness.