Enhancing misinformation detection quality on small datasets using synthetic labeling

Abstract

The object of research is the process of automated misinformation detection, with particular emphasis the use of large language models (LLMs) as scalable annotators to solve a problem of labeled data scarcity (typical for low-resource languages). Manual fact-checking and annotation are costly, slow, and difficult to scale, which limits the applicability of machine learning approaches. The research addresses the problem of enhancing the misinformation detection quality on small datasets by adding the LLM-generated synthetic labels to the training process and evaluates the achieved performance improvements. The results demonstrate that incorporating LLM-generated annotations into limited training data consistently improves classification performance. The observed gains range from approximately 2 to 3 percentage points in the F1-macro score depending on the configuration. The relative improvement is higher in scenarios with smaller manual data used. These findings can be explained by the increased volume and diversity of training data provided by synthetic annotations. LLMs provide good fact-checking across more data samples, enriching the dataset and helping the classifier generalize better. However, factors such as LLM version and size, inclusion of additional context, or the use of external knowledge retrieval influence annotation accuracy. In practical applications, the results suggest that LLM-based annotation can serve as an efficient and scalable method for bootstrapping small datasets. This approach is especially valuable for underrepresented languages, where annotated datasets and benchmarks are scarce, as it can increase misinformation detection accuracy without expensive manual annotation.