Automated Knowledge Synthesis: An LLM-refined Framework for Evolutionary Topic Modeling

Abstract

Traditional topic modeling methods, such as Latent Dirichlet Allocation (LDA) and Non-negative Matrix Factorization (NMF), are limited by their context ignorance, static nature, and low interpretability. Building upon the hybrid approach LDA+NMF+class-based Term Frequency–Inverse Document Frequency (c-TF-IDF), a new formalized framework – Dynamic Contextual Topic Modeling with Large Language Model (LLM) Refinement (DCTM-LLM) – is presented. This LLM-refined framework integrates transformer embeddings for the detection of dynamic semantic clusters and leverages an LLM for their subsequent refinement and the synthesis of high -level narratives. Experiments on a corpus of 35,000 arXiv abstracts (cs.AI (Computer Science — Artificial Intelligence), 2015–2025) showed that DCTM-LLM achieves a Normalized Pointwise Mutual Information (NPMI) of 0.53, a Silhouette score of 0.62, an Adjusted Rand Index (ARI) of 0.55, and Topic Diversity at 10 of 0.88. Crucially, with a Bidirectional Encoder Representations from Transformers (BERT)-based score (BERTScore) F1 of 0.89, the method significantly outperforms Dynamic BERTopic (0.62) and the hybrid LDA, NMF, and c-TF-IDF approach (0.65). Thus, the proposed approach shifts the paradigm of topic modeling from keyword extraction toward utomated knowledge synthesis.