Spanish Fake News Classifier: A Sequential Fine-Tuning Approach

Juan Ignacio Llaberia

Abstract:

Fake news detection in Spanish presents significant challenges due to the scarcity of highquality
annotated data and the variability in article length. While Transformer-based models
have demonstrated strong performance on short texts, their effectiveness on long-form news
articles remains limited, particularly in low-resource settings where large annotated datasets
are unavailable. In this work, we propose a Sequential Fine-Tuning approach for Spanish
fake news classification using a BETO variant, dccuchile/bert-base-spanish-wwm-cased.
Our method consists of a multi-step fine-tuning process in which the model is first adapted to
a large dataset of approximately 50k short news articles (pre-fine-tuning) and subsequently
fine-tuned on a smaller target dataset of around 2k long-form articles.
This progressive adaptation enables the model to learn task-specific representations while
improving generalization to longer documents. Experimental results on the test set show that
our approach achieves an accuracy of 0.8205, a precision of 0.7835, a recall of 0.8702, and an
F1-score of 0.8246, demonstrating improved performance and stable convergence compared
to direct fine-tuning on long documents. These findings highlight the importance of taskspecific
adaptation and provide a practica