Prompting LLMs for Relation Classification in Portuguese: Is It Worth It?

Tomás Pinto, Bruno Ferreira, Catarina Silva, Hugo Gonçalo Oliveira

Abstract: This paper explores relation classification as a step toward extracting structured information from unstructured Portuguese text. We evaluate prompt-based approaches using generative large language models and compare them with fine-tuned BERT and DeBERTa models, which serve as the baselines in this study. Experiments are conducted on two Portuguese-language datasets: RelEx-PT, a custom sentence-level dataset created by aligning Wikidata triples with Wikipedia sentences, and a second consisting of multi-sentence texts annotated with multiple triples. Results show that, while prompt-based methods offer flexibility and solid performance, they are still not sufficient to overtake fine-tuning, as the latter yields significantly better results in identifying relation types, even widening the gap in more complex task settings.