Survey Table Generation from Academic Articles

Po-Chun Chen, Hen-Hsen Huang, Hsin-Hsi Chen

Abstract: With the exponential growth in the number of scientific research papers, researchers are often challenged to rapidly grasp new advancements and make meaningful comparisons between scholarly articles. The academic community has proposed several strategies to address this issue, such as Multi-Document Scientific Summarization (MDSS) and leaderboards. However, these solutions have their limitations in capturing flexible and detailed information. Among recent developments, the Open Research Knowledge Graph (ORKG) provides a platform for custom literature comparison tables, but it still depends on manual editing. As an initiative to overcome these challenges, we introduce the task of auto-generating Academic Article Survey Table (AAST). Using tables from arXiv survey papers, we have established a unique dataset enriched with supplementary information generated by large language models (LLMs). We proposed a three-tiered evaluation method to assess the system performance at the cell, row, and table levels. Our proposed LLM-based approach seeks to automate the creation of AASTs. Considering the context windows limitations of LLMs, we employed semantic compression techniques to increase the amount of information that can be input. Experiments show that applying semantic compression to introduction section of reference paper improves the model performance.