States of LLM-generated Texts and Phase Transitions between them
Keywords: LLM, autocorrelations, phase transitions
TL;DR: LLMs can generate text that can be classified as solid, critical state or gas
Abstract: It is known for some time that autocorrelations of words in human-written texts decay according to a power law. Recent works have also shown that the autocorrelations decay in texts generated by LLMs is qualitatively different from the literary texts. Solid state physics tie the autocorrelations decay laws to the states of matter. In this work, we empirically demonstrate that, depending on the temperature parameter, LLMs can generate text that can be classified as solid, critical state or gas.
Submission Number: 42
Loading