Mind Flows: A Comparative Study of Narrative Coherence in Human and LLM-Generated Stream-of-Consciousness Essays}

Nellia Dzhubaeva, Katharina Trinley, Laura Pissani

Keywords: humanities, digital humanities, narratives, NLP, LLMs

Abstract: This paper examines differences between stream-of-consciousness narratives written by humans and those generated by large language models (LLMs) to assess narrative coherence and personality expression. We generated texts by prompting LLMs (Llama-3.1-8B \& DeepSeek-R1-Distill-Llama-8B) with the first half of essays either revealing the personality characteristics (Big Five) or hiding them. Our analysis revealed consistently low similarity between LLM-generated continuations and original human texts, as measured by cosine similarity, perplexity, and BLEU scores. Including explicit personality traits significantly enhanced Llama-3.1-8B's performance, particularly in BLEU scores. Further analysis of personality expression showed varying alignment patterns between LLMs and human texts. Further analysis of personality expression found higher openness for all models. Additionally, Llama-3.1-8B models exhibited higher Extraversion but low Agreeableness, while DeepSeek-R1-Distill-Llama-8B's showed dramatic personality shifts during its thinking process, particularly when provided with personality traits in the prompt.

Archival Status: Archival

Paper Length: Long Paper (up to 8 pages of content)

Submission Number: 324