Who Plays Which Role When? Communication Role Dynamics for Peer Recognition and Team Performance Prediction

Yifan Song, Wenxuan Wendy Shi, Brian Bailey, Tal August

Keywords: role, teamwork, team communication, education, group chat, Slack

Abstract:

Effective teamwork depends not only on what a team produces, but on how members communicate. A useful lens for describing these interaction patterns is $\textit{team roles}$. Behavioral and organizational science conceptualizes roles as complementary communicative functions enacted through interaction (e.g., $\textit{initiator}$ who proposes new ideas and tasks, and $\textit{arbitrator}$ who solves disagreement), rather than fixed personality traits. This perspective naturally raises a core question for computational analysis of team communication: who enacts which roles, when, and how do these patterns relate to collaboration quality?

Prior work has modeled functional roles for meeting participants from simple speech features, or used behavior patterns (e.g., turn-taking) to predict "latent" roles and team outcomes. Recently, large language models (LLMs) have renewed interest in roles through human-agent and agent-agent collaboration, where agents are assigned explicit social roles in games or simulated organizations. However, roles are often operationalized as domain-specific personas or functions (e.g., developer vs. manager) or clustered from data, rather than theory-grounded constructs as interactive/communicative functions. In addition, most prior work studies roles in large-scale (e.g., Wikipedia), controlled (e.g., crowdsource), or synthetic (e.g., agent simulation) contexts. While these contexts offer scale and control, they are different from the close-knit, long-lived teams that are common in educational or organizational settings, often missing the longitudinal evolution of real teams.

In this paper, we ground team roles to an authentic longitudinal setting: an in-person semester-long computer science course project in which teams relied on Slack for day-to-day coordination. We collect a dataset of 6307 messages from 55 students in 18 teams across eight project deliverable windows. We operationalize a role taxonomy grounded by education literature and develop a human expert annotation protocol for labeling roles across deliverable windows. We then evaluate whether LLMs can serve as an annotator for a scalable approximation of expert coding: the best model yields F1 = 0.86 and Kripp. $\alpha$ = 0.80.

Using these role labels, we provide descriptive analyses of role prevalence and trajectories over the project lifecycle in an educational context. For example, work-related roles like $\textit{explorer}$ and $\textit{facilitator}$ peak during the heavy implementation work phase, while $\textit{gatekeeper}$ substantially increases near the final week, the busiest and most stressful period. We also observe a progressive increase in the number of unique roles an individual exhibits simultaneously as project complexity grows.

To investigate the usefulness of our role framework on downstream tasks, we use roles to predict peer recognition, an individual-level performance measure by normalizing the peer evaluation scores collected in our dataset. To further validate the generalizability of our framework beyond educational settings, we use the same role constructs with LLM annotations on a public group dialogue benchmark (DelidData) to predict team performance gain on deliberation tasks. We show that role-based features improve predictive performance beyond lexical, conversational, and zero-shot prompting baselines in both downstream tasks, and can be further enhanced when supplementing with conversational statistics.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 139