What company do words keep? Revisiting the distributional semantics of J.R. Firth & Zellig Harris

Mikael Brunila, Jack LaViolette

Abstract:

The power of word embeddings is attributed
to the linguistic theory that similar words will
appear in similar contexts. This idea is specifically invoked by noting that “you shall know
a word by the company it keeps,” a quote
from British linguist J.R. Firth who, along with
his American colleague Zellig Harris, is often credited with the invention of “distributional semantics.” While both Firth and Harris are cited in all major NLP textbooks and
many foundational papers, the content and differences between their theories is seldom discussed. Engaging in a close reading of their
work, we discover two distinct and in many
ways divergent theories of meaning. One focuses exclusively on the internal workings of
linguistic forms, while the other invites us
to consider words in new company—not just
with other linguistic elements, but also in a
broader cultural and situational context. Contrasting these theories from the perspective of
current debates in NLP, we discover in Firth
a figure who could guide the field towards a
more culturally grounded notion of semantics.
We consider how an expanded notion of “context” might be modeled in practice through
two different strategies: comparative stratification and syntagmatic extension.