Making Machine Learning Easy with Embeddings

Dan Shiebler, Abhishek Tayal

Abstract:

Modeling teams at Twitter face a variety of uniquely hard, yet fundamentally related machine learning problems. For example, tasks
as different as ad serving, abuse detection and user timeline construction all rely on powerful representations of user and content
entities. In addition, because of Twitter’s realtime nature, entity
data distributions are constantly in flux, so these representations
must be frequently updated. By generating high quality, up-to-date
representations and sharing them broadly across teams, we can
decrease duplication of efforts and multiplicatively increase crossteam modeling productivity. At Twitter Cortex, we are making these
representations "first class citizens" of the Twitter ML platform by
commoditizing tools and pipelines that create high quality, custom,
regularly retrained, benchmarked and centrally hosted embeddings.