Profile generation from web sources: an information extraction system

Rishabh Ranjan, H. Vathsala, Shashidhar G. Koolagudi

Abstract: The Internet space has a vast collection of information which is not always structured. These sources of information such as social media, news articles, blogs, speeches and videos often contain information that could be utilized to generate decision making tools such as reports about events and individuals. Using this information is a long and tedious process if done manually. Over the years, a lot of research has been done in data mining and natural language processing techniques to facilitate the consumption of this vast amount of data. The current work describes ProfileGen, an information extraction system that uses a variety of these data sources to form a profile of a given person. There are two parts to this application: The first part uses information publicly available on social media sites, news articles on news websites and blogs and compiles this information to form a corpus about the given person, and in the second part, the information is ranked using machine learning techniques, so as to provide information in the order of importance.