Large-Scale Encryption in the Hadoop Environment: Challenges and Solutions

Raj R. Parmar, Sudipta Roy, Debnath Bhattacharyya, Samir Kumar Bandyopadhyay, Tai-Hoon Kim

Abstract: Data is growing at an enormous rate in the present world. One of the finest and most popular technologies available for handling and processing that enormous amount of data is the Hadoop ecosystem. Enterprises are increasingly relying on Hadoop for storing their valuable data and processing it. However, Hadoop is still evolving. There is much vulnerability found in Hadoop, which can question the security of the sensitive information that enterprises are storing on it. In this paper, security issues associated with the framework have been identified. We have also tried to give a brief overview of the currently available solutions and what are their limitations. At the end a novel method is introduced, which can be used to eliminate the found vulnerabilities in the framework. In the modern era, information security has become a fundamental necessity for each and every individual. However, not everyone can afford the specialized distributions provided by different vendors to their Hadoop cluster. This paper presents a cost-effective technique that anyone can use with their Hadoop cluster to give it 3-D security.