A Framework of Security and Performance Enhancement for Hadoop

Adhishtha Tyagi, Sonia Sharma

Abstract


Hadoop framework has been emerged as the most effective and widely adopted framework for Big Data processing. Map Reduce programming model is used for processing as well as generating large data sets. Data security has become an important issue as far as storage is concerned. By default theres no security mechanism in hadoop and it is the first choice of the business analyst and industrialists to store and manage data as well as theres a need to introduce security solutions to Hadoop in order to secure the important data in the Hadoop environment. We implemented and evaluated Dynamic Task Splitting Scheduler (DTSS) which explores the tradeoffs between fairness and data performance by splitting the tasks dynamically before processing in hadoop along with AES-MR (an Advanced Encryption Standard based encryption using mapreduce) encryption in MapReduce paradigm. This paper would be useful for beginners and researchers for understanding DTSS scheduling along with security.

Full Text:

PDF

References


YongLiang Xu, Wentong Cai, “Hadoop Job Scheduling with Dynamic Task Splitting”, International Conference on Cloud Computing Research and Innovation, 2015.

B. Saraladevi, N. Pazhaniraja, P. Victer Paula, M.S. Saleem Basha, P. Dhavachelvan,” Big Data and Hadoop-A Study in Security Perspective”, 2nd International Symposium on Big Data and Cloud Computing ,2015,ISBCC.

Divya M, Annappa B,” Workload Characteristics and Resource Aware Hadoop Scheduler”, 2nd International Conference on Recent Trends in Information Systems, 2015, ReTIS.

Qinghua Lu, Shanshan Li, Weishan Zhang,” Genetic Algorithm based Job Scheduling for Big Data ”, International Conference on Identification, Information, and Knowledge in the Internet of Things,IEEE,2015

Asami Higai, Atsuko Takefusa, Hidemoto Nakada, Masato Oguchi,” A Study of Effective Replica Reconstruction Schemes at Node Deletion for HDFS”, ACM International Symposium on Cluster, Cloud and Grid Computing,2014,IEEE

Kala Karun. A , Chitharanjan. K,” A Review on Hadoop – HDFS Infrastructure Extensions”, Conference on Information and Communication Technologies,2013,IEEE

Jisha S Manjaly, Varghese S Chooralil,” TaskTracker Aware Scheduling for Hadoop MapReduce”, Third International Conference on Advances in Computing and Communications,2013.

Li Liu, Yuan Zhou1, Ming Liu, Guandong Xu, Xiwei Chen, Dangping Fan, Qianru Wang,” Preemptive Hadoop Jobs Scheduling under a Deadline”, Eighth International Conference on Semantics, Knowledge and Grids,2012,IEEE.

Viplove Kadre , Sushil Chaturvedi,” AES – MR: A Novel Encryption Scheme for securing Data in HDFS Environment using MapReduce”, International Journal of Computer Applications,2015,IEEE

Adhishtha Tyagi, Sonia Sharma,” A brief reviw of scheduling algorithms of MapReduce model using Hadoop”, International Journal of Engineering Trends and Technology, 2017, IJETT

https://blog.cloudera.com/blog/2008/11/job-scheduling-in-hadoop/

https://www.ibm.com/developerworks/library/os-hadoop-scheduling/

https://en.wikipedia.org/wiki/Ubuntu_(operating_system)

https://en.wikipedia.org/wiki/Apache_Hadoop

https://en.wikipedia.org/wiki/Wireshark




DOI: https://doi.org/10.23956/ijarcsse/V7I6/0171

Refbacks

  • There are currently no refbacks.


Bookmark and Share


© International Journals of Advanced Research in Computer Science and Software Engineering (IJARCSSE)| All Rights Reserved | Powered by Advance Academic Publisher.