Conference Papers

Permanent URI for this collectionhttps://idr.nitk.ac.in/handle/123456789/28506

Browse

Author: search.filters.author.Chandrasekaran, K.Subject: search.filters.subject.Hadoop

Search Results

Now showing 1 - 3 of 3
  • No Thumbnail Available
    Item
    Analysis of MapReduce scheduling and its improvements in cloud environment
    (Institute of Electrical and Electronics Engineers Inc., 2015) D'Souza, S.; Chandrasekaran, K.
    MapReduce has become a prominent Parallel processing model used for analysing large scale data. MapReduce applications are increasingly being deployed in the cloud along with other applications sharing the same physical resources. In this scenario, efficient scheduling of MapReduce applications is of utmost importance. Also, MapReduce has to consider various other parameters like energy efficiency and meeting SLA goals besides achieving performance when executing jobs in cloud environments. In this work, we have classified MapReduce Scheduling as Cluster based Scheduling and Objective based Scheduling. We then summarize and analyse the different class of schedulers highlighting the strong points and limitations of each of the scheduling approaches. The Adaptive scheduling techniques provide dynamic resource management and meet performance goals. The Energy efficient scheduling techniques aim to cut data centre costs by using different approaches. Finally, we discuss the current challenges and future work. © 2015 IEEE.
  • No Thumbnail Available
    Item
    Improving false alarm rate in intrusion detection systems using Hadoop
    (Institute of Electrical and Electronics Engineers Inc., 2016) Mukund, Y.R.; Nayak, S.S.; Chandrasekaran, K.
    Intrusion Detection Systems are a vital part of an organization's security. This paper gives an account of the existing algorithms for Intrusion Detection using Machine Learning, along with certain new ideas for improving the same. The paper mainly talks about employing the Decision Tree mechanism for Intrusion Detection and improve it with the distributed file system, Hadoop. Initially a method that uses a dirty-flags to check the consistency of the Decision Tree, which changes with every wrong classification of the system is employed. The wrong classification is identified by a certain user who informs the system about the same and helps it learn. In the further sections, a new method which does not use a dirty-flag, but rather modifies the Key-Value pair in the results of the reduce() function is tested as an improvement to the previous method. The two methods are compared, with the help of the Hadoop Simulation Tool - YARN. The main aim of the paper is to propose the use of the Distributed File System for Machine Learning along with some improvements to the current Hadoop File System, so that it reduces the total Time Taken, when Machine Learning algorithms are employed along with it. © 2016 IEEE.
  • No Thumbnail Available
    Item
    A Machine Learning Approach for Daily Temperature Prediction Using Big Data
    (Springer Science and Business Media Deutschland GmbH, 2022) Divakarla, U.; Chandrasekaran, K.; Hemant Kumar Reddy, K.H.K.; Reddy, R.V.; Rao, M.
    Due to global warming, weather forecasting becomes complex problem which is affected by a lot of factors like temperature, wind speed, humidity, year, month, day, etc. weather prediction depends on historical data and computational power to analyze. Weather prediction helps us in many ways like in astronomy, agriculture, predicting tsunamis, drought, etc. this helps us to be prepared in advance for any kinds disasters. With rapid development in computational power of high end machines and availability of enormous data weather prediction becomes more and more popular. But handling such huge data becomes an issue for real time prediction. In this paper, we introduced the machine learning-based prediction approach in Hadoop clusters. The extensive use of map-reduce function helps us distribute the big data into different clusters as it is designed to scale up from single servers to thousands of machines, each offering local computation and storage. An ensemble distributed machine learning algorithms are employed to predict the daily temperature. The experimental results of proposed model outperform than the techniques available in literature. © 2022, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.