Hadoop based real-time intrusion detection for high-speed networks

The rate of data generation is enormously growing due to the number of internet users and its speed. This increases the possibility of intrusions causing serious financial damage. Detecting the intruders in such high-speed data networks is a challenging task. Therefore, in this paper, we present a high-speed Intrusion Detection System (IDS), capable of working in Big Data environment. The system design contains four layers, consisting of capturing layer, filtration and load balancing layer, processing layer, and the decision-making layer. Nine best parameters are selected for intruder flows classification using FSR and BER, as well as by analyzing the DARPA datasets. Among various machine learning approaches, the proposed system performs well on REPTree and J48 using the proposed features. The system evaluation and comparison results show that the system has better efficiency and accuracy as compare to existing systems with the overall 99.9 % true positive and less than 0.001 % false positive using REPTree.