Big data analytical architecture using divide-and-conquer approach in machine-to-machine communication

Awais Ahmad, Anand Paul, M. Mazhar Rathore, Seungmin Rho

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Machine-to-Machine (M2M) technology unremit-tingly motivates any time-place-objects connectivity of the devices in and around the world. Every day, a rapid growth of large M2M networks and digital storage technology, lead to a massive heterogeneous data depository, in which the M2M data are captured and warehoused in the diverse database frameworks as a magnitude of heterogeneous data sources. Hence, the M2M that handles Big Data might perform poorly or not according to the goals of their operator due to massive heterogeneous data sources may face various incompatibilities, such as data quality, processing and computational efficiency, analysis and feature extraction applications. Therefore, to address the aforementioned constraints, this paper presents a Big Data Analytical architecture based on Divide-and-Conquer approach. The designed system architecture exploits divide-and-conquer approach, where big data sets are first transformed into a several data blocks that can be quickly processed, then it classifies and reorganizes these data blocks from the same source. In addition, the data blocks are aggregated in a sequential manner based on a machine ID, and equally partitions the data using filtration and load balancing algorithms. The feasibility and efficiency of the proposed system architecture are implemented on Hadoop single node setup. The results show that the proposed system architecture efficiently extract various features (such as River) from the massive volume of satellite data.

Original languageEnglish
Title of host publicationProceedings - 2015 IEEE 12th International Conference on Ubiquitous Intelligence and Computing, 2015 IEEE 12th International Conference on Advanced and Trusted Computing, 2015 IEEE 15th International Conference on Scalable Computing and Communications, 2015 IEEE International Conference on Cloud and Big Data Computing, 2015 IEEE International Conference on Internet of People and Associated Symposia/Workshops, UIC-ATC-ScalCom-CBDCom-IoP 2015
EditorsJianhua Ma, Ali Li, Huansheng Ning, Laurence T. Yang
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1819-1824
Number of pages6
ISBN (Electronic)9781467372114
DOIs
StatePublished - 20 Jul 2016
EventProceedings - 2015 IEEE 12th International Conference on Ubiquitous Intelligence and Computing, 2015 IEEE 12th International Conference on Advanced and Trusted Computing, 2015 IEEE 15th International Conference on Scalable Computing and Communications, 2015 IEEE International Conference on Cloud and Big Data Computing, 2015 IEEE International Conference on Internet of People and Associated Symposia/Workshops, UIC-ATC-ScalCom-CBDCom-IoP 2015 - Beijing, China
Duration: 10 Aug 201514 Aug 2015

Publication series

NameProceedings - 2015 IEEE 12th International Conference on Ubiquitous Intelligence and Computing, 2015 IEEE 12th International Conference on Advanced and Trusted Computing, 2015 IEEE 15th International Conference on Scalable Computing and Communications, 2015 IEEE International Conference on Cloud and Big Data Computing, 2015 IEEE International Conference on Internet of People and Associated Symposia/Workshops, UIC-ATC-ScalCom-CBDCom-IoP 2015

Conference

ConferenceProceedings - 2015 IEEE 12th International Conference on Ubiquitous Intelligence and Computing, 2015 IEEE 12th International Conference on Advanced and Trusted Computing, 2015 IEEE 15th International Conference on Scalable Computing and Communications, 2015 IEEE International Conference on Cloud and Big Data Computing, 2015 IEEE International Conference on Internet of People and Associated Symposia/Workshops, UIC-ATC-ScalCom-CBDCom-IoP 2015
Country/TerritoryChina
CityBeijing
Period10/08/1514/08/15

Keywords

  • Big Data
  • Divide-and-conquer
  • Efficiency
  • Machine ID

Fingerprint

Dive into the research topics of 'Big data analytical architecture using divide-and-conquer approach in machine-to-machine communication'. Together they form a unique fingerprint.

Cite this