TY - GEN
T1 - Big data analytical architecture using divide-and-conquer approach in machine-to-machine communication
AU - Ahmad, Awais
AU - Paul, Anand
AU - Rathore, M. Mazhar
AU - Rho, Seungmin
N1 - Publisher Copyright:
© 2015 IEEE.
PY - 2016/7/20
Y1 - 2016/7/20
N2 - Machine-to-Machine (M2M) technology unremit-tingly motivates any time-place-objects connectivity of the devices in and around the world. Every day, a rapid growth of large M2M networks and digital storage technology, lead to a massive heterogeneous data depository, in which the M2M data are captured and warehoused in the diverse database frameworks as a magnitude of heterogeneous data sources. Hence, the M2M that handles Big Data might perform poorly or not according to the goals of their operator due to massive heterogeneous data sources may face various incompatibilities, such as data quality, processing and computational efficiency, analysis and feature extraction applications. Therefore, to address the aforementioned constraints, this paper presents a Big Data Analytical architecture based on Divide-and-Conquer approach. The designed system architecture exploits divide-and-conquer approach, where big data sets are first transformed into a several data blocks that can be quickly processed, then it classifies and reorganizes these data blocks from the same source. In addition, the data blocks are aggregated in a sequential manner based on a machine ID, and equally partitions the data using filtration and load balancing algorithms. The feasibility and efficiency of the proposed system architecture are implemented on Hadoop single node setup. The results show that the proposed system architecture efficiently extract various features (such as River) from the massive volume of satellite data.
AB - Machine-to-Machine (M2M) technology unremit-tingly motivates any time-place-objects connectivity of the devices in and around the world. Every day, a rapid growth of large M2M networks and digital storage technology, lead to a massive heterogeneous data depository, in which the M2M data are captured and warehoused in the diverse database frameworks as a magnitude of heterogeneous data sources. Hence, the M2M that handles Big Data might perform poorly or not according to the goals of their operator due to massive heterogeneous data sources may face various incompatibilities, such as data quality, processing and computational efficiency, analysis and feature extraction applications. Therefore, to address the aforementioned constraints, this paper presents a Big Data Analytical architecture based on Divide-and-Conquer approach. The designed system architecture exploits divide-and-conquer approach, where big data sets are first transformed into a several data blocks that can be quickly processed, then it classifies and reorganizes these data blocks from the same source. In addition, the data blocks are aggregated in a sequential manner based on a machine ID, and equally partitions the data using filtration and load balancing algorithms. The feasibility and efficiency of the proposed system architecture are implemented on Hadoop single node setup. The results show that the proposed system architecture efficiently extract various features (such as River) from the massive volume of satellite data.
KW - Big Data
KW - Divide-and-conquer
KW - Efficiency
KW - Machine ID
UR - http://www.scopus.com/inward/record.url?scp=84983432829&partnerID=8YFLogxK
U2 - 10.1109/UIC-ATC-ScalCom-CBDCom-IoP.2015.330
DO - 10.1109/UIC-ATC-ScalCom-CBDCom-IoP.2015.330
M3 - Conference contribution
AN - SCOPUS:84983432829
T3 - Proceedings - 2015 IEEE 12th International Conference on Ubiquitous Intelligence and Computing, 2015 IEEE 12th International Conference on Advanced and Trusted Computing, 2015 IEEE 15th International Conference on Scalable Computing and Communications, 2015 IEEE International Conference on Cloud and Big Data Computing, 2015 IEEE International Conference on Internet of People and Associated Symposia/Workshops, UIC-ATC-ScalCom-CBDCom-IoP 2015
SP - 1819
EP - 1824
BT - Proceedings - 2015 IEEE 12th International Conference on Ubiquitous Intelligence and Computing, 2015 IEEE 12th International Conference on Advanced and Trusted Computing, 2015 IEEE 15th International Conference on Scalable Computing and Communications, 2015 IEEE International Conference on Cloud and Big Data Computing, 2015 IEEE International Conference on Internet of People and Associated Symposia/Workshops, UIC-ATC-ScalCom-CBDCom-IoP 2015
A2 - Ma, Jianhua
A2 - Li, Ali
A2 - Ning, Huansheng
A2 - Yang, Laurence T.
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - Proceedings - 2015 IEEE 12th International Conference on Ubiquitous Intelligence and Computing, 2015 IEEE 12th International Conference on Advanced and Trusted Computing, 2015 IEEE 15th International Conference on Scalable Computing and Communications, 2015 IEEE International Conference on Cloud and Big Data Computing, 2015 IEEE International Conference on Internet of People and Associated Symposia/Workshops, UIC-ATC-ScalCom-CBDCom-IoP 2015
Y2 - 10 August 2015 through 14 August 2015
ER -