Data Query Processing Approach in Apache Spark

Abstract

In the current scenario of Big Data the Spark tool is widely being used to analyse, maintain, and transform the data into information. Spark has finished up the usage of Apache Hadoop. Apache Hadoop uses Map-Reduce frame for data processing while Spark has come with the in-processing feature that increases the speed of processing the large amount of data. Data Science is a current stream of development and technology, so Data Query Processing as part of Data Science is need to be evaluate. The main objective of the research is to provide valuable study about Data Science and Data Query Processing. This paper will help in analyse the efficiency and data transformation, data analysis and data manipulation features of Apache Spark that makes it differ from Apache Hadoop. The optimization of interactive queries is done based on two parameters, the first one is cost of evaluation and second one is rule-based optimization.