A Spatial SQL Based on SparkSQL

The volume of spatial data increased tremendously, and growing attention has been paid to the research of distributed system for spatial data analysis. Spark, an in-memory distributed system which performs much better than Hadoop in speed and many other aspects, lacks spatial SQL query extensions. In this paper, we study the technology framework of Spark SQL, and implement the spatial query extension system tightly combined with the native Spark system. The extensions in the system include spatial types, spatial operators, spatial query optimizations and spatial data source formats. The spatial extension system on Spark retains the scalability and can be further extended with more query optimizations and data source formats. In this paper, the spatial data type system and spatial operator system follow OGC standards. In addition, the extension method is also a general method of query extensions on Spark SQL in other fields.