Smart city platforms needs to integrate a large volume of data from different sources that must be properly managed to produce valuable outcomes for cities. To provide reusable data processing services for several client applications, such platforms needs to use Big Data tools and techniques in both historical and real-time data processing. In this sense, smart city platforms must provide generic services and interfaces that allow different applications to meet data processing requirements in a scalable, performatic fashion.
With the aim to let the InterSCity platform be more suitable for scenarios of larger data, we are developing Shock, a data processing service that provides extensible mechanisms to process data from sensors and services integrated to the InterSCity platform. Shock leverages the Kappa architecture handling the entire communication between applications, the InterSCity platform, Apache Spark, and Apache Kafka. Our final goal is to improve the development support of InterSCity by giving smart cities applications the option to configure data processing pipelines in a generic and extensible way.
