Managing spatiotemporal big data stores

What is the spatiotemporal big data store?

The ArcGIS Data Store, available with ArcGIS for Server, has been enhanced to work with observational data by way of introducing the spatiotemporal big data store. Observation data can be moving objects, changing attributes of stationary sensors, or both. The spatiotemporal big data store enables archival of high volume observation data, sustains high velocity write throughput, and can run across multiple machines (nodes). Adding additional machines adds capacity, enabling you to store more data, implement longer retention policies of your data, and support higher data write throughput. The spatiotemporal big data store is scalable. For example, thousands of machines supporting a spatiotemporal big data store ring is feasible.

Requirements for using the spatiotemporal big data store

The spatiotemporal big data store requires a Web GIS. This includes installing and configuring the following software, a three machine site is recommended.

• Portal for ArcGIS
• ArcGIS for Server (federated with Portal and configured as the hosting server)
• ArcGIS Web Adaptor (required for Portal installation and configuration)
• ArcGIS Data Store (provides a traditional relational RDBMS for the hosting server)
• ArcGIS Data Store (on a separate server, supports the Spatiotemporal Big Data Store)
• ArcGIS GeoEvent Extension for Server (installed on a separate server that could be federated)

Sending data to a spatiotemporal big data store

The Extension GeoEvent provides Output Connectors for writing observational data to a spatiotemporal big data store.

• Add a Feature to an ArcGIS Spatiotemporal Big Data Store allows you to add new features.
• Update a Feature in an ArcGIS Spatiotemporal Big Data Store allows you to update existing features.
• Send Features to a Stream Service allows the latest observations (based on TRACK_ID) broadcast by a stream service to be persisted (stored) if Store Latest is enabled.

Working with data in a spatiotemporal big data store

The high volume observation data stored in a spatiotemporal big data store can be accessed via a feature service and visualized using a map service. The map service provides dynamic rendering and aggregated data representation allowing observations to be visualized at various levels of detail specific to each user session. The map service's density aggregations are produced on-the-fly as you zoom in, so a web map's representation of your data updates dynamically. This is important for real-time GIS use cases because observations are constantly updating; the data representation displayed by a web map must reflect the data most recently received.

The high volume observation data stored in a spatiotemporal big data store can also be accessed via a feature service. Feature service queries to a spatiotemporal big data store are returned very quickly. You can query by any combination of id, time, space and/or attributes. The raw features in a spatiotemporal big data store can be visualized on the client side using the feature service. However, due to the high volume of features, you will likely only want to view raw features at large map scales.

As a best practice, you can configure scale dependent rendering to switch between a map service layer and the feature service layer for a better visualization experience. The map service would render the aggregated data at various levels of detail at the smaller scales. The feature service would turn on whenever you zoom in beyond to a specified map scale appropriate for the visualization of your raw features.

Getting started with the spatiotemporal big data store

To get started working with the spatiotemporal big data store, refer to the ArcGIS Data Store help and the tutorial available from Extension GeoEvent tutorials.