What is the spatiotemporal big data store?
The ArcGIS Data Store, available with ArcGIS Enterprise, has been enhanced to work with observational data by way of introducing the spatiotemporal big data store. Observation data can be moving objects, changing attributes of stationary sensors, or both. The spatiotemporal big data store enables archival of high volume observation data, sustains high velocity write throughput, and can run across multiple machines (nodes). Adding additional machines adds capacity, enabling you to store more data, implement longer retention policies of your data, and support higher data write throughput.
Requirements for using the spatiotemporal big data store
The spatiotemporal big data store requires ArcGIS Enterprise, for more information on ArcGIS Enterprise, see What is ArcGIS Enterprise?. For more information on base ArcGIS Enterprise deployments, see Base ArcGIS Enterprise deployment. For more information on adding functionality and capacity to your site with additional servers, specifically ArcGIS GeoEvent Server, see Additional server deployment.
When enabling the spatiotemporal big data store in ArcGIS Data Store, you will need to register the spatiotemporal big data store with your portal's hosting server. A data store connection to your portal must be registered with GeoEvent Server, in GeoEvent Manager, in order to view/write to the spatiotemporal big data store. The ArcGIS GIS Server manages multiple nodes of the spatiotemporal big data store to form a data ring.
Nota:
When archiving streaming data into a spatiotemporal big data store, depending on the size of the data, disk space can fill up quickly. Administrators need to monitor the spatiotemporal big data store logs to ensure machines have sufficient disk space. If the disk space on a machine exceeds its capacity new data will not be archived in the spatiotemporal big data store. For more information on ArcGIS Data Store and managing disk space, see Disk space requirements in the ArcGIS Data Store system requirements.
Sending data to a spatiotemporal big data store
GeoEvent Server provides output connectors for writing observational data to a spatiotemporal big data store.
- Add a Feature to an Spatiotemporal Big Data Store Output Connector allows you to add new features.
- Update a Feature in an Spatiotemporal Big Data Store Output Connector allows you to update existing features.
- Send Features to a Stream Service Output Connector allows the latest observations (based on TRACK_ID) broadcast by a stream service to be persisted (stored) if Store Latest is enabled.
Working with data in a spatiotemporal big data store
The high volume observation data stored in a spatiotemporal big data store can be accessed via a feature service and visualized using a map service. The map service provides dynamic rendering and aggregated data representation allowing observations to be visualized at various levels of detail specific to each user session. The map service's density aggregations are produced on-the-fly as you zoom in, so a web map's representation of your data updates dynamically. This is important for real-time GIS use cases because observations are constantly updating; the data representation displayed by a web map must reflect the data most recently received.
The high volume observation data stored in a spatiotemporal big data store can also be accessed via a feature service. Feature service queries to a spatiotemporal big data store are returned very quickly. You can query by any combination of id, time, space and/or attributes. The raw features in a spatiotemporal big data store can be visualized on the client side using the feature service. However, due to the high volume of features, you will likely only want to view raw features at large map scales.
As a best practice, you can configure scale dependent rendering to switch between a map service layer and the feature service layer for a better visualization experience. The map service would render the aggregated data at various levels of detail at the smaller scales. The feature service would turn on whenever you zoom in beyond to a specified map scale appropriate for the visualization of your raw features.
Getting started with the spatiotemporal big data store
To get started working with the spatiotemporal big data store, refer to the ArcGIS Data Store help and the Spatiotemporal Big Data Store tutorial available from GeoEvent Server tutorials.