Create a data store

You can use the Data Store Configuration wizard to create most types of data stores and join them to the GIS Server site that acts as the ArcGIS Enterprise portal's hosting server.

If you do not have access to a web browser or you require any of the following, run the configuredatastore utility on the machine where you installed ArcGIS Data Store rather than using the Data Store Configuration wizard:

• You need to deploy an object store in cluster mode.
• You need to deploy a tile cache data store in cluster mode.

Considerations before creating a data store

Be aware of the following before you create a data store.

Disk space and memory

Consider the amount of disk space and memory needed on the machine where you will create a data store.

The following are examples of how a data store can use large amounts of disk space:

• Relational data stores can store data for thousands of hosted feature layers, which can take up several gigabytes of disk space.
• Caches for 3D data can be multiple gigabytes or even terabytes, depending on the area included in the scene layer and the complexity of the data.
• If you archive data that is continually coming in from a stream layer or you track locations for numerous field workers, the spatiotemporal big data store can fill up quickly.
• To take advantage of cached query responses to improve drawing performance for feature layers, the caches must persist for as long as possible in the object store. This means disk space on the object store can fill up quickly. To facilitate storage of query response caches for longer periods of time, use a single machine for the object store, and ensure that the single machine contains large amounts of free disk space. How much disk space you need depends on how many layers are enabled for caching, how many unique queries are made on those layers, and how long the queries persist. (Using a cluster of object stores has not been shown to provide as much improvement in drawing performance as using a single machine with large amounts of free disk space.)

In all cases, it is important to monitor the data store machines and add disk space as necessary.

You also need to ensure that the machines have plenty of memory available. For example, spatiotemporal big data stores consume approximately 50 percent of available memory on the machine when initially configured. Relational data stores also preallocate memory to ensure there is enough to process data. High-volume traffic on hosted feature and scene layers can also consume a considerable amount of memory. Knowledge graphs are complex layers that model relationships among entities. Traversing and analyzing these relationships is a memory-intensive process.

For these reasons, Esri recommends that you configure spatiotemporal big data stores, object stores, and graph stores on machines separate from other data stores. Esri also recommends that, whenever possible, you configure the relational data store and tile cache data store on machines separate from one another. If you do configure more than one type of data store on the same machine, use robust machines with plenty of memory and disk space.

In all cases, monitor the data store logs for messages about disk space and memory. If your machine runs out of either resource, your data store will stop functioning.

See ArcGIS Data Store system requirements for information on the minimum disk space and memory needed to create each type of data store, and add to that the estimated size of the data or caches you intend to store in each and the processes that will be run on each machine.

Multiple NIC cards or DNS entries

If you install ArcGIS Data Store on a machine that contains more than one network interface controller (NIC) card or multiple DNS entries (also called records), identify which IP address and host name the data store will use for network communication. Before you create a data store on the machine, determine which IP address can be used by other machines on your network to communicate with the ArcGIS Data Store machine, open the ArcGIS Data Store hostidentifier.properties file, and add the IP address to the file.

1. Determine which IP address to use. Open a command prompt on different machines on the same network and ping the ArcGIS Data Store machine.
2. Sign in to the machine where ArcGIS Data Store is installed and open hostidentifier.properties in a text editor. Use the same login account used when ArcGIS Data Store was installed.

   The file is installed in framework\etc of the ArcGIS Data Store installation directory. By default, that location is /home/ags/arcgis/datastore/framework/etc.

3. Remove the comment mark (#) from the hostidentifier property, and set the property to the IP address of the NIC card or DNS entry you want the data store to use for network communication.

   In this example, the IP address used to communicate with the machine is 12.34.56.789.

   hostidentifier=12.34.56.789

4. Save and close the hostidentifier.properties file.
5. Restart the ArcGIS Data Store service on that machine.

If necessary, follow the steps to configure other ArcGIS Enterprise software components on machines with multiple NIC cards or DNS entries:

• ArcGIS Server
• Portal for ArcGIS

The hosting server

An ArcGIS Enterprise deployment includes one GIS Server site to act as its hosting server. At minimum, the hosting server must be configured with a relational data store created through ArcGIS Data Store. This relational data store holds the data used by hosted feature layers created when you add files to your organization and publish feature layers and when you publish hosted feature layers from ArcGIS Pro. It also allows portal users to add comma separated values (CSV) files directly to a map.

If the GIS Server site you intend to use as the hosting server has an existing enterprise geodatabase registered as its managed database, follow the instructions in Change the managed database to an ArcGIS Data Store relational data store to reconfigure the hosting server with a relational data store.

Create a data store using the Data Store Configuration wizard

If you run the installation wizard, the Data Store Configuration wizard automatically opens in your default web browser. Alternatively, you can open the wizard from the computer's shortcut menu or type the URL in a browser. The Data Store Configuration wizard URL is in the format https://datastoremachine.domain.com:2443/arcgis/datastore.

1. Specify the fully qualified domain name of a machine in the ArcGIS GIS Server site that will be used as the portal's hosting server.

   Provide the machine in the format gisserver.domain.com. If you have a multiple-machine site, you can use the fully qualified domain name of any of the machines in the GIS Server site.

2. Type the user name and password of an ArcGIS Server administrator for that site and click Next.

   The user must be a built-in account.

3. Choose the type of data store to create on this machine and click Next.

   Although it's possible to create more than one type of data store on the same machine, Esri recommends that you do not do this because the data stores will compete for memory resources, which can lead to poor performance.

   If a data store type option is not enabled, it could be that type is not installed on the machine or that data store type has already been configured on this machine.

4. Type a location for the ArcGIS Data Store directory.

   This directory contains the data store files and the relational data store backup directory (if you created a relational data store).

   Note:

   The directory path cannot contain a space when you specify an ArcGIS Data Store directory for a tile cache or spatiotemporal big data store.

   You cannot use a UNC path when specifying the data store file directory.

   Do not use a file share for this directory.

   Keep data store files on the same machine where you install ArcGIS Data Store.

5. Click Next.
6. Review the information on the Configuration Summary dialog box. If it is correct, click Finish. If changes are needed, click Back and make corrections.

   The first relational data store you register with a GIS Server site is the primary machine. If you install ArcGIS Data Store on a second machine and create another relational data store registered to the same GIS Server site, it will be a standby machine. This is also true for tile cache data stores running in primary-standby mode.

   When the setup completes successfully, you have a data store machine.

   If there were errors with the setup, click Next on the Configuration Summary dialog box to reopen the Data Store Configuration wizard so you can provide different information. See Troubleshoot ArcGIS Data Store for causes and resolutions for common setup errors.

After you create a data store, configure backup directory for in a location that is separate from the ArcGIS Data Store installation. See Manage data store backups for more information.

Create a data store using the configuredatastore utility

You can use the configuredatastore utility to create all data store types implemented through ArcGIS Data Store. When you configure a data store, it is registered with the GIS Server site used for the ArcGIS Enterprise portal's hosting server. The configuredatastore utility is installed in <ArcGIS Data Store installation directory>\arcgis\datastore\tools.

Note:

If you script the creation of multiple spatiotemporal big data store machines, one spatiotemporal big data store machine must be manually configured with the GIS Server site before you can script the creation of additional spatiotemporal big data store machines. Include wait times in your script to be sure the additional spatiotemporal big data store machines are not added at the same time.

1. Open a command shell.
2. Run the configuredatastore.sh file and specify the fully qualified name of one of the machines in the GIS Server site with which you want to register the data store, the user name and password of an ArcGIS Server administrator, the path to use for the ArcGIS Data Store data directory, and the type of data store you want to create.

   If you have a multiple-machine GIS Server site, you can use the fully qualified domain name of any of the machines in the site.

   If you specify a data store type that is not installed on the machine where you run the configuredatastore utility, you'll receive an error message.

   In the following examples, the name of a machine in the GIS Server site is myserver.domain.com, the administrator user name and password are siteadmin and T1n@sp, and the data directory is /home/dsuser/data.

   In this example, a relational data store is created:

   ./configuredatastore.sh myserver.domain.com siteadmin T1n@sp /home/dsuser/data --stores relational

   In this example, a spatiotemporal big data store is created:

   ./configuredatastore.sh myserver.domain.com siteadmin T1n@sp /temporal/data --stores spatiotemporal

   When you create a tile cache data store, you can use the --mode operation to specify the deployment mode.

   For tile cache data stores, mode options are primary-standby (the default) or cluster mode. If you do not specify the --mode operation, the tile cache data store is created in primary-standby mode.

   In this example, a tile cache data store is created in cluster mode on a separate machine but is registered with the same hosting server as the relational data store created in the last example. Run the utility on all tile cache data store machines to be added to the cluster.

   ./configuredatastore.sh myserver.domain.com siteadmin T1n@sp /home/dsuser/scenedata --stores tileCache --mode cluster

   The --mode operation is also used to specify the deployment mode of an object store. For object stores, the default deployment is a single-machine deployment (in primary-standby mode). If you intend to use multiple machines in the object store, specify --mode cluster when creating the object store.

   In this example, a cluster of three machines are deployed as an object store. Run the same utility from each of the three ArcGIS Data Store object store machines.

   ./configuredatastore.sh myserver.domain.com siteadmin T1n@sp /home/dsuser/ftcacheds 
   --stores object --mode cluster

   Note:

   You cannot change an object store's mode after you create it.

   The directory path cannot contain a space when you specify an ArcGIS Data Store directory for a tile cache data store, spatiotemporal big data store, or object store.

   To create more than one type of data store on the same machine using the same data directory, specify the values separated by a comma. For example, you can create a relational and tile cache data store on the same machine by specifying relational,tileCache with the --stores operation. However, Esri recommends that you do not put more than one data store on the same machine because the data stores compete for resources such as memory, which can negatively affect performance.

Next steps

If you require more than one type of data store, create additional data store types. While it's possible to create another type of data store on the same machine, instead install and configure additional data store types on separate machines.

If you require more than one machine for each type of data store, add a machine to your relational data store or tile cache data store running in primary-standby mode, or add at least two machines to your object store, spatiotemporal big data store, or tile cache data store running in cluster mode.

Ensure that the GIS Server site with which you registered your data store has been configured as your portal's hosting server.

Also, configure a secure location for data store backups.