Create a data store

You can use the Data Store Configuration wizard to create most ArcGIS Data Store types and join them to the GIS Server site that acts as the ArcGIS Enterprise portal's hosting server.

If you do not have access to a web browser or you need to replace a machine in an existing tile cache data store, run the configuredatastore utility on the machine where you installed ArcGIS Data Store rather than using the Data Store Configuration wizard.

Considerations before creating a data store

Be aware of the following before you create a data store.

Disk space and memory

Consider the amount of disk space and memory needed on the machine where you will create a data store.

The following are examples of how a data store can use large amounts of disk space:

• The relational data store can store data for thousands of hosted feature layers, which can take up several gigabytes of disk space.
• If you archive data that is continually coming in from a stream layer or you track locations for numerous mobile workers, the spatiotemporal big data store can fill up quickly.
• Caches for 3D data can be multiple gigabytes or even terabytes, depending on the area included in the scene layer or 3D tiles layer and the complexity of the data.
• To take advantage of cached query responses to improve drawing performance for feature layers, the caches must persist for as long as possible in the object store. This means disk space on the object store can fill up quickly. To facilitate storage of query response caches for longer periods of time, use a single machine for the object store, and ensure that the single machine contains large amounts of free disk space. How much disk space you need depends on how many layers are enabled for caching, how many unique queries are made on those layers, and how long the queries persist. When used for query response caches, using a cluster of object stores has not been shown to provide as much improvement in drawing performance as using a single machine with large amounts of free disk space.

In all cases, it is important to monitor the data store machines and add disk space as necessary.

You also need to ensure that the machines have plenty of memory available. For example, the spatiotemporal big data store consumes approximately 50 percent of available memory on the machine when initially configured. The relational data store also preallocates memory to ensure there is enough to process data. High-volume traffic on hosted feature and scene layers can also consume a considerable amount of memory. Traversing and analyzing the relationships in knowledge graphs is a memory-intensive process.

For these reasons, it is recommended that you configure each type of ArcGIS Data Store on machines separate from each other and separate from ArcGIS Server sites. If you do configure more than one type of data store on the same machine, use robust machines with plenty of memory and disk space, but be aware that resource contention could render one or more of the data stores inoperable.

In all cases, monitor the data store logs for messages about disk space and memory. If a machine runs out of either resource, the data store will stop functioning.

See ArcGIS Data Store system requirements for information on the minimum disk space and memory needed to create each type of data store, and add to that the estimated size of the data or caches you intend to store in each and the processes that will be run on each machine.

Identify the IP address and host name for machines with multiple NIC cards or DNS entries

If you install ArcGIS Data Store on a machine that contains more than one network interface controller (NIC) card or multiple DNS entries (also called records), you must identify which IP address and host name the data store will use for network communication. Before you create a data store on the machine, determine which IP address can be used by other machines on your network to communicate with the ArcGIS Data Store machine, open the ArcGIS Data Store hostidentifier.properties file, and add the IP address to the file.

1. Determine which IP address to use. Open a command prompt on different machines on the same network and ping the ArcGIS Data Store machine.
2. Sign in to the machine where ArcGIS Data Store is installed and open hostidentifier.properties in a text editor.

   Use the same login account used when ArcGIS Data Store was installed.

   The file is installed in framework\etc of the ArcGIS Data Store installation directory. By default, that location is /home/ags/arcgis/datastore/framework/etc.

3. Remove the comment mark (#) from the hostidentifier property, and set the property to the IP address of the NIC card or DNS entry you want the data store to use for network communication.

   In this example, the IP address used to communicate with the machine is 12.34.56.789.

   hostidentifier=12.34.56.789

4. Save and close the hostidentifier.properties file.
5. Restart the ArcGIS Data Store service on that machine.

If necessary, follow the steps to configure other ArcGIS Enterprise software components on machines with multiple NIC cards or DNS entries:

• ArcGIS Server
• Portal for ArcGIS

The hosting server

An ArcGIS Enterprise deployment includes one GIS Server site to act as its hosting server. At minimum, the hosting server must be configured with a relational data store created through ArcGIS Data Store. This relational data store holds the data used by hosted feature layers created when you add files to your organization and publish feature layers and when you publish hosted feature layers from ArcGIS Pro. It also allows portal users to add comma separated values (CSV) files directly to a map.

Create a data store using the Data Store Configuration wizard

If you run the installation wizard, the Data Store Configuration wizard automatically opens in your default web browser. Alternatively, you can open the wizard from the computer's shortcut menu or type the URL in a browser. The Data Store Configuration wizard URL is in the format https://datastoremachine.example.com:2443/arcgis/datastore.

1. Specify the fully qualified domain name of a machine in the ArcGIS GIS Server site that will be used as the portal's hosting server.

   Provide the machine in the format gisserver.example.com. If you have a multiple-machine site, you can use the fully qualified domain name of any of the machines in the GIS Server site.

2. Type the username and password of the ArcGIS Server primary site administrator for that site and click Next.
3. Choose the type of data store to create on this machine and click Next.

   Although it's possible to create more than one type of data store on the same machine, Esri recommends that you do not do this because the data stores will compete for memory resources, which can lead to poor performance.

   If a data store type option is not enabled, it could be that type is not installed on the machine or that data store type has already been configured on this machine.

   The first three spatiotemporal big data store machines you configure using the Data Store Configuration wizard are created in both the coordinator and data roles. Additional spatiotemporal big data store machines you configure using the Data Store Configuration wizard are created in the data role. See Add machines to a data store for information about machine roles.

4. Type a location for the ArcGIS Data Store content directory.

   This directory contains the data store files and the relational data store backup directory (if you created a relational data store).

   Note:

   The directory path cannot contain a space when you specify an ArcGIS Data Store directory for a spatiotemporal big data store.

   You cannot use a UNC path when specifying the data store file directory.

   Do not use a file share for this directory.

   Keep data store files on the same machine where you install ArcGIS Data Store.

5. Click Next.
6. Review the information on the Configuration Summary dialog box. If it is correct, click Finish. If changes are needed, click Back and make corrections.

   The first relational data store you register with a GIS Server site is the primary machine. If you install ArcGIS Data Store on a second machine and create another relational data store registered to the same GIS Server site, it will be a standby machine. This is also true for the graph store.

   When the setup completes successfully, you have a data store machine.

   If there were errors with the setup, click Next on the Configuration Summary dialog box to reopen the Data Store Configuration wizard so you can provide different information. See Troubleshoot ArcGIS Data Store for causes and resolutions for common setup errors.

After you create a data store, configure a backup directory in a location that is separate from the ArcGIS Data Store installation. See Manage data store backups for more information.

Create a data store using the configuredatastore utility

You can use the configuredatastore utility to create all data store types implemented through ArcGIS Data Store. When you configure a data store, it is registered with the GIS Server site used for the ArcGIS Enterprise portal's hosting server, which you specify when you run the utility.

The configuredatastore utility is installed in <ArcGIS Data Store installation directory>/tools.

Note:

If you script the creation of multiple spatiotemporal big data store machines, one spatiotemporal big data store machine must be manually configured with the GIS Server site before you can script the creation of additional spatiotemporal big data store machines. Include wait times in your script to be sure the additional spatiotemporal big data store machines are not added at the same time.

1. Open a command shell.
2. Run the configuredatastore.sh utility and specify the fully qualified name of one of the machines in the GIS Server site with which you want to register the data store, the username and password of an ArcGIS Server administrator, the path to use for the ArcGIS Data Store data directory, and the type of data store to create.

   If you have a multiple-machine GIS Server site, you can use the fully qualified domain name of any of the machines in the site.

   If you specify a data store type that is not installed on the machine where you run the configuredatastore utility, you'll receive an error message.

   In the following examples, the name of a machine in the GIS Server site is myserver.example.com, the administrator username and password are siteadmin and T1n@sp, respectively, and the data directory is /home/dsuser/data.

   In this example, a relational data store is created:

   ./configuredatastore.sh myserver.example.com siteadmin T1n@sp /home/dsuser/data --stores relational

   In the following example, a spatiotemporal big data store is created. This machine will act as a coordinator.

   ./configuredatastore.sh myserver.example.com siteadmin T1n@sp /temporal/data --stores spatiotemporal --roles coord

   When you configure a tile cache data store on a machine, use the --mode operation to specify the deployment mode.

   For tile cache data stores, mode options are primary-standby (the default) or cluster mode. If you do not specify the --mode operation, the tile cache data store is created in primary-standby mode.

   In this example, a tile cache data store machine is configured to become part of an existing tile cache data store running in cluster mode.

   ./configuredatastore.sh myserver.example.com siteadmin T1n@sp /home/dsuser/scenedata --stores tileCache --mode cluster

   The --mode operation is also used to specify the deployment mode of an object store. For object stores, the default deployment is a single-machine deployment (in primary-standby mode). If you intend to use multiple machines in the object store, specify --mode cluster when creating the object store.

   In this example, a cluster of three machines are deployed as an object store. Run the same utility from each of the three ArcGIS Data Store object store machines.

   ./configuredatastore.sh myserver.example.com siteadmin T1n@sp /home/dsuser/ftcacheds 
   --stores object --mode cluster

   Note:

   You cannot change an object store's or mode after you create it.

   The directory path cannot contain a space when you specify an ArcGIS Data Store directory for a tile cache data store, spatiotemporal big data store, or object store.

   To create more than one type of data store on the same machine using the same data directory, specify the values separated by a comma. However, Esri recommends that you do not put more than one data store on the same machine because the data stores compete for resources such as memory, which can negatively affect performance.

Next steps

If you require more than one type of data store, create additional data store types. While it's possible to create another type of data store on the same machine, instead install and configure additional data store types on separate machines.

If you require more than one machine for each type of data store, add a machine or machines to it.

Ensure that the GIS Server site with which you registered the data store has been configured as the ArcGIS Enterprise portal's hosting server.

Also, configure a secure location for data store backups.