Upgrade a spatiotemporal big data store on AWS

How you upgrade depends on how and when you created the ArcGIS Data Store spatiotemporal big data store on Amazon Web Services (AWS).

• Any spatiotemporal big data store on AWS can be upgraded using a manual upgrade procedure. If you created the data store before 10.8, you must use the manual procedure to upgrade.
• If you created the spatiotemporal big data store using a 10.8 or 10.8.1 Esri Amazon Web Services CloudFormation template, you can upgrade using Chef scripts.
• If you created the spatiotemporal big data store using a 10.9, 10.9.1, 11.0, 11.1, 11.2, or 11.3 Esri CloudFormation template, you can use the corresponding 11.4 CloudFormation template to upgrade it.
• If you created the spatiotemporal big data store using 10.9, 10.9.1, 11.0, 11.1, 11.2, or 11.3 ArcGIS Enterprise Cloud Builder for Amazon Web Services, you can use 11.4 ArcGIS Enterprise Cloud Builder for AWS to upgrade it.

Before you upgrade

Regardless of the upgrade method you use, review the following information before you upgrade your deployment:

• Use the backupdatastore utility installed with ArcGIS Data Store to create a backup of the spatiotemporal big data store. See Manage data store backups for instructions.
• You must upgrade the ArcGIS GIS Server site (the hosting server) with which the spatiotemporal big data store is registered before you upgrade the spatiotemporal big data store. If the spatiotemporal big data store is registered to a single-machine ArcGIS Enterprise deployment or highly available ArcGIS Enterprise deployment on two machines, you must upgrade the ArcGIS Enterprise deployment before upgrading the spatiotemporal big data store.
• If you use the manual procedures or Chef scripts to upgrade, you must remove the spatiotemporal big data store Amazon Elastic Compute Cloud (EC2) instances from the AWS Auto Scaling group (if it is in one) before you can upgrade the data store. Once upgraded, you cannot put the data store instances back in the Auto Scaling group. Therefore, you must manually add instances to or delete instances from the data store when needed after upgrading, and you cannot use the Lambda functions created using Amazon Web Services CloudFormation templates provided by Esri to stop and start these instances.

  If you don't remove the instances or if you put the instances back in the Auto Scaling group after you upgrade, instances added to the spatiotemporal big data store by the Auto Scaling group will be running the old version of the software. You cannot have multiple versions of ArcGIS Data Store, so this can corrupt your deployment.

Upgrade using manual upgrade procedures

The manual upgrade procedure for spatiotemporal big data stores on AWS is similar to upgrading one on-premises. The difference on AWS is that, if the spatiotemporal big data store Amazon Elastic Compute Cloud instances are deployed in an AWS Auto Scaling group, you must remove them from the Auto Scaling group before you upgrade.

Follow these steps to remove all spatiotemporal big data store EC2 instances from their Auto Scaling group before you upgrade ArcGIS Data Store.

1. Sign in to AWS Management Console and find the Auto Scaling group associated with your spatiotemporal big data store stack.

   The Auto Scaling group includes the name of the CloudFormation stack. You can also open instance management for the group and confirm the EC2 instances in the group participate in your spatiotemporal big data store.

2. Change the minimum capacity and maximum capacity properties of the stack's Auto Scaling group to 0.
3. Remove all the EC2 instances from the Auto Scaling group.
4. Change the desired capacity property of the Auto Scaling group to 0.

Once you remove all EC2 instances, delete the Auto Scaling group to prevent anyone from adding the instances back after the data store is upgraded.

Now you can proceed with upgrading the spatiotemporal big data store. Prerequisites and instructions for upgrading can be found in the ArcGIS Data Store installation guide.

• ArcGIS Data Store on Microsoft Windows installation guide
• ArcGIS Data Store on Linux installation guide

Upgrade using Chef scripts

If you created the spatiotemporal big data store using a 10.8 or 10.8.1 Esri Amazon Web Services CloudFormation template, you can upgrade using Chef scripts.

Upgrade steps are different for deployments on Microsoft Windows and Ubuntu. Follow the instructions applicable to the operating system of your deployment.

Upgrade a spatiotemporal big data store on Windows

Before running the upgrade script, remove EC2 instances from the AWS Auto Scaling group. Next, download the Chef client to all EC2 instances and run scripts to install ArcGIS Data Store. Finally, run the upgrade script on one of the EC2 instances.

Adjust the Auto Scaling group settings

Follow these steps to remove all EC2 instances from the Auto Scaling group.

1. Sign in to AWS Management Console and find the Auto Scaling group associated with your spatiotemporal big data store stack.

   The Auto Scaling group includes the name of the CloudFormation stack. You can also open instance management for the group and confirm the EC2 instances in the group participate in your spatiotemporal big data store.

2. Change the minimum capacity and maximum capacity properties of the stack's Auto Scaling group to 0.
3. Remove all the EC2 instances from the Auto Scaling group.
4. Change the desired capacity property of the Auto Scaling group to 0.

Once you remove all EC2 instances, delete the Auto Scaling group to prevent anyone from adding the instances back after the data store is upgraded.

Download Chef client and ArcGIS Data Store to all but one EC2 instance

Log in to all but one of the EC2 instances in the spatiotemporal big data store to install the Chef client and install the latest version of ArcGIS Data Store.

1. Make a remote desktop connection to the first EC2 instance in the spatiotemporal big data store.
2. Download and install a supported Chef or Cinc client.

   See the ArcGIS Cookbook compatibility matrix for supported versions.

3. Download the ArcGIS Data Store 11.4 installation file (ArcGIS_DataStore_Windows_114_192943.exe) from your account in My Esri and upload the files to a folder on the EC2 instance.
4. Download the ArcGIS Cookbook version 5.0.0 (arcgis-5.0.0-cookbooks.zip) to the EC2 instance, unzip the file, and place the contents of the unzipped file in the C:\chef folder.
5. Delete the following folders:

   • C:\chef\nodes
   • C:\Users\Administrator\.chef\local-mode-cache

6. Create a copy of your original C:\chef\node.json file and rename it to upgr-bds-node-step2.json.
7. Open the upgr-bds-node-step2.json file in a text editor and modify the following parameters:

   • Set the ArcGIS version to 11.4.

     "arcgis" : {
         "version": "11.4",

   • Set the ArcGIS archive repository to the folder where you placed the software file on the EC2 instance.

     "arcgis" : {
         "repository": {
     	  "archives": "<drive>\\<folder>"

   • Rename the private_url parameter url.

     "arcgis" : {
         "server": {
     	  "url": 

   • Modify the run list to use the following recipe.

     "recipe[arcgis-enterprise::install_datastore]"

8. Save and close the upgr-bds-node-step2.json file.
9. Open a command line window as an administrator, change the directory to C:\chef, and run the following command: chef-client -z -j c:\chef\upgr-bds-node-step2.json.
10. If there are more than two EC2 instances in the spatiotemporal big data store, make a remote desktop connection to the next EC2 instance and repeat steps 2 through 9. Once there is only one instance left to upgrade, proceed to the next section.

Download Chef client and ArcGIS Data Store to remaining EC2 and run upgrade script

Now you can install the Chef client and the latest version of ArcGIS Data Store to the final spatiotemporal big data store instance and run the upgrade from that instance.

1. Make a remote desktop connection to the remaining EC2 instance in the spatiotemporal big data store that has not been upgraded.
2. Download and install a supported Chef or Cinc client.

   See the ArcGIS Cookbook compatibility matrix for supported versions.

3. Download the ArcGIS Data Store 11.4 installation file (ArcGIS_DataStore_Windows_114_192943.exe) from your account in My Esri and upload the files to a folder on the EC2 instance.
4. Download the ArcGIS Cookbook version 5.0.0 (arcgis-5.0.0-cookbooks.zip) to the EC2 instance, unzip the file, and place the contents of the unzipped file in the C:\chef folder.
5. Delete the following folders:

   • C:\chef\nodes
   • C:\Users\Administrator\.chef\local-mode-cache

6. Create a copy of your original C:\chef\node.json file and rename it to upgr-bds-node-step3.json.
7. Open the upgr-bds-node-step3.json file in a text editor and modify the following parameters:

   • Set the ArcGIS version to 11.4.

     "arcgis" : {
         "version": "11.4",

   • Set the ArcGIS archive repository to the folder where you placed the software file on the EC2 instance.

     "arcgis" : {
         "repository": {
     	  "archives": "<drive>\\<folder>"

   • Rename the private_url parameter url.

     "arcgis" : {
         "server": {
     	   "url": 

   • Modify the run list to use the following recipe:

     "recipe[arcgis-enterprise::datastore]"

8. Save and close the upgr-bds-node-step3.json file.
9. Open a command line window as an administrator, change the directory to C:\chef, and run the following command: chef-client -z -j c:\chef\upgr-bds-node-step3.json.

The spatiotemporal big data store is upgraded.

Upgrade a spatiotemporal big data store on Ubuntu

Before running the upgrade script, remove EC2 instances from the AWS Auto Scaling group. Next, download the Chef client to all EC2 instances and run scripts to install ArcGIS Data Store. Finally, run the upgrade script on one of the EC2 instances.

Adjust the Auto Scaling group settings

Follow these steps to remove all EC2 instances from the Auto Scaling group.

1. Sign in to AWS Management Console and find the Auto Scaling group associated with your spatiotemporal big data store stack.

   The Auto Scaling group includes the name of the CloudFormation stack. You can also open instance management for the group and confirm the EC2 instances in the group participate in your spatiotemporal big data store.

2. Change the minimum capacity and maximum capacity properties of the stack's Auto Scaling group to 0.
3. Remove all the EC2 instances from the Auto Scaling group.
4. Change the desired capacity property of the Auto Scaling group to 0.

Once you remove all EC2 instances, delete the Auto Scaling group to prevent anyone from adding the instances back after the data store is upgraded.

Download Chef client and ArcGIS Data Store to all but one EC2 instance

Log in to all but one of the EC2 instances in the spatiotemporal big data store to install the Chef client and install the latest version of ArcGIS Data Store.

1. As the ubuntu user, make a secure shell protocol (SSH) connection into the first EC2 instance in the spatiotemporal big data store.
2. Use one of the following commands to download and install a supported Chef or Cinc client.

   See the ArcGIS Cookbook compatibility matrix for supported versions. Replace the version number text in the following statements with the client version you install, for example 18.30.

   • Chef client: sudo curl -L https://omnitruck.chef.io/install.sh | sudo bash -s -- -v <version_number>
   • Cinc client: sudo curl -L https://omnitruck.cinc.sh/install.sh | sudo bash -s -- -v <version_number>
3. Download the ArcGIS Data Store 11.4 installation file (ArcGIS_DataStore_Linux_114_192981.tar.gz) from your account in My Esri and upload the files to a folder on the EC2 instance.
4. Download the ArcGIS Cookbook version 5.0.0 (arcgis-5.0.0-cookbooks.tar.gz) to the EC2 instance, untar and unzip the file, and place the files contents in the /var/chef directory.
5. Delete the /var/chef/nodes directory.
6. Create a copy of your original /etc/chef/node.json file and rename it to upgr-bds-node-step2.json.
7. Open the upgr-bds-node-step2.json file in a text editor and modify the following parameters:

   • Set the ArcGIS version to 11.4.

     "arcgis" : {
         "version": "11.4",

   • Set the ArcGIS archive repository to the folder where you placed the software file on the EC2 instance.

     "arcgis" : {
         "repository": {
     	  "archives": "<drive>\\<folder>"

   • Rename the private_url parameter url.

     "arcgis" : {
         "server": {
     	  "url": 

   • Modify the run list to use the following recipes:

     "recipe[arcgis-enterprise::install_datastore]"
     "recipe[arcgis-enterprise::start_datastore]"

8. Save and close the upgr-bds-node-step2.json file.
9. As the ubuntu user, change the directory to /var/chef, and run the following command: sudo chef-client -z -j /etc/chef/upgr-bds-node-step2.json.
10. If there are more than two EC2 instances in the spatiotemporal big data store, SSH as the ubuntu user to the next EC2 instance and repeat steps 2 through 9. Once there is only one instance left to upgrade, proceed to the next section.

Download Chef client to remaining EC2 and run upgrade script

Now you can install the Chef client and the latest version of ArcGIS Data Store to the final spatiotemporal big data store instance and run the upgrade from that instance.

1. As the ubuntu user, SSH into the last EC2 instance in the spatiotemporal big data store.
2. Use one of the following commands to download and install a supported Chef or Cinc client.

   See the ArcGIS Cookbook compatibility matrix for supported versions. Replace the version number text in the following statements with the client version you install, for example 18.30.

   • Chef client: sudo curl -L https://omnitruck.chef.io/install.sh | sudo bash -s -- -v <version_number>
   • Cinc client: sudo curl -L https://omnitruck.cinc.sh/install.sh | sudo bash -s -- -v <version_number>
3. Download the ArcGIS Data Store 11.4 installation file (ArcGIS_DataStore_Linux_114_192981.tar.gz) from your account in My Esri and upload the files to a folder on the EC2 instance.
4. Download the ArcGIS Cookbook version 5.0.0 (arcgis-5.0.0-cookbooks.tar.gz) to the EC2 instance, untar and unzip the file, and place the files contents in the /var/chef directory.
5. Delete the /var/chef/nodes directory.
6. Create a copy of your original /etc/chef/node.json file and rename it to upgr-bds-node-step3.json.
7. Open the upgr-bds-node-step3.json file in a text editor and modify the following parameters:

   • Set the ArcGIS version to 11.4.

     "arcgis" : {
         "version": "11.4",

   • Set the ArcGIS archive repository to the folder where you placed the software file on the EC2 instance.

     "arcgis" : {
         "repository": {
     	  "archives": "<drive>\\<folder>"

   • Rename the private_url parameter url.

     "arcgis" : {
         "server": {
     	  "url": 

   • Modify the run list to use the following recipe:

     "recipe[arcgis-enterprise::datastore]"

8. Save and close the upgr-bds-node-step3.json file.
9. As the ubuntu user, change the directory to /var/chef, and run the following command: sudo chef-client -z -j /etc/chef/upgr-bds-node-step3.json.

The spatiotemporal big data store is upgraded.

Upgrade using a CloudFormation template

If you created the spatiotemporal big data store using 10.9, 10.9.1, 11.0, 11.1, 11.2, or 11.3 Amazon Web Services (AWS) CloudFormation templates provided by Esri, you can use the corresponding 11.4 CloudFormation upgrade template to upgrade it.

For spatiotemporal big data stores, there is a second CloudFormation template to use to upgrades, as indicated below.

The following table lists corresponding deployment and upgrade templates and provides links to instructions for using the upgrade templates.

10.9, 10.9.1, 11.0, 11.1, 11.2, or 11.3 deployment template	11.4 upgrade template	Instructions to use upgrade template
arcgis-datastore-singlemachine.template.json	arcgis-datastore-singlemachine-upgrade.template.json	Upgrade a single machine ArcGIS Data Store using CloudFormation
arcgis-datastore-cluster.template.json	arcgis-datastore-cluster-upgrade.template.json	Upgrade a clustered ArcGIS Data Store deployment using CloudFormation
arcgis-datastore-cluster-enhanced.template.json (11.3)	arcgis-datastore-cluster-role-upgrade.template.json	Upgrade a clustered spatiotemporal big data store with separate roles using CloudFormation