There are many methods you can use to deploy ArcGIS Enterprise and ArcGIS Server on Amazon Web Services (AWS). This topic describes ways to troubleshoot problems you may encounter with each deployment option.
Troubleshoot ArcGIS Enterprise Cloud Builder for Amazon Web Services
There are two folders under the Logs folder in the location where you extracted ArcGIS Enterprise Cloud Builder for Amazon Web Services and its associated files.
The Logs folder contains the following two subfolders:
- Log—This folder contains console logs, which are created every time you run the utility. The Log folder also contains CloudFormation event logs for each stack you create as part of deployment every time you run the CREATE command with the ArcGIS Enterprise Cloud Builder CLI for AWS utility or add a deployment in the ArcGIS Enterprise Cloud Builder for Amazon Web Services app.
- ErrorLog—This folder contains log files to record any errors that occur when you run the ArcGIS Enterprise Cloud Builder CLI for AWS utility or ArcGIS Enterprise Cloud Builder for Amazon Web Services app. Read these error logs for effective troubleshooting.
See the following two sections for information on specific errors you could encounter when running the Cloud Builder app or ArcGIS Enterprise Cloud Builder CLI for AWS utility. Both the app and utility launch CloudFormation stacks, so you should also read the section on troubleshooting CloudFormation stack creation to help identify and correct deployment problems.
Errors encountered when running the ArcGIS Enterprise Cloud Builder for Amazon Web Services app
The ArcGIS Enterprise Cloud Builder for Amazon Web Services app validates many of the values you provide as you use the app. Additional information is available in the log file locations mentioned in the previous section.
I receive a message that my domain name is invalid.
There are several reasons that Cloud Builder cannot validate your domain name, but the most likely cause is related to a missing or improper mapping of the domain name in your domain name server (DNS). For example, the domain name may not be mapped inside your DNS, or it may be mapped improperly.
Ensure that the domain name you provided resolves to a CNAME or A record in your DNS. To check whether the domain name you provided in Cloud Builder is properly mapped, use the nslookup command in a Microsoft Windows command prompt on the same machine where you are running the Cloud Builder app.
If you think your domain name mapping is correct, try clearing the local DNS cache on the machine where you are running Cloud Builder, and then validate the domain name again in the Cloud Builder app.
If your domain name still doesn't validate, contact your IT administrator for further support.
My deployment failed to create.
If the Cloud Builder app cannot create your deployment, error messages are displayed on the Cloud Builder job completion page. You can see further details on the errors by clicking View Process Log and View Error Log on the job completion page.
If you see a generic error such as Wait condition received failed message: 'Chef run failed. See 'c:\\chef\chef-run.log' for details', check the Amazon CloudWatch logs by clicking the log URLs in the View AWS CloudWatch Logs section of the Cloud Builder summary page. This opens the appropriate CloudWatch log in your default web browser, where you can view logs for each Amazon Elastic Compute Cloud (EC2) instance in your deployment and figure out the reason for the failure.
Once you figure out what caused the failure, you can click Back in the Cloud Builder app to fix the required parameter and retry the deployment. Alternatively, you can delete the failed deployment and create a new one.
The ArcGIS Enterprise Cloud Builder for Amazon Web Services app continuously crashes and returns an error indicating ArcGIS Enterprise Cloud Builder for AWS has stopped working
If the ArcGIS Enterprise Cloud Builder for Amazon Web Services app crashes and returns the following error message: ArcGIS Enterprise Cloud Builder for AWS has stopped working. A problem caused the program to stop working correctly. Windows will close the program and notify you if a solution is available., your machine is likely missing the required Microsoft Visual C++ Redistributable package. See Cloud Builder prerequisites for more information and a link to download the package.
Where can I find the CloudWatch logs for deployments I create using the ArcGIS Enterprise Cloud Builder for Amazon Web Services app?
If the Cloud Builder app cannot create your deployment, it returns error messages on the job completion page. To see further details on the errors, click the View Process Log and View Error Log links on the job completion page.
Errors encountered when using the ArcGIS Enterprise Cloud Builder CLI for AWS utility
The following are some common messages or problems you may encounter when using the ArcGIS Enterprise Cloud Builder CLI for AWS utility and suggestions on how to correct them.
I receive a permissions error.
The user who runs the ArcGIS Enterprise Cloud Builder CLI for AWS utility must have recursive write access to the Logs and Output folders. Ensure these permissions have been granted.
I receive the message Cannot find file <file_name>.
Beginning with 10.8.1, you must place the JSON configuration files in the Configuration folder or one of its subfolders in your working directory; do not place the JSON configuration files in the Configuration folder available in the installation directory. If your configuration files were not in the Configuration folder in your working directory, move the files here and run the utility again.
For 10.8 and earlier, you must place the JSON configuration files in the Configuration folder or one of its subfolders in the installation directory. If your configuration files were not in this location, move them to the Configuration folder and run the utility again.
I receive the message Invalid deployment configuration file extension. It must be '.json'.
The configuration file you use must have a .json extension. If it does not, rename it to include this file extension and run the utility again.
I receive the message Invalid JSON format for file <file_name>.
Ensure the configuration file contains valid JSON formatting.
Tip:
You can use a JSON validator such as JSONLint to validate formatting.
I receive the message Failed to access AWS account with provided credentials.
Make sure you provided valid AWS credentials in the JSON configuration file so that the utility can connect to AWS. If credentials are valid, be sure your account is accessible from the machine where you are running the ArcGIS Enterprise Cloud Builder CLI for AWS utility.
I receive an error saying access is denied when I run the ArcGIS Enterprise Cloud Builder CLI for AWS utility with the PREP command.
If you are using an existing Amazon Simple Storage Service (S3) bucket for your deployment files, be sure you have appropriate permissions to access and write to the bucket.
When I open the output file after running the ArcGIS Enterprise Cloud Builder CLI for AWS utility with the CREATE command, I do not see all the components I expected.
The output file created when you use the CREATE command contains headings for each component the utility created on AWS. These headings correspond to the nodes you included in your configuration JSON file. The nodes in the configuration file tell the utility which components to create. If you leave out a node from the configuration file, the utility will not create that component.
Each node you include in your configuration file must contain at least one parameter. Even if you set all the parameters under the "Default" node, you must include a node for every part of the deployment you want created and that node must contain at least one parameter and value. If a node is present in the configuration file but does not include a parameter, the utility skips that node. For example, you cannot set parameters for an ArcGIS GIS Server site in the "Default" node and place an empty "Server" node in the configuration file. If you do, the utility will not create the ArcGIS GIS Server site.
Check the configuration file to ensure you have added all required nodes and at least one parameter for each component of the deployment. See ArcGIS Enterprise Cloud Builder CLI for AWS parameters for a list of required parameters for each node in the configuration file.
When I run the ArcGIS Enterprise Cloud Builder CLI for AWS utility, the command prompt appears to stop running.
If you find the ArcGIS Enterprise Cloud Builder CLI for AWS utility or any utility you run in a Microsoft Windows command window stops running, disable QuickEdit Mode for the command window.
Troubleshoot AWS CloudFormation stack creation
ArcGIS Enterprise on Amazon Web Services deployment tools create AWS CloudFormation stacks. Use the AWS CloudFormation console to monitor the status of your AWS CloudFormation stack and detect if stack creation fails. Sign in to the AWS CloudFormation console (which is part of the AWS Management Console) and open the Events tab to find information on stack creation, updates, and deletions. If the stack fails to create, information on the Events tab usually gives you a general idea what went wrong.
ArcGIS Enterprise on Amazon Web Services deployment tools also create log files on the virtual machines to help you troubleshoot issues. To be sure the files are preserved even when your deployment fails to launch, sign in to the AWS CloudFormation console and disable the Rollback on failure option.
If your deployment fails when launching from an ArcGIS Enterprise on Amazon Web Services deployment tool, make a remote desktop connection or SSH to the Amazon Elastic Compute Cloud (EC2) instance to view the logs. Log types and locations are listed in the following table:
| EC2 instance type | Log file and location on EC2 instance | Log file description |
|---|---|---|
Ubuntu | /var/log/cfn-init.log | Log file for the CloudFormation helper script used to retrieve and interpret the resource metadata, install packages, create files, and start services |
/var/log/chef-run.log | Chef configuration management tool log file | |
/var/lib/tomcat7/logs/catalina.out | Apache Tomcat application server log file | |
Windows | C:\cfn\log\cfn-init.log | Log file for the CloudFormation helper script used to retrieve and interpret the resource metadata, install packages, create files, and start services |
C:\chef\chef-run.log | Chef configuration management tool log file |
Note:
If CloudFormation stack creation succeeds, the stack output parameters provide a link to the log group in the AWS Management Console. If stack creation fails, go to the CloudFormation Resources list in the AWS Management Console to find the log group. Note that if stack creation fails before any instances are launched, a log group may not be created.
Where can I find the CloudWatch logs for deployments I create using ArcGIS Enterprise on Amazon Web Services deployment tools?
To access CloudWatch logs for deployments you create with ArcGIS Enterprise on Amazon Web Services deployment tools, sign in to the CloudFormation console, which is part of the AWS Management Console. See View CloudFormation Logs in the Console on the AWS website.
Why can't I find CloudWatch logs for the deployment I created using ArcGIS Enterprise on Amazon Web Services deployment tools?
Beginning with 10.8.1, ArcGIS Enterprise Cloud Builder for AWS uses version 2 (V2) of the Amazon Instance Metadata Service (IMDS) when creating deployments. Starting with 10.8.1, Esri CloudFormation templates use IMDS V2 by default as well. At this time, the CloudWatch log agent AWS installs on EC2 instances is not compatible with IMDS V2. This is a known limitation.
Until AWS corrects this limitation, you need to do the following if your stack fails to launch and you want to view logs to troubleshoot:
- Sign in to the CloudFormation console in the AWS Management Console.
- Access the failed CloudFormation stack and check the Events tab to find the reason for the failure.
- If you see a message similar to Received FAILURE signal with UniqueId <instanceid>, use remote desktop or SSH and the instance ID from the message to access the affected EC2 instance.
- Check the Chef logs to identify why the stack did not launch. You can find Chef logs in one of the following locations, depending on the operating system of the EC2 instance:
- Microsoft Windows—C:\chef\chef-run.log
- Ubuntu—/var/log/chef-run.log
Tip:
If you use Esri CloudFormation templates so that your new deployment uses IMDS version 1 (V1) and, therefore, avoid the CloudWatch log agent known limitation. However, because IMDS V1 is less secure than IMDS V2, this is not a recommended workaround.
Errors encountered when launching an AWS CloudFormation stack
The following are some common messages or problems you may encounter when you deploy using an AWS CloudFormation template and suggestions on how to correct them.
Why do I see an error about insufficient capacity when I click Launch to launch an instance?
This is an error from Amazon Elastic Compute Cloud meaning that it does not have the available capacity to meet your request for a new instance. If your deployment architecture permits, you may be able to work around this error by requesting an instance in a different availability zone or allowing EC2 to choose the availability zone for you. Other options are to try launching a different size of instance or launching the instance later.
What does the message Error encountered during build of config: Failed to retrieve https:// .s3.amazonaws.com/ in cfn-init.log mean?
If you see this message in the cfn-init.log file, ensure the deployment S3 bucket name is correct and that the S3 object key name of authorization files and SSL certificates are correct.
What does the message Unable to connect to WebAdaptor URL : https://agsportalssl.esri.com/server/webadaptor in the catalina.out log file mean?
If you see this message in the catalina.out log file on an Ubuntu instance, ensure the SSL certificate in the deployment S3 bucket is valid and in PKCS 12 format. Also be sure the provided SSL certificate password is correct.
What does the message OpenSSL::PKCS12::PKCS12Error: PKCS12_parse: mac verify failure in the chef-run.log file mean?
If you see this message in the chef-run.log file on a Windows instance, ensure the SSL certificate in the deployment S3 bucket is valid and in PKCS 12 format. Also be sure the provided SSL certificate password is correct.
Troubleshoot AWS Management Console
You might encounter one of the following error messages when you use the AWS Management Console and Esri Amazon Machine Images to manually create your site:
Why do I receive the message No password was found when I try to retrieve the administrator password for my EC2 instance on Windows?
This message can appear if you try to use Get Windows Password after you have stopped and started an EC2 instance. To avoid this error, the first time you sign in, change the Administrator password to a value you can more easily remember.
I got a message in the AWS Management Console that my instance is scheduled for retirement. What does this mean?
You may see this message if your instance happens to be running on degraded hardware that Amazon needs to replace. If you see this message, you should stop your site and start it again.
After you restart the site, the message should go away.