The Esri arcgis-notebook-server-ha.template.json Amazon Web Services (AWS) CloudFormation template creates an ArcGIS Notebook Server site on multiple Linux Amazon Elastic Compute Cloud (EC2) instances to provide high availability.
The EC2 instances are deployed in an AWS Auto Scaling group, which offers advantages such as capacity planning based on certain criteria, better fault tolerance, better availability, and better cost management.
ArcGIS Notebook Server sites on AWS are only supported on Linux instances.
For specific operating system versions, see Operating systems supported when using CloudFormation to ArcGIS deploy on AWS.
This template creates the following architecture in Amazon Web Services:
License:
Certain icons in the diagram are used with permission from Amazon Web Services.
Prerequisites
Prerequisites can be grouped by the items—such as files and accounts—that you must obtain and the tasks you must perform before running the CloudFormation template.
Required items
You need the following before running this template:
- An Amazon Web Services account.
The account must have access to basic AWS services such as CloudFormation, Amazon Elastic Compute Cloud (EC2), Amazon Virtual Private Cloud (VPC), Amazon Simple Storage Service (S3), Systems Manager, Amazon CloudWatch, Lambda, AWS Identity and Access Management (IAM), Amazon DynamoDB, Secrets Manager, AWS Certificate Manager, and Amazon Relational Database Service (RDS).
- An ArcGIS Notebook Server license file (.ecp or .prvc).
- An SSL certificate file (in .pfx format) and corresponding password.
The certificate must be from a certifying authority.
- If you create a deployment on Ubuntu EC2 instances in AWS GovCloud, you need an AMI ID. If you want to use the base canonical Ubuntu AMI, follow the instructions in Esri Amazon Web Services CloudFormation templates to identify the ID.
- An Amazon Virtual Private Cloud (VPC) and subnets.
All components in the same ArcGIS Enterprise deployment must run in the same VPC. If your portal exists, use the same VPC and subnets you used for the portal. If you have not created a portal yet and do not have an existing VPC, you can use one of the following CloudFormation templates to create a VPC: VPC with two public subnets or VPC with two public and private subnets with a NAT Gateway.
- The arcgis-notebook-server-ha.template.json CloudFormation template.
Required tasks
Complete the following tasks before running this template:
- Prepare a deployment Amazon Simple Storage Service (S3) bucket in the AWS account. Specify the bucket name in the template when you launch the stack.
- Create a bucket or use an existing S3 bucket. You must be the owner of the bucket.
- Upload the ArcGIS Notebook Server authorization file to the bucket.
- Upload the SSL certificate file to the bucket.
- Configure passwords in AWS Secrets Manager (optional but recommended).
You can configure the passwords for accounts such as the site administrator username and the Windows arcgis user password in AWS Secrets Manager. This provides you with a secret Amazon Resource Name (ARN). Use the ARN in place of a password in the template parameters when you launch a stack. If you don't use AWS Secrets Manager for storing passwords, you must type passwords in plain text in the template parameter when launching the stack.
Note:
When creating a secret ARN in AWS Secrets Manager for a password to be used with Esri CloudFormation templates, you must use the Other types of secrets secret type and use the Plaintext option. For more information on creating an Amazon Resource Name for passwords, see AWS CloudFormation and ArcGIS.
- Configure a Domain Name System (DNS).
You must have a fully qualified domain name for the ArcGIS Notebook Server site. This domain name must exist before you launch this stack, and it must be resolvable. Contact your IT department if you are unsure how to obtain a fully qualified domain name and configure a DNS.
If you use an Elastic Load Balancer (ELB) in your ArcGIS Notebook Server deployment (optional), you must create the ELB before you create the ArcGIS Notebook Server site. Only application load balancers are supported. You can use the application load balancer template to create it.
- If you deployed on a Red Hat Enterprise Linux Server EC2 instance, you must install and configure Docker. See the ArcGIS Notebook Server Linux installation guide for instructions.
Tip:
By default, CloudFormation deletes partially created resources if stack creation fails. This is helpful because it removes unusable deployments from your account, but it can make it difficult to troubleshoot. To retain the stack in its failed state, disable the Rollback on failure CloudFormation stack creation option before launching the stack. See Setting AWS CloudFormation options in the AWS help for more information.
Parameters
Refer to the following tables for descriptions of the parameters used in this CloudFormation template. Tables are grouped by parameter type.
Amazon EC2 Configuration
| Parameter name | Required? | Parameter description |
|---|---|---|
EC2 Instance AMI ID | Optional | You can leave this parameter value empty. If you do, the CloudFormation template will use the latest Amazon Machine Image (AMI) ID for Ubuntu Server 20.0.4 LTS. Note:You cannot leave this parameter empty if you deploy in AWS GovCloud on a Linux platform. See Esri Amazon Web Services CloudFormation templates for instructions for finding the ID for the base Ubuntu AMI from Canonical. If you deploy on a supported Linux operating system other than Ubuntu, you can find the AMI ID using AWS Management Console. To use your own custom AMI, type the AMI ID using one of the following:
If you use a custom AMI, ensure that it meets the following requirements:
|
EC2 Instance Keypair Name | Required | Choose an EC2 keypair name to allow remote access to EC2 instances. |
Amazon VPC Configuration
| Parameter name | Required? | Parameter description |
|---|---|---|
VPC ID | Required | Choose a VPC ID. Note:All ArcGIS Enterprise components that are part of the same deployment must be deployed in the same VPC. If you need to create a VPC, you can use one of the VPC sample templates: VPC with public subnets or VPC with public and private subnets and a NAT Gateway. |
Subnet ID 1 | Required | Choose a subnet ID. The subnet ID that you select must be within the VPC you have selected above. If you used an Esri CloudFormation template to create the VPC, you can get the subnet ID from that template's output parameters. |
Subnet ID 2 | Required | Choose a second subnet ID. This must be a different subnet ID than you used for the Subnet ID 1 parameter. The subnet ID that you select must be within the VPC you specified for this deployment. If you used an Esri CloudFormation template to create the VPC, you can get the subnet ID from that template's output parameters. |
Domain Name System (DNS) Configuration
| Parameter name | Required? | Parameter description |
|---|---|---|
ArcGIS Notebook Server Site Domain Name | Required | Provide the fully qualified domain name for the ArcGIS Notebook Server site. The domain name must exist and be resolvable. Contact your IT administrator if you are not sure what domain name to use. |
Elastic Load Balancer DNS Name | Optional | To use an ELB with the site, provide the value for an application or ELB DNS name. It must be an existing ELB. If you do not want to use an ELB or want to configure it by yourself later, leave this parameter value empty. You can get the ELB DNS name by browsing to the Load Balancers section of the Amazon EC2 service within the AWS Management Console or, if you used an Esri CloudFormation template to create the ELB, you can get it from that template's output parameters. Valid ELB DNS names must end with .elb.amazonaws.com. |
ArcGIS Notebook Server Configuration
| Parameter name | Required? | Parameter description |
|---|---|---|
EC2 Instance Type | Required | Specify an EC2 instance type. The default is m5.2xlarge. This is the instance type that will be used for the ArcGIS Notebook Server machines. The EC2 instances will be configured in an AWS Auto Scaling group. These instances will join the ArcGIS Notebook Server site by accessing the ArcGIS Notebook Server configuration store. |
EC2 Instance Root Drive Disk Space | Required | The is the size of the root drive disk space for the ArcGIS Notebook Server EC2 instances. Provide the size of the root drive in GB. The default is 100 GB. Minimum is 100 GB. Maximum is 1024 GB. |
Number of EC2 Instances | Required | Provide the number of EC2 instances that participate in the site. The default is 2. The maximum is 10. The minimum is 1. The EC2 instances will be configured in an AWS Auto Scaling group. See the Considerations section below for information about resizing an Auto Scaling group. |
Deployment Bucket Name | Required | Provide the name of the Amazon S3 bucket that contains your software license files and SSL certificates. This bucket must already exist and contain the license file and SSL certificate for your deployment. You must be the owner of the bucket and it must reside in the same AWS account as your deployment. |
License File Name | Required | Provide the ArcGIS Notebook Server authorization file object key name. You must upload the license file (.ecp or .prvc file) to the deployment bucket before launching this stack. You can get the file object key name by browsing to the file within the deployment bucket in the Amazon S3 console. Examples include notebook.prvc or resources/licenses/server/notebook.prvc. License file names are case sensitive. Ensure that you type the name correctly. |
License Level | Required | Choose the ArcGIS Notebook Server license level. Options are as follows:
|
Site Administrator User Name | Required | Provide a username for the primary site administrator for the ArcGIS Notebook Server site. The name must be 6 or more alphanumeric or underscore (_) characters and must start with a letter. |
Site Administrator User Password | Required | Provide a password for the primary site administrator for the ArcGIS Notebook Server site. You can either type a plain text password or the ARN of your secret ID from AWS Secrets Manager. For more information on creating an Amazon Resource Name for passwords, see AWS CloudFormation and ArcGIS. The password must be 8 or more alphanumeric characters and can also contain dots (.). The password cannot contain any other special characters or spaces. |
Configuration Store Type | Required | Choose the ArcGIS Notebook Server configuration store type. The default is FileSystem.
Note:Even if you choose CloudStore, a separate file server is created to host the ArcGIS Notebook Server shared directories. |
Web Adaptor Name | Required | Type a web adaptor name for the ArcGIS Notebook Server site. Access to the ArcGIS Notebook Server site will be through a URL in the format https://<fully qualified domain name>/<web adaptor name>. The name must begin with a letter and contain only alphanumeric characters. |
SSL Certificate File Name | Required | Provide an SSL certificate from a certifying authority (.pfx file). You must upload the certificate to the deployment bucket before launching this stack. You can get the file object key name by browsing to the file within the deployment bucket in the AWS S3 console, for example, domainname.pfx or resources/sslcerts/domainname.pfx. |
SSL Certificate Password | Required | Provide the password for the SSL certificate. You can either type a plain text password or the ARN of your secret ID from AWS Secrets Manager. For information on creating an Amazon Resource Name for passwords, see AWS CloudFormation and ArcGIS. |
Outputs
When your stack is created successfully, you can see the following output parameters on the Outputs tab of the CloudFormation stack in AWS Management Console.
| Output name | Output description |
|---|---|
DeploymentLogsURL | This is the URL for the Amazon CloudWatch logs where all deployment logs are stored. You can refer to these logs for troubleshooting purposes if your deployment fails. |
ServerAdminURL | The ArcGIS Notebook Server administrator directory URL. |
ServerServicesURL | The ArcGIS Notebook Server services URL. You can use this URL as an input parameter in the federate server template. |
StopStackFunctionName | This is the Stop Stack Lambda function URL. You can use this lambda function to stop all EC2 instances in the stack. |
StartStackFunctionName | This is the Start Stack Lambda function URL. You can use this lambda function to start all EC2 instances in the stack that you previously stopped. |
Considerations
The following are important points to consider after creating a CloudFormation stack containing ArcGIS deployments:
- In highly available ArcGIS Notebook Server deployments, EC2 instances are configured in an AWS Auto Scaling group. Once the ArcGIS Notebook Server site is created, you can add more machines (EC2 instances) to the site by adjusting the size of the Auto Scaling group. Use AWS Management Console to increase the capacity of the Auto Scaling group to add machines. To remove machines, decrease the capacity. See the AWS documentation for instructions on changing the capacity limits of an Auto Scaling group to add machines to or remove machines from an ArcGIS Notebook Server site.
Tip:
To find the AWS Auto Scaling group created for a CloudFormation stack, review the Resources tab of the related CloudFormation stack. The logical ID of the resource is AutoScalingGroup.
Although it is technically possible to change the AWS Auto Scaling group size to zero, keep at least one instance running in the Auto Scaling group; otherwise, the ArcGIS Notebook Server site may not function properly.
Once you change the capacity of the group, it may take up to an hour to add machines or up to 15 minutes to remove machines. Once the change is applied, use the ArcGIS Server administrator API to confirm that the number of machines in the ArcGIS Server site has increased or decreased appropriately.
- The configuration store for the ArcGIS Notebook Server site is stored in an S3 bucket and Amazon DynamoDB. The DynamoDB provision capacity units are set to the following:
- Read capacity units = 250 tables
- Write capacity units = 25 tables
Esri testing indicates that these settings work well for publishing approximately 500 services. You can edit this setting in AWS Management Console to decrease the units and lower the cost or increase the units to accommodate more services. Keep in mind that some functionality, such as publishing, will fail if you don't have enough capacity.
- Do not delete any AWS resource created by this CloudFormation template. If you want to know what AWS resources have been created by this template, refer to the Resources tab of this stack in the AWS Management Console. Each resource created by an Esri CloudFormation template also has metadata tags. However, some of the resources do not show tags in the AWS Management Console.
- If you use AWS Secrets Manager for passwords, such as the site administrator user password or the Windows arcgis user password, and later (after you create the deployment) you change those passwords, be sure you update the appropriate AWS Secrets Manager ARN's with the updated passwords.
- If you use an Application Load Balancer in your deployment, you can adjust the Application Load Balancer idle timeout after you create the deployment. For example, you may need to increase the idle timeout to ensure that operations that take a long time to complete on any of the machines in the deployment. Adjust the Application Load Balancer idle timeout value in AWS Management Console.
- You can use the AWS Lambda functions that appear in your stack outputs (listed in the Outputs section of this page) to stop EC2 instances in this stack when they are not in use and start them again when required. These functions are useful to help you manage costs.
When you run the Lambda function to stop the EC2 instances in the deployment, the function returns a message that the instances are stopped. However, if the instances participate in an Auto Scaling group, the function must also detach the instances from the group. This can take up to 10 minutes to complete. Therefore, wait at least 10 minutes before you run the Lambda start function to restart the deployment.
Note:
You must use the Lambda functions to stop and start EC2 instances that are part of an Auto Scaling group. Auto Scaling groups add machines based on the capacity set for them; if you use AWS Management Console to shut down EC2 instances in an Auto Scaling group, new instances will be added.
Troubleshooting
If you observe any failures when creating this CloudFormation stack, see Troubleshoot ArcGIS deployments on AWS.