After you upgrade an ArcGIS Enterprise deployment to 11.4, it is recommended that you add an object store to the deployment and migrate hosted scene layer caches to it. When you migrate, the caches are moved from the ArcGIS Data Store tile cache data store to the object store.
The MigrateSceneServices utility allows you to gather information about the services and caches before migrating. This includes the names of the scene services and the size of their caches. After you determine what you need to migrate, you can use the MigrateSceneServices utility to automate a bulk migration of all hosted scene layer caches or provide a list of service names to migrate at a time. For example, if you have one or two extremely large caches, you can migrate them overnight, as those will take a long time to migrate. After those finish and you test the hosted scene layers, run the utility again to migrate the remaining hosted scene layer caches.
When migrating, the utility does the following per each hosted scene layer:
- It copies the cache to the object store.
- Once the cache for that layer is copied to the object store, it is deleted from the tile cache data store.
That means that there is a period of time when a scene layer's cache exists in both data stores. If you have both the tile cache data store and object store on the same machine (which is not recommended), you must account for this temporary increased disk space requirement.
The cache that you migrate to the ArcGIS Data Store object store will use approximately the same amount of disk space on the object store as it did on the tile cache data store machine.
Note:
Alternatively, the ArcGIS Enterprise organization administrator or the owner of a hosted scene layer can migrate individual hosted scene layers from the scene layer's item page in the portal. See Manage scene layers for instructions.
Whichever method you use, migrating the layer's cache uses large amounts of CPU on the tile cache data store and object store machines.
Reason to migrate hosted scene layers
The ArcGIS Data Store tile cache data store is deprecated at 11.4. Support for the tile cache data store will be dropped in a future release. For this reason, you are encouraged to start moving hosted scene layer caches to a system object store.
Prerequisites for migration
Before you can migrate hosted scene layer caches, you must do the following:
- Upgrade the ArcGIS Enterprise deployment to 11.4.
- If the ArcGIS Enterprise deployment does not already have an object store configured, you must configure one.
It is strongly recommended that you create a backup before you begin migrating scene layer caches. See ArcGIS Enterprise backups and the backupdatastore utility reference for information about creating backups.
MigrateSceneServices utility parameters and syntax
The syntax to run the MigrateSceneServices utility is as follows:
MigrateSceneServices.so -m <analysis | execution> -s <hosting server URL> -u <administrator username> -p <administrator password> [-t <portal token>] [-r <token referrer>] [--select <list of services to migrate>] [-o <directory>] [-l <log level>] [--num-threads <threads to use>]
The following are descriptions of each parameter:
| Parameter | Description |
|---|---|
| -m, --mode | Two modes are supported.
|
| -s, --server_url | Provide the URL of the hosting server site. |
| -u, --username | Specify the username of a login that is a member of the default administrator role in the ArcGIS Enterprise portal associated with this hosting server site. |
| -p, --password | Provide the password of the administrator account you specified with the -u parameter. |
| -r, --referrer | The referrer for the portal token. Use this to further restrict usage, designate the specific referrer URLs or IPs that can access the service. For example, you can restrict to the URL of your ArcGIS Enterprise organization. |
| -t, --token | A token can be generated and used in place of the username and password. When a token is provided, it overrides any username or password that is provided. If you specify a referrer, you must provide a portal token. |
| --select | This optional parameter allows you to provide a set list of services to migrate. The list must be in the following format: "[servicename1, servicename2, servicename3]" |
| -o, --output-dir | This optional parameter allows you to specify the directory in which the utility report file will be created (when you run the utility in analysis mode) or the log file will be created (when you run the utility in execution mode). If undefined, the utility generates the report or log file in a temporary directory on the machine where you run the utility. |
| -l, --log_level |
Use this optional parameter to set the logging level used for the log file that is generated when you run the utility. The default logging level used is info if you do not specify this parameter with a different value. Valid values, listed from most verbose to least verbose, are as follows:
|
| --num-threads | Overrides the default number of threads used for migration. The default number of threads used is 16. If you notice 100% CPU usage during cache migration, and migration fails or cannot complete, decrease the number of threads used by the utility. When you alter this setting, it is recommended that you set the number of threads to the number of cores on the machine. For example, if the machine has four cores, set --num-threads to 4. |
| -h, --help | Use this parameter to return the syntax for the utility. |
Run the utility
The MigrateSceneServices utility is installed on ArcGIS Server machines in the <ArcGIS Server installation directory>\tools\MigrateSceneServices folder.
You must run the utility from one of the machines in the hosting server of the ArcGIS Enterprise portal.
When you run the MigrateSceneServices utility in either mode, a log file is generated. Both modes display counts of the services (scene layers) that are eligible to be migrated.
The report generated when you run the utility in analysis mode provides the size of each cache.
Tip:
If the object store is running on the same machine as the tile cache data store, identify the largest cache size. Ensure that there is enough free disk space on the data store machine to accommodate at least twice the size of this largest cache, because there will be a small period of time during migration when the cache has been created in the object store but has not yet been deleted from the tile cache data store.
In execution mode, log entries relating to the migration operation are displayed, as well as a count of the services that have been migrated.
Follow these steps to access and run the utility:
- Sign in to one of the machines in the hosting server site using the ArcGIS Server account login or using a login that has read privileges to all ArcGIS Server directories.
- Open a command window and change to the tools directory listed above.
- Run the MigrateSceneServices utility in analysis mode.
In the following example, the utility is run in analysis mode, and the log file is written to a directory named analysislogs:
./MigrateSceneServices.so -m analysis -s https://hostingserver.example.com/waname -u portaladmin -p L00kAway! -o /temp/analysislogs
- If the report generated from running the utility in analysis mode indicates that the deployment is configured to allow migration and that there are scene layer caches to migrate, run the utility in execution mode.
The following example migrates the caches of all scene layers:
./MigrateSceneServices.so -m execution -s https://hostingserver.example.com/waname -u portaladmin -p L00kAway! -o /temp/migrationlogs -l error
- After all scene layer caches are migrated and tested, run the utility one last time in analysis mode to confirm no caches remain that need to be migrated.
Example usage
After the administrator configures an object store, the ArcGIS Server administrator runs the MigrateSceneServices utility in analysis mode to determine which scene services need to be migrated.
The ArcGIS Server administrator signs in to one of the machines in the ArcGIS Server site that is used as the hosting server. The administrator uses the account that installed ArcGIS Server.
The administrator opens a command window, accesses the tools directory, and runs the command in analysis mode. Because the report will be shared with colleagues, the administrator specifies a shared directory for the output location.
cd /data/arcgis/server/tools/MigrateSceneServices
./MigrateSceneServices.so -m analysis -s https://hostingserver.example.com/hserver -r https://webadaptorhost.example.com/webadaptor/admin -t abcdefg1234567 -o /net/shareddir/utiloutput
The utility runs and determines that four of the scene layers still have their caches in the tile cache data store and can be migrated. Results are shown at the command line and are listed in the report that was created in the utiloutput directory. The report shows that sceneservice1, sceneservice2, sceneservice3, and sceneservice4 still have their caches in the tile cache data store.
In the report, each service is listed with its cache size, as shown below:
{
"mode":"analysis",
"messages":[
"INFO: Started: Fri Aug 2 17:19:58 2024",
"INFO: Found 4 migratable service(s)",
"INFO: Largest service: [sceneservice2] (96459081B / ~(91.99MB)",
"INFO: Ended: Fri Aug 2 17:20:01 2024"
],
"services":[
{
"sceneservice1":{
"messages":[
"INFO: size: (48887B / ~5.1MB)"
]
},
"sceneservice2":{
"messages":[
"INFO: size: (96459081B / ~91.99MB)"
]
},
"sceneservice3":{
"messages":[
"INFO: size: (722299B / ~11.37MB)"
]
},
"sceneservice4":{
"messages":[
"INFO: size: (61005191B / ~70.11MB)"
]
}
}
]
}GIS staff members review the report that was written to the shared location, and they check each of the scene layers before migration takes place. This allows them to compare the layer's content and performance pre- and postmigration. They discover that sceneservice3 has an associated feature service that is actively being edited. Therefore, sceneservice3 will not be migrated until the editor has finished working.
The ArcGIS Server administrator runs the MigrateSceneServices utility to move caches for the other three scene layers.
./MigrateSceneServices.so -m execution -s https://hostingserver.example.com/hserver -r https://fully.qualified.domain:6443/arcgis/admin -t abcdefg1234567 -f [sceneservice1,sceneservice2,sceneservice4] -o /net/shareddir/utiloutput -l debug
When the utility completes, each eligible scene layer cache has been migrated. The ArcGIS Server administrator runs the MigrateSceneServices utility again in analysis mode to confirm that no additional scene layer caches need to be migrated.
After all migrated scene layers have been tested and confirmed to function, the ArcGIS Data Store administrator runs the disabledatastore utility to unregister the tile cache data store from the hosting server and delete all its contents.