Manage data store backups
In this topic
- Define a backup location
- Change backup frequency
- Change how long backup files are retained
- Manually create a backup
- Delete manual backups
- Export
ArcGIS Data Store managed database
You need backups to recover your data in the event of a disaster such as data corruption or data store failure. If you create backups of your data store and place them in a safe location, you can set up a new ArcGIS Data Store, access your backup, and restore the data used by your hosted feature layers if for any reason your data store crashes and cannot be restarted.
You decide where and when to create backups of the feature layer data in your data store for disaster recovery. By default, the data store creates backups of feature layer data automatically in /usr/arcgisdatastore/backups. That means backup files are stored on the same machine as the data store. If the data store or primary machine fails, you can't access the backup files and, therefore, cannot restore your feature layer data. For that reason, you should store your backups in a location other than the default.
The following diagram shows a data store made up of one computer (the primary machine) for storing your data, plus a network drive to store backup files. Storing the backup files on a machine separate from the data store protects you from losing the backup files if the machine running the data store fails.
Note:
Be aware that backup files only contain the data used by your portal's hosted feature layers. Backup files do not maintain a backup of scene layer data or caches, the ArcGIS Server site, or your portal. Rather, backups help you recover feature layer data lost if the data store's primary machine fails. If you want your hosted feature layers to work, even if the primary machine fails, set up a standby machine.
In addition to defining a different backup directory, you can also control how often full backups are created and how long backup files are retained.
If needed, you can manually create a backup of the feature layer data using the backupdatastore utility.
If you need to move your data store's hosted feature layer data to a newer release of ArcGIS Data Store, or you want to restore the data to an installation of ArcGIS Data Store on a machine with a different operating system, export the data store's feature layer data, and import it to the other ArcGIS Data Store.
Define a backup location
Backups let you recover your data if a disaster occurs, such as when your server fails or flood waters destroy your server. If your backup is on the server that is destroyed in a flood, you cannot recover your data. Therefore, you need to store your backup files on a different server than your data store.
Create a shared directory on another machine to store backup files and run the changebackuplocation utility to configure ArcGIS Data Store to write backup files to that location. Be sure the login that runs the changebackuplocation utility has read and write permissions on the shared directory. Also ensure there is enough storage space to hold all the files that are included in a data store backup. The number of files and size of the files will vary depending on your data, your backup schedule, and the number of days you retain backups. Be sure to monitor the size of the backup directory and adjust these settings as needed.
The changebackuplocation utility is installed in the <ArcGIS Data Store installation directory>/datastore/tools directory.
The syntax to run the changebackuplocation utility is as follows:
changebackuplocation <new directory path> --is-shared-folder <true|false> --keep-old-backups <true|false>
For example, type the following to create backup files in a shared directory named ds_backup on a computer named sysshare and opt to keep existing backup files:
./changebackuplocation.sh /net/sysshare/ds_backups --is-shared-folder true --keep-old-backups true
You are going to change the backup location of the data store. Existing backups will be copied to the new location and it could take a few moments.
Please do not interrupt the process once it has started.
Do you want to continue (Yes or No)? Yes
Tip:
If your remote backup directory goes offline for more than a few minutes, perform a full manual backup of the data store as soon as the shared backup location is available.
Change backup frequency
By default, the data store creates a full backup of feature layers once a week and incremental backups either when the log files are full or every 5 minutes, whichever comes first. (Log files allow you to recover to a specific point in time.) The database controls incremental backup creation, but you can change how often the data store creates a full backup by running the updatebackupschedule utility. If your users will be publishing and/or editing large numbers of hosted feature layers, you should increase the frequency with which full data store backups are created.
Note that your backup location must have enough room to store all the backups. Backup size varies depending on the amount of data you have but, if you use default backup settings, backups will contain five full backups and 31 days of incremental backup files. The size of these files depends on the amount and size of your data.
The updatebackupschedule utility is installed in the <ArcGIS Data Store installation directory>/datastore/tools directory.
- Open a command shell.
- Run the updatebackupschedule utility to specify the backup frequencies you require.
The syntax to run the utility is as follows:
updatebackupschedule --starttime <local server time> --frequency <number of days>
For example, type the following to schedule full backups at 3:00 a.m. (local server time) every 14 days (every two weeks):
./updatebackupschedule.sh --starttime 03:00:00 --frequency 14
- Stop and start ArcGIS Data Store using the stopdatastore.sh and startdatastore.sh commands found in <ArcGIS Data Store installation directory>/datastore.
Changes are not applied until you restart ArcGIS Data Store.
Change how long backup files are retained
The backup directory retains backup files for 31 days by default. That means if you keep the default backup frequency (every 7 days) and retention schedules (31 days), the backup directory will contain five full backups and 31 days of incremental backup files. The size of these files depends on the amount and size of your data. The machine that stores your backups must have enough disk space for all these files. If you increase the backup frequency, consider decreasing the retention period for the backup files. In the previous section, the backup frequency was increased to every 14 days. To prevent your backup directory from becoming too large, decrease the backup file retention period.
The syntax to run the updatebackupretaindays utility is as follows:
updatebackupretaindays <number of days>
In the following example, backup file retention time is changed to 27 days:
./updatebackupretaindays.sh 27
Manually create a backup
You can use the backupdatastore utility to make a full backup of the feature layer data in your data store. You might manually create a full backup prior to making a large number of changes to the data store or prior to upgrading the data store. Or you might create a backup to preserve a copy of the data in a particular state, for example, at the end of the first phase of a project. The login with which you connect to the data store machine to run the backupdatastore utility must have read and write access to the data store backup location.
The syntax to run the backupdatastore utility is as follows:
backupdatastore <backup_name>
Provide a meaningful name for the file so you can find it when you want to restore the data. If you do not specify a name, the utility assigns a default name to the file. The default name is in the format datastorename-timestamp. For example, if your data store is named corpds and you create the backup on July 10, 2014 at 14:25:49:554 UTC, the backup file name is corpds-20140710142549554.
You will be asked to confirm that you want to create a backup. Type yes or y to proceed with backup creation.
Tip:
If you want to script manual backups, include a flag to suppress the confirmation prompt, as in the following example:
backupdatastore --prompt no
In this example, the data store will generate the backup file name. This is necessary in a script to ensure a unique backup file name.
Delete manual backups
If you no longer need to keep a backup file you created using the backupdatastore utility (a manual backup), you can run the listbackups utility to get the name of the specific backup file, and then run the deletebackup utility to remove the unneeded file. For example, once you have upgraded your data store and confirmed all feature layers are performing as expected, you could delete the data store backup you created prior to upgrading.
In this example, the database preupgrade1104_bu is deleted.
./listbackups.sh Backup_Name Status Backup_Time Mode ==================================================================================== phase1proj_bu BackupComplete 2014-03-08 14:12 manual phase2proj_bu BackupComplete 2014-06-21 11:43 manual preupgrade_bu BackupComplete 2014-10-04 09:30 manual ds_gdt1oomh-20141103160748082 BackupComplete 2014-11-01 03:00 scheduled /deletebackup preupgrade1104_bu You are attempting to delete backup 'preupgrade1104_bu'. This operation is irreversible. Do you wish to continue (Yes or No)?yes Operation completed successfully
Export ArcGIS Data Store managed database
Managed databases store feature layer data. They are the data store for hosted feature layers. When you use ArcGIS Data Store as the managed database for your portal's hosting server, you can export the managed database. You would do this if you need to make a backup of the managed database to be restored to an ArcGIS Data Store installation that is on a machine with a different operating system or is a different ArcGIS Data Store release. Use the exportmanageddb utility to export the database and relevant files. Files are output to a directory that you can move to another ArcGIS Data Store machine. Use the importmanageddb utility to restore the database to the new data store.
For example, if ArcGIS Data Store is running on a Windows server, but your information technology department is switching to Linux servers, you can do the following to move the data store to the new server:
- Install ArcGIS Data Store on the Linux server, but do not proceed with configuring a data store.
- Export the managed database from the existing ArcGIS Data Store on the Windows server using the exportmanageddb utility. Export to a shared location that can be accessed by the new machine.
In this example, the managed database is exported to a directory named ds2move on a shared location, \\sharedbox\exports.
exportmanageddb ds2move --destination \\sharedbox\exports
- Import the managed database to the new ArcGIS Data Store installation using the importmanageddb utility on the new machine. Specify --bound true so your ArcGIS Server site will use the data store in this new location, and existing hosted feature layers will continue to work.
In this example, the new data store's data directory is /usr/arcgis/datastore.
importmanageddb ds2move --source-loc /net/sharedbox/exports/dbdump/ds2move --data-dir /usr/arcgis/datastore --bound true
- After you import, restart the data store on the new machine.