Manage ArcGIS Data Store backups

You need backups to recover data in the event of a disaster such as data corruption or data store failure. If you create backups of the data stores implemented through ArcGIS Data Store, and place the backup files in a safe location, you can set up a new ArcGIS Data Store, access the backup files, and restore the data if the data store crashes and cannot be restarted.

Read the ArcGIS Data Store backup considerations, then use the information in the remaining sections to configure and manage ArcGIS Data Store backups.

Change the initial default backup location for a relational store or define a default backup location for all other data store types.
Configure automatic backups, including setting a schedule.
Manually create backups as necessary.

ArcGIS Data Store backup considerations

Keep the following in mind when implementing a backup and recovery strategy for ArcGIS Data Store:

Backups allow you to recover data if a disaster occurs, such as when a server fails or floodwaters destroy the server. If the backup is on the server that is destroyed in a flood, you cannot recover the data. Therefore, always store backup files on a different server than the data store and, if possible, in a different physical location than the data store.
ArcGIS Data Store backup files only contain the relational store, graph store, object store, or spatiotemporal big data store. Backup files do not maintain a backup of the GIS Server site, the portal, or user-managed data stores you register with the GIS Server site. ArcGIS Data Store backup files also do not contain the object store if you registered a cloud-provided location in Amazon Web Services or Microsoft Azure. You must create backups of those components separately.

You can use the webgisdr tool installed with Portal for ArcGIS to create a backup that includes all ArcGIS Data Store types except the spatiotemporal big data store. When you use the webgisdr tool, a backup is also created of the portal, hosting server, and federated servers. See ArcGIS Enterprise backups for information about using this tool. You'll still need to create separate backups of the user-managed data stores you register with the GIS Server site and the spatiotemporal big data store.
ArcGIS Data Store backups help you recover data lost if the ArcGIS Data Store machine fails or data gets corrupted. They do not provide high availability. If you require that hosted web layers continue to be available even if a single ArcGIS Data Store machine fails, add a machine to each data store type to make them highly available.
Due to changes in the underlying storage mechanisms and in ArcGIS software, data store backups created with older ArcGIS Data Store versions cannot be used to restore data to newer ArcGIS Data Store versions. Always create a full backup of each of the data stores after you upgrade ArcGIS Data Store. The only exception to this is relational stores; backup files from the previous release may work. To determine which backup files can be used to restore the data store, run the listbackups utility.
When you deploy ArcGIS Enterprise on-premises but your data store backup files are in cloud storage, creating backups and restoring from those backups will take longer than when backup files are stored in local file shares.
There is no automatic cleanup of graph store, object store, or spatiotemporal big data store backup files. There is no automatic cleanup of any backups you create using the backupdatastore utility.
When you create a backup of the object store, feature layer query caches are not included.

ArcGIS Data Store backup locations and behavior

All ArcGIS Data Store types require a default backup location. The backups automatically created by ArcGIS Data Store are always created in the default backup location.

You can define additional backup locations that you can specify for output when you run the backupdatastore utility to create full backups. This type of backup is also referred to as a manual backup. If you do not specify a location when you run the backupdatastore utility, the default backup location is used.

The following table summarizes the differences in backup locations for each type of data store. Read the information pertinent to the type of data store (or stores) you manage.


Data store	Default backup location	Locations for manual backups
Graph store	You must register a default backup location before you can create a backup of the graph store.	You can register additional file share backup locations for the graph store.
Object store	You must register a default backup location before you can create a backup of the object store.	You can register additional file share backup locations for the object store.
relational store	Default backups are written to a subfolder in the ArcGIS Data Store content directory until you change it. Change the initial default backup location to be a file share on a separate machine.	You can register additional backup locations for the relational store, including other file shares, Amazon Simple Storage Service (S3) buckets, and Microsoft Azure Blob Storage containers.
Spatiotemporal big data store	You must register a default backup location before you can create spatiotemporal big data store backup files. The default backup location can be a file share location, an Amazon S3 bucket, or a Microsoft Azure Blob Storage container.	You can register additional backup locations for the spatiotemporal big data store, including other file shares, Amazon S3 buckets, and Microsoft Azure Blob Storage containers.

Default backup locations

The default backup location is where scheduled, automated backups write files and where backup files you create when running the backupdatastore utility are created by default.

The relational store backup location is automatically configured on the same machine as the relational store. This is the default backup location. However, it is not a good practice to keep backups on the same machine as the data store. Therefore, define a shared directory on another machine to store backups for the relational store.

The graph store, object store, and spatiotemporal big data store do not have a default backup location. You must register at least one backup location before you can create backups of these types of data stores.

For more information and instructions, see Configure ArcGIS Data Store default backup location.

Register additional backup locations

You can register additional backup locations. When you run the backupdatastore utility to manually create backups, you can specify one of these predefined locations.

Follow these steps to add another location for backup files:

Create another location for backup files.
- To register a shared directory (file share), create the directory on another machine. Ensure the storage space is large enough to hold all backup files. Also ensure that the login you use when you connect to the ArcGIS Data Store machine to run the backupdatastore utility has write access to this directory. If you create an additional shared directory for the spatiotemporal big data store, all machines in the same ArcGIS Data Store deployment must have access to this shared directory. If you create an additional shared directory for the relational store backups, the standby machine must have access to the shared directory.
- To register an S3 bucket, create the bucket under your Amazon Web Services account. Choose a bucket size that can accommodate the backup files.
- To register an Azure Blob Storage container, create the container under your Azure Blob Storage account.
Run the configurebackuplocation utility with the register operation to register this additional backup location.

In this example, a second file share location is registered to store manual backups for the relational store. The backup location is given the name rbu_manual.
```
./configurebackuplocation.sh --operation register --store relational
 --location "type=fs;location=/net/sysshar2e/ds_manual_backups;name=rbu_manual"
```
```
configurebackuplocation --operation register --store relational
 --location "type=fs;location=\\sysshare2\ds_manual_backups;name=rbu_manual"
```
The steps and examples for registering additional backup locations for the graph store, object store, and spatiotemporal big data store are the same when adding backup locations as when registering an initial backup location.

Specify one of the backup locations as the default location

When you have multiple backup locations defined for the graph store, object store, or spatiotemporal big data store, you can designate one of them as the default backup location. After you do that, backups created by ArcGIS Data Store (automatic backups) will write the backup files to that location.

Tip:

To change the default location for a relational store, use the change operation.

Run the configurebackuplocation utility with the setdefault operation and specify the registered backup location. If you named the backup location when you registered it, you can use the name to designate the location.

Graph store example

In the following example, the graph store has multiple file share backup locations. The setdefault operation is used to designate one of them as the new default backup location.

./configurebackuplocation.sh --operation setdefault --store graph --location "location=/net/server/gsbackups24"

configurebackuplocation --operation setdefault --store graph --location "location=\\sysshare2\gsbackups24"

Object store example

In the following example, one of the backup locations configured with the object store—s3_backups—is set as the default backup location.

./configurebackuplocation.sh --operation setdefault --store object --location "name=s3_backups"

configurebackuplocation --operation setdefault --store object --location "name=s3_backups"

Spatiotemporal big data store example

In the following example, one of the spatiotemporal big data store's additional backup locations (one named bu_container) is designated as the new default backup location:

./configurebackuplocation.sh --operation setdefault --store spatiotemporal --location "name=bu_container"

configurebackuplocation --operation setdefault --store spatiotemporal --location "name=bu_container"

Determine all backup locations

It can become confusing to keep track of all the data store backup locations when you have multiple data store types and multiple locations for each. Or maybe you took over the job of ArcGIS Data Store administrator from someone else. In these cases, you can run the configurebackuplocation utility with the list operation to query ArcGIS Data Store for a list of all the backup locations for a specific data store type.

In the following example, all registered backup locations are returned for the spatiotemporal big data store in an ArcGIS Data Store deployment. To get a list for the other data store types, specify that type with the --store option, using relational, graph, or object.

./configurebackuplocation.sh --operation list --store spatiotemporal

configurebackuplocation --operation list --store spatiotemporal

Manage automatic backups

By default, ArcGIS Data Store creates a full backup of relational stores every four days, but you can change how often ArcGIS Data Store creates a full relational store backup by running the updatebackupschedule utility. For all other data store types, there is no backup schedule until you set one using the updatebackupschedule utility.

Change backup frequency

If your organization members publish and edit large numbers of hosted layers, or you archive large volumes of streaming data, increase the frequency of backups.

Note:

By default, incremental backups are disabled for relational stores. If you enable point-in-time recovery, incremental backups are created either when the log files are full or every five minutes, whichever comes first. The database controls incremental backup creation; you cannot control the frequency with which incremental backups are created.

The backup location must have enough room to store all the backup files. Backup size varies depending on the amount of data you have, but if you use default backup settings, backups contain two full backups. The size of these files depends on the amount and size of the data. If you reenable point-in-time recovery for relational stores, the backups also include seven days of incremental backup files by default.

If you decide to create backups manually and want to disable automatic backups, set the backup frequency to 0. If you disable automatic backups, you must create the backups yourself to guard against data loss in the event of machine failure or other data catastrophe.

Note:

When you disable automatic backups, you cannot use point-in-time recovery for the relational store; all relational store backups will be full backups.

The updatebackupschedule utility is installed in the <ArcGIS Data Store installation directory>/datastore/tools directory.

The updatebackupschedule utility is installed in the <ArcGIS Data Store installation directory>\datastore\tools directory.

Open a command shell.
Open a command prompt using the Run As Administrator option.
Run the updatebackupschedule utility to specify the backup frequency you require.

The syntax to run the utility is as follows:
```
updatebackupschedule [--store relational|spatiotemporal|graph|object]
[--starttime <local server time>] --frequency <number of days>
```
For example, type the following to schedule full relational store backups at 3:00 a.m. (local server time) every day:

./updatebackupschedule.sh --store relational --starttime 03:00:00 --frequency 1

updatebackupschedule --store relational --starttime 03:00:00 --frequency 1

In this example, a backup of the spatiotemporal big data store is scheduled for 11:30 p.m. (local server time) every three days:

./updatebackupschedule.sh --store spatiotemporal --starttime 23:30:00 --frequency 3

updatebackupschedule --store spatiotemporal --starttime 23:30:00 --frequency 3

Change how long automatic relational store backup files are retained

The backup directory retains relational store backup files for seven days by default. This means if you keep the default backup frequency (every four days) and retention schedules (seven days), the backup directory contains two full backups. If you enable point-in-time recovery, the backup directory also contains seven days of incremental backup files. The size of these files depends on the amount and size of the data. The machine that stores the backups must have enough disk space for all these files. If you increase the backup frequency, consider decreasing the retention period for the backup files. In the previous section, the backup frequency was increased to every day. To prevent the backup directory from becoming too large, decrease the backup file retention period.

The syntax to run the updatebackupretaindays utility is as follows:

updatebackupretaindays <number of days>

In the following example, backup file retention time is changed to four days:

./updatebackupretaindays.sh 4

updatebackupretaindays 4

Manually create and delete backups

Even if you use automatic backups, there may be times when you want to create a backup for a specific purpose outside of the usual backup schedule, such as before upgrading the system or to create a secondary full backup in a different location.

If you disable automatic backups, it is recommended that you create manual backups on a regular basis.

You can use the deletebackup utility to delete backup files that you create for relational stores and object stores.

Run a utility to create a data store backup

You can use the backupdatastore utility to make a full backup of the specified data store. You may want to manually create a full backup before making a large number of changes to the data store or before upgrading the data store. Or you may want to create a backup to preserve a copy of the data in a particular state; for example, at the end of the first phase of a project.

Running the backupdatastore utility for a relational store always creates a full backup of the data store.

The first time you run the backupdatastore utility for the spatiotemporal big data store, a full backup is created. Because this type of data store can be very large, each time you run the backupdatastore utility after the first time, the utility only creates backup copies of data that was created since the last time you ran the utility.

The login you use to connect to the data store machine to run the backupdatastore utility must have read and write access to the data store backup location.

The syntax to run the backupdatastore utility is as follows:

backupdatastore [<backup file name>] [--store {relational | spatiotemporal | graph | object}] [--location <backup_arguments>] [--prompt {yes | no}]

If you do not specify the --location option, backup files are written to the data store's default backup location.

Provide a meaningful backup name for the file so you can find it when you want to restore the data. If you do not specify a name, the utility assigns a default name to the file. The default name is in the format datastorename-timestamp. For example, if the data store is named corpds and you create the backup on July 10, 2014, at 14:25:49:554 UTC, the backup file name is corpds-20140710142549554.

You will be asked to confirm that you want to create a backup. Type yes or y to proceed with backup creation.

Tip:

If you want to script manual backups, include a flag to suppress the confirmation prompt as in the following example:

backupdatastore --store relational --prompt no

In this example, the data store generates the backup file name. This is necessary in a script to ensure a unique backup file name.

Delete manual data store backups

When you no longer need a relational store or object store backup file that you created using the backupdatastore utility, you can delete them using the deletebackup utility. Before you delete the backup files, though, you must determine the file locations and names.

To find the locations and names of backup files for the relational store or object store that you created using the backupdatastore utility, and delete the files that you no longer need, complete the following steps:

Run the configurebackuplocaton utility with the --operation option set to list to get all the backup locations for a data store.

Note these locations for use in the next step.
Run the listbackups utility with each location to get the names of the backup files in each location that was returned by the configurebackuplocaton utility.

Note the file names to be deleted in the next step.

After you determine which backup files must be deleted, run the deletebackup utility for each location and file to remove the unneeded file.

For example, after you upgrade the relational store and confirm that all layers are performing as expected, you can delete the relational store backups you created before upgrading.

In this example, the relational store backup file preupgrade1104_bu is deleted from backup location /net/systemserver/backups:

./configurebackuplocation.sh  --operation list --store relational
Backups locations for relational store:
=====================================================================================
Type  Location                                         isDefault
=====================================================================================
fs    /net/ourserver/datastore/backups/rel                true
fs    /net/systemserver/backups                           false

./listbackups.sh --store relational --location "/net/systemserver/backups"

Backup_Name                      Status           Backup_Time         Mode
====================================================================================
phase1proj_bu                    BackupComplete   2023-03-08 14:12    manual
phase2proj_bu                    BackupComplete   2023-06-21 11:43    manual
preupgrade1104_bu                BackupComplete   2022-11-04 09:30    manual

./deletebackup.sh preupgrade1104_bu --store relational --location "/net/systemserver/backups"
You are attempting to delete backup 'preupgrade1104_bu'. This operation is irreversible.

Do you wish to continue (Yes or No)?yes

Operation completed successfully

In this example, the relational store backup file preupgrade1104_bu is deleted from backup location \\systemserver\backups:

configurebackuplocation  --operation list --store relational
Backups locations for relational store:
=====================================================================================
Type  Location                                         isDefault
=====================================================================================
fs    \\ourserver\datastore\backups\rel                true
fs    \\systemserver\backups                           false

listbackups --store relational --location "\\systemserver\backups"

Backup_Name                      Status           Backup_Time         Mode
====================================================================================
phase1proj_bu                    BackupComplete   2023-03-08 14:12    manual
phase2proj_bu                    BackupComplete   2023-06-21 11:43    manual
preupgrade1104_bu                BackupComplete   2022-11-04 09:30    manual

deletebackup preupgrade1104_bu --store relational --location "\\systemserver\backups"
You are attempting to delete backup 'preupgrade1104_bu'. This operation is irreversible.

Do you wish to continue (Yes or No)?yes

Operation completed successfully