ISE – Backups

Depending on the needs of your organization, backups can be performed as on-demand or scheduled ‘kron’ jobs. This can be accomplished with either the CLI or GUI of the primary admin node (or monitoring node if you intend to backup log data) or a standalone node that’s assuming all personas. This post covers backups for both the 1.1.x and 1.2.x release of ISE.

Distributed deployments with 2000+ endpoints can take up to 3 hours to complete a backup, so plan accordingly.

A repository can be configured using the the local disk of an ISE node, tftp, ftp, sftp and nfs, or for read only, http(s) or the local cdrom drive. However, read-only repositories cannot be used for backup or restore jobs. Cisco recommends your repository size be between 10GB and 200GB depending on the number of endpoints in your deployment.

ise01v1-1x/admin(config-Repository)# url ?
  WORD  Enter repository URL, including server and path info (Max Size - 80)
  cdrom:  Local CD-ROM drive (read only)
  disk:   Local hard disk storage
  ftp:    URL using a FTP server
  http:   URL using a HTTP server (read only)
  https:  URL using a HTTPS server (read only)
  nfs:    URL using a NFS server
  sftp:   URL using a SFTP server
  tftp:   URL using a TFTP server

The following example demonstrates how to add a repository using sftp to allow backup and restore jobs to run. The repository server below (10.20.130.50) is a Linux machine allowing SFTP connections and is using the ‘cisco’ user’s home directory as the repository url.

ise01v1-1x/admin(config)# repository REPO-SERVER1
ise01v1-1x/admin(config-Repository)# url sftp://10.20.130.50/home/cisco
ise01v1-1x/admin(config-Repository)# user cisco password plain C1sc0123
ise01v1-1x/admin(config-Repository)# host-key host 10.20.130.50
host key fingerprint added
# Host 10.20.130.50 found: line 1 type RSA
2048 ef:58:df:7b:71:f4:45:22:b7:c0:be:26:bf:b0:84:7e 10.20.130.50 (RSA)

When configuring an sftp repository you must add a ‘host-key’. Basic connectivity to your repository server is required to successfully add the host-key fingerprint. Below is an example of a failed attempt to add a host-key.

ise01v1-1x/admin(config-Repository)# host-key host 10.20.130.50
%host-key add failed
% Failure occurred during request
% Warning: Host key of the server must be added using host-key command before sftp repository can be used.

Following the addition of a repository, connectivity and directory contents can be verified by issuing the “show repository REPOSITORY-NAME” command.

ise01v1-1x/admin# show repository REPO-SERVER1
% Repository is empty

Once we have verified the repository is reachable we can kick off an on-demand backup with the following syntax:

ise01v1-1x/admin# backup Test-Backup repository REPO-SERVER1 application ise encryption-key plain C1sc0123
% Creating backup with timestamped filename: Test-Backup-140602-1805.tar.gpg 

Backups can also be scheduled from the CLI using the “kron” syntax. The “kron policy-list” defines what command(s) to run and the “kron occurrence” defines the what is run (policy-list), at what time and if the operation should be repeated.

ise01v1-1x/admin(config)# kron policy-list Weekly_Backup
ise01v1-1x/admin(config-Policy List)# cli backup Test-Backup repository REPO-SERVER1 application ise encryption-key plain C1sc0123
ise01v1-1x/admin(config-Policy List)# exit
ise01v1-1x/admin(config)# kron occurrence KRON-JOB-01
ise01v1-1x/admin(config-Occurrence)# at 00:30 Saturday
ise01v1-1x/admin(config-Occurrence)# recurring 
ise01v1-1x/admin(config-Occurrence)# policy-list Weekly_Backup
ise01v1-1x/admin(config-Occurrence)# end

On-demand and scheduled backups can be verified as completed by issuing the “show backup history” command.

ise01v1-1x/admin# show backup history 
Mon Jun  2 18:29:30 CDT 2014: backup Test-Backup-140602-1805.tar.gpg to repository REPO-SERVER1: success
Tue Jun  4 16:21:20 CDT 2013: backup test-backup-130604-1607.tar.gpg to repository REPO-SERVER1: success
Tue Jun  4 16:55:25 CDT 2013: backup test-backup-130604-1643.tar.gpg to repository REPO-SERVER1: success
Tue Jun  4 17:11:07 CDT 2013: backup test-backup-130604-1656.tar.gpg to repository REPO-SERVER1: success
Tue Jun  4 17:37:18 CDT 2013: backup test-backup-130604-1725.tar.gpg to repository REPO-SERVER1: success
Tue Jun  4 18:09:20 CDT 2013: backup test-backup-130604-1757.tar.gpg to repository REPO-SERVER1: success

If a repository is not configured, one can be added by navigating to “Administration” > “Maintenance” on the Primary Admin or Standalone ISE node.

Next, click “Repository” in the “Maintenance” pane on the left. Click “Add” to configure a repository appropriate for your environment.

Even when creating an SFTP repository in the GUI, you will need to jump in the CLI and manually add a host-key before it can be used.

Adding a host-key to the repository REPO-SERVER1 in the CLI:

ise01v1-1x/admin(config)# repository REPO-SERVER1
ise01v1-1x/admin(config-Repository)# host-key host 10.20.130.50
host key fingerprint added
# Host 10.20.130.50 found: line 1 type RSA
2048 ef:58:df:7b:71:f4:45:22:b7:c0:be:26:bf:b0:84:7e 10.20.130.50 (RSA)

After we have verified our repository is in place, expand “Data Management > Administrative Node (or Monitoring Node)” and click “Full Backup On Demand” in the “Maintenance” pane on the left. Simply enter the backup file name, destination repository, encryption key, and whether or not you’d like to backup the ADE-OS in addition to the application data.

Unfortunately, in ISE 1.1.x, there is no indication that a backup is in progress or it’s current status within the GUI or CLI. However, if you’re impatient and attempt to kick off another on-demand backup, ISE will inform you that there is already one in progress.

Additionally, if you log into the CLI and attempt to kick off another on-demand backup, ISE will inform you there as well.

ise01v1-1x/admin# backup test-backup repository REPO-SERVER1 application ise encryption-key plain C1sc0123
% A backup or restore is already in progress
ise01v1-1x/admin# show backup history 
backup history is empty

The history of your backups can be found by navigating to “Operations > Reports > System”, expanding “Data Management > Administration Node (or Monitoring Node)” and clicking “Backup History”. Only successful or failed backups will show within this report.

In the 1.2.x release, backups have gone through quite a (much needed) change. Repositories created in the CLI are exclusive to an ISE node and are not available in the GUI. Additionally, scheduled backups (kron jobs) can no longer be performed from the CLI. The repository server in the 1.2.x environment (10.35.200.50) is a Linux machine allowing SFTP connections and is using the ‘cisco’ user’s home directory as the repository url.

ise01v1-2x/admin(config)# repository REPO-SERVER1
ise01v1-2x/admin(config-Repository)# url sftp://10.35.200.50/home/cisco
% Warning: Repositories configured from CLI cannot be used from the ISE web UI and are not replicated to other ISE nodes. If this repository is not created in the ISE web UI, it will be deleted when ISE services restart.

ise01v1-2x/admin(config)# kron policy-list Commands_For_KRON
ise01v1-2x/admin(config-Policy List)# cli backup Test_Backup_2 repository REPO-SERVER2 ise-config encryption-key plain C1sc0123
% Error: Scheduling backups only supported from ISE Backup/Restore UI.

Creating a repository is the same as 1.1.x; Navigate to “Administration” > “Maintenance” on the Primary Admin or Standalone ISE node, click “Repository” in the “Maintenance” pane on the left and click “Add” to configure a repository.

However, even when creating an sftp repository in the GUI, you must still use the CLI to add a host-key.

The syntax has changed to “crypto host_key add host [IPaddress/hostname]”:

ise01v1-2x/admin# crypto host_key add host 10.35.200.50
host key fingerprint added
# Host 10.35.200.50 found: line 1 type RSA
2048 ef:58:df:7b:71:f4:45:22:b7:c0:be:26:bf:b0:84:7e 10.35.200.50 (RSA)

Data Management (1.1.x) has been replaced with the “Backup & Restore” page which provides greater detail such as progress percentages and a clear distinction between configuration(includes ADE-OS) backups and operational backups. Navigate to “Administration” > “Backup & Restore” and click “Backup Now” to perform an on-demand backup. Backup names cannot include dashes and can now only contain characters, numbers, underscores and periods.

When initiating an on-demand backup you are now presented with an indication of progress, the option to cancel a backup in progress as well as additional status details that can be seen below.

Similar detail is now provided in the CLI as well:

ise01v1-2x/admin# show backup status 
%% Configuration backup status
%% ----------------------------
%      backup name: Test_Backup2
%       repository: REPO-SERVER2
%       start date: Mon Jun 02 21:04:01 CDT 2014
%        scheduled: no
%   triggered from: Admin web UI
%             host: ise01v1-2x.alice.lab
%           status: Backup is in progress...
%       progress %: 20
% progress message: Backing up ISE Configuration Data

%% Operation backup status
%% ------------------------
%  No data found. Try 'show backup history' or ISE operation audit report

A scheduled backup can be created under the same “Backup & Restore” area for either configuration or operational backups.

Backup history can be viewed by selecting a given repository next to “History for Repository” with the option to restore.

An on-demand backup can still be initiated from the CLI with the choices being configuration or operational data backup.

ise01v1-2x/admin# backup Test_Backup2 repository REPO-SERVER2 ?
  ise-config       ISE Configuration data(includes ADE-OS)
  ise-operational  ISE Operational data

Fortunately, the backup status is now displayed in the CLI to provide a clear indication of backup status and progress when compared to 1.1.x.

ise01v1-2x/admin# backup Test_Backup2 repository REPO-SERVER2 ise-config encryption-key plain C1sC0
% Creating backup with timestamped filename: Test_Backup2-CFG-140603-0019.tar.gpg 
% backup in progress: Starting Backup...10% completed 
% backup in progress: Validating ISE Node Role...15% completed
% backup in progress: Backing up ISE Configuration Data...20% completed
% backup in progress: Backing up ISE Logs...45% completed
% backup in progress: Completing ISE Backup Staging...50% completed
% backup in progress: Backing up ADEOS configuration...55% completed 
% backup in progress: Moving Backup file to the repository...75% completed 
% backup in progress: Completing Backup...100% completed 

ise01v1-2x/admin# show backup status
%% Configuration backup status
%% ----------------------------
%      backup name: Test_Backup2
%       repository: REPO-SERVER2
%       start date: Tue Jun  3 00:19:11 CDT 2014
%        scheduled: no
%   triggered from: CLI
%             host: 
%           status: backup Test_Backup2-CFG-140603-0019.tar.gpg to repository RE
PO-SERVER2: success

%% Operation backup status
%% ------------------------
%  No data found. Try 'show backup history' or ISE operation audit report
ise01v1-2x/admin# show backup history 
Mon Jun  2 21:19:31 CDT 2014: backup Test_Backup2-CFG-140602-2104.tar.gpg to rep
ository REPO-SERVER2: success
Tue Jun  3 00:31:56 CDT 2014: backup Test_Backup2-CFG-140603-0019.tar.gpg to rep
ository REPO-SERVER2: success