Challenges to doing backups: The problem to solve

The quantity and number of files can put demand on the headnode and may take too long

• Will the backup put too much demand on the computer, competing with the computer's primary research purpose?
• Will the backups take too long?

The number of files

• 100's of millions of small files, along with "normal" files.

The amount of data

• 3.1TB of data

Cost is an issue since note originally budgeted.

Goals

Capability to restore system and data hardware to all-new hardware

Deal with various failures, such as:

• file system corruption#
• hard disk failures
• RAID controller card failures
• motherboard failure
• other server hardware of sub-system failure
• fire damage of room*
• water damage of room*
• theft*
• malicious incursion into the system*#

* Requires an off-site copy to be effective.

# May require versioning to get prior copies of since-corrupted data.

Ability to have some versioning, if not too expensive an add-on.

Options

Option1: In-room, off-box, copy

HDs in a dedicated computing box.

Option1: Out of room

HDs in a dedicated computing box.

Hosted file service.

EZ-Backup.

• $850-1,650/yr, depending on level of file compression.

Ideas to consider for above options

Software to sync data, ideally with versioning

EZ-Backup

CrashPlan

Compare

• http://www.crashplan.com/consumer/compare.html

How CrashPlan backup works (technical)

• http://support.crashplan.com/doku.php/articles/how_backup_works

Real-time Backup Version Retention

• http://support.crashplan.com/doku.php/feature/version_retention_and_backup_frequency

Backup Frequency and Version Retention

• http://support.crashplan.com/doku.php/reference/version_retention_and_backup_frequency

Backing Up Very Large File Selections

• http://support.crashplan.com/doku.php/how_to/seed_archive#backing_up_very_large_file_selections

Use software that backs up changes to the partition, not looking at files per se.