• Created by user-29411, last updated on Oct 14, 2013 1 minute read

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 6 Next »

Add true backup functionality to the existing cluster. Discussion started Oct. 2013.

Challenges to doing backups: The problem to solve

The quantity and number of files can put demand on the headnode and may take too long

  • Will the backup put too much demand on the computer, competing with the computer's primary research purpose?
  • Will the backups take too long?

The number of files

  • 100's of millions of small files, along with "normal" files.

The amount of data

  • 3.1TB of data

Cost is an issue since note originally budgeted.

Goals

Capability to restore system and data hardware to all-new hardware

Deal with various failures, such as:

  • file system corruption#
  • hard disk failures
  • RAID controller card failures
  • motherboard failure
  • other server hardware of sub-system failure
  • fire damage of room*
  • water damage of room*
  • theft*
  • malicious incursion into the system*#

* Requires an off-site copy to be effective.

# May require versioning to get prior copies of since-corrupted data.

Ability to have some versioning, if not too expensive an add-on.

Options

Option1: In-room, off-box, copy

HDs in a dedicated computing box.

Option1: Out of room

HDs in a dedicated computing box.

Hosted file service.

EZ-Backup.

  • $850-1,650/yr, depending on level of file compression.

Ideas to consider for above options

Software to sync data, ideally with versioning

EZ-Backup

CrashPlan

Compare

How CrashPlan backup works (technical)

Real-time Backup Version Retention

Backup Frequency and Version Retention

Backing Up Very Large File Selections

Use software that backs up changes to the partition, not looking at files per se.

  • No labels