Status for 8/25/14 Monday check-in

Scheraga's group must confirm decision on (1) partitions AND (2) their capabilities and use.

ChemIT requires this information to set up the new Matrix. Soon, a lack of a decision will bring this project to a full-stop until the information is provided.

• Email sent 7/29/2014 5:30:05 PM to Czarek and Gia. No written confirmation, as of 8/25/14 (Monday).

Current idea:

On Synology system:

• /home (3TB total; on Synology)
  
  • Can be seen by compute nodes. Data stored on compute nodes until work is done, then moved to /home.
  • Data on /home removed when not actively needed for their cluster work. Can move the data to /storage, for safe-keeping and convenient access.
  • Data is backed up via EZ-Backup.
  • A new user gets 50GB, by default.
  • The process to request more space is through group designate.
• /storage (8TB total)
  
  • Can NOT be seen by compute nodes.
  • This is to store data related only to Scheraga research. No personal files.
  • Data is backed up via EZ-Backup.
  • The process to request more space is through group designate.
• TBD (1TB total)
  • To be used for either expanding /home or /storage at a later date (when we know which needs more space sooner than the other).

On internal 4TB regular hard drive:

• /notbackedup] (4TB total)
  
  • For when a researcher needs more temporary space than an individual compute node can provide.
  • Researcher must move data off as soon as they are done with it.
  • This data is NOT BACKED UP. Use only temporarily, and only when a researcher needs "surge" space.
  • No quotas. Thus, potential for researchers to "step on each other".
• Q: Make more robust by adding hard-ware RAID and a 2nd identical drive?
  • Cost of RAID card:
  • Cost of 2nd identical hard drive:

On internal 256GB SSD hard drive:

• OS
• Swap
• Applications for supporting the cluster
• Applications for researchers
  • /software

Drive is backed up via EZ-Backup.

Scheraga group must provide to ChemIT two (2) quota numbers for every active user (and info might inform partitions' minimum sizes, required above)

For each current researcher, a quota must separately be specified for their:

• /home
• /storage

ChemIT has provided Gia the current use and single quota for each researcher.

• This quota is a combination of both /home-type use and /storage-type use.

Gia has asked every researcher:

• Desired quota for /home?
• Desired quota for/storage?
• How much data to transfer to Matrix's /storage when it is made available (from their local hard drives, etc.)?

Many researchers have not replied to Gia.

• ChemIT will thus create a report to Gia with when they last logged in.

Scheraga group must approve process for ChemIT to copy researchers' files from old Matrix to the new Matrix.

Idea:

• Create empty /home directories for each user. Leave empty.
• ChemIT copies all a user's data in old Matrix to the new Matrix's "storage" partition. Each researcher selectively moves what is required to their new Matrix home directory.
• OUTCOMES
  • Clean /home directories, making restores and debugging much easier over time.
  • Pawel supports this approach. He estimates perhaps 1/2 - 1 hour work for each researcher.
• To do: Figure out dates and other timing, including complete downtime.

GPU

Yi must test.

Group documentation

Develop and communicate a community standard of practice.

• Where to host?
  • Example: CU's CIT Confluence (such as this page!). It's free and well supported by CIT.
• Who to create, organize, and maintain content and associated conventions?
  • Negotiate decisions among group, and ensure technical decisions are ones ChemIT can support.

Initial ideas from Pawel and Gia (documented by Oliver, as a favor):

Regulations:

1) Default quotas for new researchers.

• Example:

How does a user request a bigger quota?

• Document workflow, specifying who needs to approve what, and what info do they need to do that.
• Is there a difference between a temporary quota increase and one which is more permanent?

(2) Rules and expectations (behaviors) of group members

• Ex: Respond in a timely manner, and completely, to requests from group's designate.

(3) Support for researchers using Matrix.

• For a given question, how get it?
  • What questions go through group designate, and what goes directly to ChemIT, and how is that done?
• What response can one expect? 

Also may want to add:

• How-to's