Active Data Storage – User Guide

Overview

Active Data Storage (ADS) provides projects with reliable mid-scale storage at an affordable cost.  Use models include: online backups, storage for the hosting of large data sets, operational data storage and scratch space for analysis, mid-term data storage for archival needs and more. The ADS offers scalable storage from a few terabytes to over a petabyte of storage exported as a private file system for each project available through multiple protocols with good transfer rates. The storage provides high-availability and high-reliability through the use of standard RAID technologies and the use of hardware providing redundant paths to tolerate the loss of a controller or entire server.

Hardware

The Active Data Storage hardware is comprised of 2 Dell 730XD servers  and 2 Dell 630 servers each with 20 cores/256GB memory and 10GigE network interfaces and NetApp E2700 storage system with 12G SAS connections to the disk environment. There are currently 2 drawers with 60 4TB disks each for a total of 480TB of raw storage. The file system is the high performance General Parallel File System (GPFS) from IBM. The servers run the Red Hat 6 operating system.

Active data service storage overview

For security purposes, the disk environment is encrypted. The current encryption model is not FERPA or HIPAA compliant.  The NetApp storage system uses a public/private key encryption on the individual drives in the array.  The current encryption algorithm is AES-256.  The private key is maintained on the controllers and the public keys get written to each of the individual drives in the chassis.  At boot and/or drive replacement, the keys get paired to make sure they are valid.  Disks with an invalid key are in-accessible.

Access and Accounts

Please contact Leslie Froeschl for any questions regarding pricing, accounts, and access to ADS.

Policies

  • Preventive Maintenance/Downtimes:
    Preventive maintenance (PM) on the ADS will be scheduled as needed, but not more often than once per month on average, and outside of normal business hours if at all possible. ADS Operators will provide a seven day advance notice for PM periods whenever possible (if the timing of a PM is inconvenient, they will try to accommodate). However, the Operators reserve the right to take emergency downtimes for security reasons or hardware problems as needed.
  • Backups:
    Backup services are not available.  While ADS Operators will do everything to make the service as reliable as possible, having only one copy of the data is a risk. So no data on the ADS system should be considered safe with a single copy. ADS customers are encouraged to back up their data.
  • Mirroring:
    Mirroring your data within the ADS system is an option. However, your data would still be only on one system. This would be part of your storage amount that you purchase.</li
  • Snapshots:

The ADS system offers a snapshot option for your data. This is NOT considered a backup, but an oops option if files are accidentally deleted or overwritten.   There are many options for snapshots, but the two main questions are: how often do you want a snapshot and how long should the snapshots be kept? They are part of the file system and therefore part of the storage that you purchase – so this can impact how often you take snapshots and how long you keep them. A typical option would be to snapshot each day, and keep them for 30 days.

Access Options

For projects requiring persistent access to data on a local system/server:

Connect to Server Option:

Currently the ADS service is for Illinois staff and faculty to login and manage their data. It uses the university active directory infrastructure for security login. The storage can be exported and mounted on any machine in the UofI machine environment.  If you have outside UofI access to your machine that is mounting the storage from ADS, your content would be accessible to third parties outside of the UofI system.  The ADS mission doesn’t include the security of user storage once it’s mounted on a non-IT maintained server system, however.  It’s your storage, and if you put it on your machine, you can give access to whomever you wish.

DISCLAIMER: No security guarantees can be made by ADS operators on these options since access is outside the hands of ADS.

Data Transfer Options

System Status Updates

Should ADS experience any problems, updates will be posted to the campus system status page at https://status.illinois.edu/SystemStatus/jsp/current_events.jsp.

Support

For any questions or problems, contact the Active Data Storage support team.