SAN and Virtual Machines

May 1, 2007

We've been quietly doing a lot of work on the Library's technical infrastructure the last several months and I've been remiss in letting you all know about it.  The other day in conversation I described our situation as "upgrading and/or replacing virtually everything the Library's systems depend on."  There's so much going on in fact that I can't possibly describe it all in one message.  Let me know if you have questions about these topics or are interested in hearing more details. 

Storage Area Network (SAN)

After months of product investigations, needs gathering, technical demos, and the bid process, we have purchased and installed a Compellent brand SAN in the Main Library.  For years the lack of flexible SAN storage has hampered our abilities to provide and reallocate data storage as needs continually grow and change.  We've been on the edge of being able to justify the expense of a SAN and the Open Content Alliance and Illinois Harvest digitization projects finally pushed us over that edge to make it a necessity.

Our initial configuration provides 12 Terabytes raw data storage, and can be greatly expanded (at additional cost, of course).  The SAN we chose supports both fiber-channel and iSCSI connections, allowing us to dynamically allocate chunks of that storage to any of our servers, including machines located outside this building, and have that storage work just like locally-attached disks.  There are other nifty features like no-overhead snapshots, dynamic storage virtualization, and data progression that I don't have space to describe here.  (The Grainger Library has had a SAN for several years with some of the same basic capabilities, but it isn't capable of supporting central Library IT infrastructure.)

Virtual Machines (VMs)

We've purchased and are running a product called VMware ESX.  Server virtualization allows us to dynamically create, reconfigure, destroy, and move nearly any type of server.  You may have heard about or even seen workstation virtualization software (VMware workstation for Windows, Parallels for Mac OS, etc.)  These tools let you do things like run Windows on a Mac, or Linux and Windows XP inside Windows 2003.  This is very similar technology to those, but with more advanced capabilities.

With this technology, it's possible to fit quite a number of "servers" into a small space.  This saves on energy costs, hardware troubleshooting, hardware costs. 

We are now actively upgrading and reconfiguring many production servers into the new environment of using virtual servers with all their data storage on the SAN.  In that configuration, we gain better capabilities to support production services even if server hardware fails.  In several failure situations, such production VMs will automatically move to another node in the cluster if one hardware node fails. 

Example: The recent migration of the primary file server (LibSys5) was just such a case.  So now every time you use a file on the G: or H: drives in the Library, you're making use of a virtual server and the SAN.

It's important to understand that we aren't investing time and money in these upgrades and new technologies just for the fun of it.  (Occasionally they can be fun, but not often enough.)  These infrastructure projects we're working on are nearly all interrelated.  I've mentioned a benefit of combining VMs with SAN storage above.  Other examples will be described in succeeding issues of IT Infrastructure News.

Coming Infrastructure News Topics