| Parallel File System | Lustre |
| Lnet Nodes | 30 |
| Storage Array | Sonexion 3000 |
| Storage Racks | 6 Racks + 5 Tape Library Frames |
The storage for the Cray system consists of 5 Sonexion 3000 Arrays which provides high performance Lustre file systems for User Home and Scratch areas respectively.
Sonexion file systems are made of two basic building blocks: scalable storage units (SSUs) and a metadata management unit (MMU). Each SSU houses disk drives and two OSSs that are connected to an even or odd top of-rack IB switch and then to the system storage network The Cray system consists of 30 Lnet Router Nodes which routes traffic between Sonexion Lustre Servers on the IB Network and the Lustre Clients on the Aries HSN (High Speed Network).
Below is the Configuration Details for each Storage Subsystem:
Storage Subsystem for Home
One MMU and 19 Scalable Storage Units (SSU).
2 MDS (Active/Passive) and 38 Object Storage Servers (OSS) nodes in Active/Active Configuration
The filesystem data is stored on 6TB ,7.2K RPM HDDs on GridRAID (RAID 6)
The filesystem metadata is stored on 900GB,10K RPM drive (RAID 10).
Storage Subsystem for Scratch
One MMU and 3 Scalable Storage Units (SSU).
2 MDS and 6 OSS housed in a single Sonexion Base rack.
The Scratch filesystem data is stored on 4TB ,10K RPM HDDs on GridRAID (RAID 6)
Filesystem metadata is stored on 900GB,10K RPM drive (RAID 10).
Parallel Filesystems
For Home - Lustre with a Useable capacity 6,300 TB and Sustained throughput of 152 GB/s.
For Scratch - Lustre with a useable capacity 673 TB and sustained throughput of 34 GB/s.
Analyse :
The 8 utility nodes are apt for pre and post data processing needs of the HPC users.
Archive:
The backup and archiving is enabled with two Cray tiered Adaptive Storage (TAS) systems .
The TAS Solution is paired with the TAS Connector for Lustre to provide a unique and tightly
Integrated solution for tiered data management directly from the Lustre filesystem.
The details of the HSM/Archive Tier is as below :
Total usable archival space of 16.86PB.
Storage Array Disk Cache of 233TB for HSM provided by a Netapp E5600 series device.
6 SpectraLogic Tfinity Tape Libray Frames
48 LTO-7 Drives
3640 LTO-7 Media/Slot
Versity VSM 1.0 for HSM
Cray TAS racks details :
2 Metadata Controllers
2 Connection Managers (CMM)
4 Connector Agents (CMA)
2 External Gateways
Tape Library
It also has Spectra Logic TFinity tape library for the data backup with 48 number of drives LTO-7, 3640 media slots LTO7, with HSM Solution which internally uses Versity Software and gives uncompressed capacity of 16.86 PiB
HSM Archive system
Cray TAS HSM Solution ensures data fluidity across storage tiers i.e data movement from the User Home areas to Tape Libraries and vice versa without affecting users or running applications. It has NetApp E5600 Storage Array providing a Disk cache with the useable space of 233 TiB and sustained throughput of 20 GB/s.