CHGI Transition
To meet building code requirements, the CHGI Data Centre (DC) in HSC B151 must be decommissioned. To facilitate this change, the storage and compute capabilities housed in the DC are being transferred to the University of Calgary's High Performance Computing site managed by Research Computing Services (RCS).
What does this really mean?
PIs or researchers using any of the following storage or compute infrastructure will be contacted by analysts from the RCS team on moving their workflows to ARC:
- Synergy compute cluster
- Theia compute cluster
- Galaxy compute
- Storage on /gpfs and /tiered
If necessary, analysts from RCS will provide you with one-on-one time to support you in transitioning your workflows to ARC. RCS will continue to provide support to you and your workflows on ARC after the transition is complete.
Project Objectives
The primary objectives of this transition are to:
- Develop a transition plan for the equipment and technical services CHGI offers
- Define ongoing maintenance governance, processes, data privacy, retention and other related processes
- Transition data, users and workflows to IT/RCS managed infrastructure
Background
The Center for Health Genomics and Informatics (CHGI) is an initiative that provides a wide range of next-generation genome sequencing services and access to high-performance bioinformatics for sequence analysis to all University of Calgary researchers. New researchers are provided a basic level of networking and storage to make use of CHGI services but also have the option of purchasing additional servers and storage to integrate within the CHGI network to upgrade their usage capabilities. Many of the pieces of the CHGI network equipment have been purchased by researchers and institutions.
The CHGI in collaboration with RCS is working on a planned transition of CHGI data, services and workflows to infrastructure managed by RCS. This will allow CHGI researchers to:
- Focus on core duties instead of IT services
- Open the potential for scalability in the infrastructure
- Standardize several common services to IT’s offerings.
Approach
The project team interviewed 15 Principal Investigators and researchers in the Analysis phase to understand the current use of the equipment and services provided by CHGI. This engagement took four weeks during which we interacted with the CHGI manager, System Administrators and PIs that leverage CHGI's equipment for their research. An important aspect in the interviews was the investment made in equipment.
The project team took the information available and discussed the technical options based on current IT’s practices. The equipment, services and workflows have been categorized into 2 groups which was determined based on their ease of transition: Quick Wins and Medium to High Complexity.
Implementation
Tasks identified as Quick Wins will be used as the first step in the implementation and will be used build trust with the CHGI user base. Quick Wins include:
- CHGI workflows and services that can be easily transitioned into existing IT/RCS infrastructure
- CHGI Workflows and services with very little impact and costs.
- CHGI equipment which are off or almost off warranty and have low dependence on storage.
Quick Wins workflows and data are transitioned into ARC following the schedule defined below.
All other tasks, CHGI web related services, or workflows heavily dependent on the IBM storage will be identified as Medium to High Complexity and will be implemented 12 months into the transition. The longer transition period will give researchers extra time to prepare and integrate their processes with IT infrastructure. Equipment and workflows under this category will remain in their current location.
Two years into the transition, a decision whether to transition the remaining equipment physically to IT needs to be discussed.
End of Warranty Issues
To address the inevitable problem of equipment falling off warranty and funding ceasing, clear communication to the equipment owners and researchers will offer two options if they wish to continue operations. Researchers can choose either to:
- Transition their workflows onto existing RCS offerings in ARC.
- Purchase new equipment under RCS guidelines.
Engagement and collaboration with RCS in this stage will assist in purchasing new equipment that can be integrated with the rest of the RCS infrastructure. These services and workflows will then be transitioned into standard RCS infrastructure, and the new RCS administrator will manage this by following the service levels described in a signed Operating Level Agreement (OLA). After the workflows have been transitioned, the off-warranty equipment would be decommissioned.
On Going
Once the first wave of transition happens, a revision to the plan will be made to decide whether to forklift part or all the remaining equipment to an IT data center.
Critical Success Factors
- Minimize impact to researcher’s workflows/processes
- Consolidate services and infrastructure
Scope
In Scope
- Develop a consistent process to transition users, data, workflows from CHGI to IT/RCS infrastructure
- Define ongoing maintenance governance, processes, data privacy, retention and other related processes
- Transition Individual Researchers from CHGI to IT/RCS infrastructure.
Out of Scope
- IT and CHGI will agree when equipment is ready for decommissioning. CHGI is responsible to put in the requests to dispose of equipment.
Schedule/Milestones
| Milestone | Dates |
|---|---|
| Alternatives Evaluation | Q2/2019 |
| Implementation Plan | Q3/2019 |
| Transition to ARC of identified Quick Wins | Q3/2020 |
| Analysis of remaining equipment and users | Q3/2020 |
| Additional transition work - dependent on equipment and users remaining | Q4/2020 |