CHGI Transition

To improve the sustainability of the CHGI cluster compute and storage infrastructure, the decision was made to integrate this infrastructure with the University of Calgary's High Performance Computing site managed by Research Computing Services (RCS). This will allow CHGI users to leverage the extensive RCS staff and infrastructure. For more information on RCS services, please see the RCS Home Page .

What does this really mean?

PIs, researchers, or students using any of the following storage or compute infrastructure will be contacted by analysts from the RCS team on moving their workflows to ARC:

• Synergy compute cluster
• Theia compute cluster
• Galaxy compute
• Storage on /gpfs and /tiered

If necessary, analysts from RCS will provide you with one-on-one time to support you in transitioning your workflows to ARC. RCS will continue to provide support to you and your workflows on ARC after the transition is complete.

Project Objectives

The primary objectives of this transition are to:

• Develop a transition plan for the equipment and technical services CHGI offers
• Define ongoing maintenance governance, processes, data privacy, retention and other related processes
• Transition data, users and workflows to RCS managed infrastructure

Status

User account status in CHGI and transition

What	Total CHGI Accounts Count	CHGI Accounts Active	CHGI Accounts Locked-out	With ARC Accounts	Without ARC Accounts	Logged In - Last 12 months	Logged In - More than a year ago	Not logged In	IT Account enabled	IT Account that is disabled	No IT Account
Baseline Dec 04, 2019	379	335	43	76	259	139	180	16	243	43	49
		88%	11%	23%	77%	41%	54%	5%	73%	13%	15%
Current month statistics	399	218	181	88	130	189	51	22	196	12	10
		55%	45%	40%	60%	87%	23%	10%	90%	6%	5%
Changes from baseline Dec/2019			20		138	12

Cluster status:

Hyperion - moved to ARC, users transitioned.

Theia - moved to ARC, users transitioned.

Crick - Restricted access.

Synergy - In process.

Background

The Center for Health Genomics and Informatics (CHGI) is an initiative that provides a wide range of next-generation genome sequencing services and access to high-performance bioinformatics for sequence analysis to all University of Calgary researchers. New researchers are provided a basic level of networking and storage to make use of CHGI services but also have the option of purchasing additional servers and storage to integrate within the CHGI network to upgrade their usage capabilities. Many of the pieces of the CHGI network equipment have been purchased by researchers and institutions.

The CHGI in collaboration with RCS is working on a planned transition of CHGI data, services and workflows to infrastructure managed by RCS. This will allow CHGI researchers to:

• Focus on core duties instead of managing IT services
• Open the potential for scalability in the infrastructure
• Standardize several common services to RCS offerings.

Approach

The project team interviewed 15 Principal Investigators and researchers in the Analysis phase to understand the current use of the equipment and services provided by CHGI. This engagement took four weeks during which we interacted with the CHGI manager, System Administrators and PIs that leverage CHGI's equipment for their research. An important aspect in the interviews was the investment made in equipment.

The project team took the information available and discussed the technical options based on current RCS practices.

Implementation

Tasks identified as low impact include:

• CHGI workflows and services that can be transitioned into existing RCS infrastructure
• CHGI Workflows and services with very little impact and costs.
• CHGI equipment which are off or almost off warranty and have low dependence on storage.

All other tasks, CHGI web related services, or workflows heavily dependent on the the IBM storage system will require deeper analysis and more preparation time. The longer transition period will give researchers extra time to prepare and integrate their processes into RCS infrastructure. Equipment and workflows requiring this extra work will remain in their current location.

Two years into the transition, a decision whether to physically move any remaining equipment to RCS infrastructure will be determined.

End of Warranty Issues

To address the inevitable problem of equipment falling off warranty and funding ceasing, clear communication to the equipment owners and researchers will offer two options if they wish to continue operations. Researchers can choose either to:

1. Transition their workflows onto existing RCS offerings in ARC.
2. Purchase new equipment under RCS guidelines.

Engagement and collaboration with RCS in this stage will assist in purchasing new equipment that can be integrated with the rest of the RCS infrastructure. These services and workflows will then be transitioned into standard RCS infrastructure, and the new RCS administrator will manage this by following the service levels described in a signed Operating Level Agreement (OLA). After the workflows have been transitioned, the off-warranty equipment would be decommissioned.

On Going

Continuous monitoring of the transition will guide the implementation and may improve the transition process.

Critical Success Factors

• Minimize impact to researcher’s workflows/processes
• Consolidate services and infrastructure

Scope

In Scope

• Develop a consistent process to transition users, data, workflows from CHGI to RCS infrastructure
• Define ongoing maintenance governance, processes, data privacy, retention and other related processes
• Transition Individual Researchers from CHGI to RCS infrastructure.

Out of Scope

• RCS and CHGI will agree when equipment is ready for decommissioning. CHGI is responsible to put in the requests to dispose of equipment.

Schedule/Milestones

Milestone	Dates
Alternatives Evaluation	Q2/2019
Implementation Plan	Q3/2019
Transition to ARC of appropriate users and workflows	Q3/2020
Analysis of remaining equipment and users	Q3/2020
Additional transition work - dependent on equipment and users remaining	Q4/2020