High Bandwidth Distributed HPC

Federated High-Performance Computing Platform Across Asia-Pacific

WP4 Project 2nd Call 4 Countries HPC Federation

Project Overview

The High Bandwidth Distributed HPC project represents a revolutionary approach to scientific computing, creating a federated high-performance computing platform that seamlessly links Korea, Australia, Malaysia, and Pakistan. This cutting-edge infrastructure enables collaborative research through shared GPU/CPU resources, advanced ScienceDMZ architecture, container cloud services, and specialized AI/bioinformatics applications.

Multi-Country HPC Federation

🇰🇷
Korea
KISTI
Leading HPC infrastructure and coordination, providing advanced computing resources and technical expertise for the distributed platform.
🇦🇺
Australia
NCI
Contributing high-performance computing capabilities and research expertise, enabling large-scale scientific computations and data analysis.
🇲🇾
Malaysia
Perdana University, UPM
Focusing on bioinformatics applications and medical research, providing specialized computing resources for genomics and healthcare.
🇵🇰
Pakistan
SIBAU
Developing medical AI applications and building local capacity in scientific computing and platform management.

Project Objectives

TEIN-Integrated Distributed Platform

Build a comprehensive, container-based distributed HPC platform that leverages TEIN infrastructure for seamless connectivity and resource sharing across participating countries.

Federated ScienceDMZ Infrastructure

Enable high-speed data exchange through federated ScienceDMZ nodes, ensuring optimal performance for data-intensive scientific applications and research collaborations.

AI & Bioinformatics Applications

Support collaborative AI and bioinformatics applications, enabling researchers to tackle complex computational challenges in medical research and genomics.

Local Capacity Building

Strengthen local expertise in scientific computing and platform management through comprehensive training programs and knowledge transfer initiatives.

HPC Platform Architecture

The distributed HPC platform integrates cutting-edge technologies to create a unified, high-performance computing environment that spans multiple countries and institutions, enabling unprecedented collaboration in scientific research.

🖥️
Distributed DTN Nodes
High-performance Data Transfer Nodes enabling rapid, secure data exchange between research institutions across the federated network.
☁️
Container Cloud Services
Kubernetes-based container orchestration providing scalable, flexible deployment of applications and services across the distributed infrastructure.
🔐
Secure Access Management
Keycloak-based identity federation ensuring secure, authenticated access to computing resources while maintaining data privacy and security.
💾
Distributed Storage
Ceph storage clusters providing reliable, scalable data storage solutions with built-in redundancy and high availability across nodes.
📊
Real-time Monitoring
Comprehensive monitoring and analytics systems providing real-time insights into system performance, resource utilization, and application status.
🧬
ScienceDMZ Framework
Specialized network architecture optimized for large-scale scientific data transfers and high-performance computing workloads.

Key Activities & Implementation

ScienceDMZ Node Establishment

Successfully established distributed DTN-enabled ScienceDMZ nodes across all participating countries, creating a high-performance data transfer infrastructure for scientific computing applications.

Federated Computing Environment

Developed comprehensive federated computing environments using Kubernetes for orchestration, Ceph for distributed storage, and Keycloak for secure identity management across the platform.

Medical AI Training in Pakistan

Conducted specialized training and research programs focused on medical AI applications in Pakistan, building local expertise in healthcare-focused computational research.

Bioinformatics Research in Malaysia

Implemented comprehensive bioinformatics research programs in Malaysia, enabling advanced genomics research and computational biology applications on the distributed platform.

Blockchain-based ID Federation

Integrated blockchain-based identity federation systems to enhance security and enable seamless authentication across the distributed computing environment.

Key Outcomes & Achievements

4 Countries Connected
100% Platform Operational
5 Partner Organizations
24/7 High-Speed Access

Technical Achievements

  • Fully Operational Distributed Platform: Successfully deployed and validated a distributed HPC platform spanning four countries, enabling seamless resource sharing and collaborative computing capabilities.
  • High-Speed Data Exchange Validation: Demonstrated secure access and high-speed data exchange capabilities across the federated network, ensuring optimal performance for scientific applications.
  • Local Team Training: Comprehensive training programs delivered to local teams in advanced computing infrastructure, building sustainable expertise in HPC management and applications.
  • AI & Genomics Applications: Successfully deployed and validated AI applications for medical research and bioinformatics workflows for genomics research across the distributed platform.

Research Impact

Medical AI Advancement

Enabled breakthrough research in medical AI applications, providing computational resources and expertise for healthcare innovation in Pakistan and regional partners.

Bioinformatics Research

Facilitated advanced genomics research and computational biology studies in Malaysia, contributing to scientific understanding in life sciences and healthcare.

Cross-Border Collaboration

Established a framework for seamless international research collaboration, enabling scientists to share resources and expertise across institutional and national boundaries.

Technology Transfer

Successfully transferred advanced HPC technologies and expertise to participating countries, building long-term capacity for scientific computing and research.

Challenges & Strategic Solutions

Challenges Overcome

Hardware Procurement Delays: COVID-19 pandemic and flooding in Pakistan caused significant delays in equipment procurement and delivery, requiring adaptive project management and timeline adjustments.
Customs Clearance Issues: Complex international shipping and customs procedures slowed node installation and deployment across different countries with varying regulations.
Infrastructure Variability: Power and resource variability across participating countries required adaptive deployment strategies and flexible system configurations.

Strategic Solutions

Flexible Deployment Strategy: Implemented adaptive deployment approaches that accommodated varying timelines and local conditions while maintaining platform integrity and performance.
Enhanced Coordination: Established robust international coordination mechanisms to streamline equipment procurement and customs clearance processes across borders.
Adaptive Architecture: Designed platform architecture to accommodate varying power and resource conditions, ensuring reliable operation across diverse infrastructure environments.

Contributing to Sustainable Development Goals

SDG 9 Industry, Innovation and Infrastructure
SDG 4 Quality Education
SDG 17 Partnerships for the Goals

Future Directions & Expansion

Integration & Scaling

The project roadmap includes comprehensive integration with APAN/APBioNet networks, expanding the platform's reach and capabilities to serve a broader research community across the Asia-Pacific region. This integration will enhance collaborative opportunities and resource sharing among regional research institutions.

Enhanced Use Cases

Future development will focus on expanding use case coverage and HPC network scaling, incorporating emerging computational needs in areas such as climate modeling, materials science, and advanced AI applications. The platform will evolve to support increasingly complex and diverse scientific computing requirements.

Vision for Distributed Scientific Computing

The High Bandwidth Distributed HPC project is establishing a new paradigm for international scientific collaboration, where researchers can seamlessly access and share computational resources across continents. This foundation will accelerate scientific discovery, enable breakthrough research in critical areas like healthcare and climate science, and foster unprecedented levels of international cooperation in the pursuit of knowledge.

Sustainable Development

Multilingual Training Resources

Develop comprehensive, multilingual training materials and documentation to support broader adoption and ensure accessibility across diverse linguistic communities.

Open Dataset Development

Create and maintain open datasets for research applications, enabling researchers worldwide to access high-quality data for scientific investigations and validations.

Regional Network Expansion

Extend the platform to additional countries and institutions across the Asia-Pacific region, creating a truly comprehensive distributed computing ecosystem.

Advanced Research Applications

Support next-generation research applications in emerging fields such as quantum computing simulation, advanced AI model training, and large-scale data analytics.