AtlanticWave-SDX (AW-SDX) demonstration at Supercomputing conference (SC19)

The AmLight-ExP Network Engineering Team participated in multiple collaborative SC19 Network Research Exhibitions at the Supercomputing Conference (SC19) which took place on November 17-22, 2019 at the Colorado Convention Center, in Denver, Colorado.
AmLight-ExP and AtlanticWave-SDX offered the academic community 630Gbps of upstream bandwidth, network auto-recovery and dynamic provisioning, network programmability, network telemetry, integration with SENSE project’s distributed orchestrator, and 100G DTNs.

The demonstration/presentations took place at the Caltech booth.

Title: Global Petascale to Exascale Workflows for Data Intensive Science Accelerated by Next Generation Programmable SDN Architectures and Machine Learning Applications
Lead: Harvey Newman (Caltech)
Abstract: The demonstration includes several of the latest major advances in software defined and Terabit/sec networks, intelligent global operations and monitoring systems, workflow optimization methodologies with real-time analytics, and state of the art long distance data transfer methods and tools and server designs, to meet the challenges faced by leading edge data intensive experimental programs in high energy physics, astrophysics, climate and other fields of data intensive science. The key challenges being addressed include: (1) global data distribution, processing, access and analysis, (2) the coordinated use of massive but still limited computing, storage and network resources, and (3) coordinated operation and collaboration within global scientific enterprises each encompassing hundreds to thousands of scientists.
AmLight Express and Protect (AmLight-ExP) (NSF Award #1451018) has been supporting the LSST, LHC, and many high-throughput low latency experiments using a Software-Defined Networking (SDN) that includes the new Monet submarine cable and a 100G ring network. The AmLight SDN was built to enhance collaboration, research, and education between the U.S. and South America.

Title: Big Data Express project demonstration
Lead: Wnedji Wu (Fermilab)
Abstract: Big data has emerged as a driving force for scientific discoveries. To meet data transfer challenges in big data era, DOE’s Advanced Scientific Computing Research (ASCR) office has funded the BigData Express project (http://bigdataexpress.fnal.gov). BigData Express is targeted at providing schedulable, predictable, and high-performance data transfer service for DOE’s large-scale science computing facilities and their collaborators. In this demo, we use BigData Express software to demonstrate bulk data movement over wide area networks. The following features in BigData Express will be demonstrated:

• A peer-to-peer, scalable, and extensible model for data transfer services;
• A visually appealing, easy-to-use web portal;
• A high-performance data transfer engine;
• Orchestrating and scheduling of system (DTN), storage, and network (SDN) resources involved in the file transfers.
• On-Demand provisioning of end-to-end network paths with guaranteed QoS;
• Robust data transfer services provisioning through strong error handling mechanisms;
• Safe and secure data transfer services by using multiple security mechanisms;
• The interoperation between BigData Express and SENSE;
• Integration of BigData Express with scientific workflows.

The demonstrations and presentations are a collaborative effort from the following participating institutions:
California Institute of Technology (Caltech: High Energy Physics (HEP), Laser Interferometer Gravitational-Wave Observatory (LIGO), The Large Synoptic Survey Telescope (LSST), Caltech: Information Management Systems & Services (IMSS)), University of Southern California (USC), Florida International University (FIU), USC Information Sciences Institute (ISI), Georgia Institute of Technology (GeorgiaTech), Yale University, IBM, University of Maryland, Virnao, San Diego Supercomputer Center (SDSC), University of California San Diego (UCSD), Pacific Research Platform (PRP), Massachusetts Institute of Technology (MIT) , Energy Sciences Network (ESnet), National Energy Research Scientific Computing Center (NERSC), Lawrence Berkeley National Laboratory (LBNL), São Paulo State University (UNESP), Starlight Internet2, Johns Hopkins University, SURFnet, Fermi National Accelerator Laboratory (Fermilab), Argonne National Laboratory, University of Michigan, Northeastern University, Colorado State University, Tennessee State University, University of California Los Angeles (UCLA), Tata Institute of Fundamental Research (TIFR Mumbai), Corporation for Education Network Initiatives in California (CENIC: Pacific Wave), Academic Network at São Paulo (ANSP Brazil), National Education and Research Network (Rede Nacional de Ensino e Pesquisa – RNP), National University Network (Red Universitaria Nacional – REUNA Chile), Ciena.

About the SC Conference Series
Established in 1988, the annual SC conference continues to grow steadily in size and impact each year. Approximately 5,000 people participate in the technical program, with about 11,000 people overall.
SC has built a diverse community of participants including researchers, scientists, application developers, computing center staff and management, computing industry staff, agency program managers, journalists, and congressional staffers. This diversity is one of the conference’s main strengths, making it a yearly “must attend” forum for stakeholders throughout the technical computing community.
The technical program is the heart of SC. It has addressed virtually every area of scientific and engineering research, as well as technological development, innovation, and education. Its presentations, tutorials, panels, and discussion forums have included breakthroughs in many areas and inspired new and innovative areas of computing. (http://supercomputing.org/)