Join us for a one-day conference discussing the challenges of building and supporting Hybrid Clouds powering scientific research.
We are IT infrastructure professionals managing Compute / Network / Storage in support of research activities; this is a peer-led and peer-funded event — no vendor support, no vendor marketing. Come spend a day in Seattle sharing your experiences, posing the hard questions, and kicking around current solutions.
Cloud adoption in the research space lags the commercial space: Why? What makes research difficult to port to the Cloud? Where have we been successful? Where have we struggled? What lessons can we learn from each other about what works and what doesn’t? What lessons from our experience can the Cloud providers glean in order to acquire not only more of our business but also to acquire a competitive edge in expanding their offerings to their commercial customers?
The 2021 conference will be rescheduled, the new event date is to be determined.
Please see our White Paper posted below.
How scientists are using the Public Cloud without really trying – opportunities / challenges of workflow managers, technical requirements for implementation, progress to date, and next steps.
How the Broad Institute manages billing and cost controls for a wide variety of research projects with different funding sources across multiple public clouds
Chargebacks and cost control across numerous budgets lay the foundation for R&D using the Cloud; on the technical side, deploying and favoring serverless over containers has enabled greater agility leading to increased Cloud adoption within the private sector.
The nature of academic research presents specific challenges when adopting or supporting new technologies. Initiating a conversion to cloud computing represents one of the biggest stumbling blocks for academic researchers who have primarily used local computing resources throughout their careers. What are the conceptual and technical skills necessary for successful adoption of cloud computing? How can we manage expectations related to usage of cloud-based resources? How do you assess what your community needs to implement cloud-based options, and then respond with relevant resources? In this talk, I’ll discuss my experiences developing computational training for a large, heterogeneous community of researchers, including common impediments to both technological and cultural change, and suggest ways to facilitate community building as a way to alleviate these challenges.
Current state of our use of various cloud platforms in support of research, education, and controlled access data projects, with a particular glance at how we use both on-prem and cloud resources to accommodate seasonal bursts in HPC demand.
Scientific research faces challenges when it looks to the Cloud. Yes, elasticity is attractive and scaling beyond what we can host on-prem is tantalizing. However, Cloud economics dictate that increased resource consumption incurs increased costs, which suits the retail world neatly: more people buying your widgets leads to more revenue which pays the growing Cloud bill. But consumption of compute / network / storage in research tends to be poorly correlated with a revenue stream; this mismatch acts as an early barrier to leveraging the Cloud to support research activities.
Retooling our expertise to move from traditional programming techniques to Cloud-native paradigms (e.g. POSIX file semantics to object storage manipulation) requires retraining both IT and scientific staff. The Cloud comes with security defaults which are hard to replicate on-prem: how will we evolve our security models, handle compliance, and communicate risk in the Hybrid world? Capping costs and charge-backs schemes require evolution. And of course, the heterogeneity of the Hybrid Cloud user base requires additional flexibility: what are the challenges here?
Researchers vary in their financial resources — some can afford to trade $$ for time-to-completion, bursting into the Cloud to reduce wall-time; others are conserving their $ and want to live within the constraints of the on-prem systems. Some researchers rely on data sets already stored in the Cloud; others produce petabytes of images using custom instruments housed at their institution — these profiles experience different cost / performance trade-offs when comparing on-prem and Cloud hosted resources.
For IT infrastructure staff, Hybrid Cloud pushes us to support dual-stacks for the foreseeable future, and pushes us to understand our costs more clearly, both on-prem and sky-based, so that we can more effectively trade-off where to host which effort.
Unlike startups or retailers, IT infrastructure in a research environment tends to support multiple and sometimes conflicting goals: stability for customer A, agility & speed for customer B, regulatory compliance for customer C. Also, we want to anticipate rapid changes in requirements: “Oh, I forgot to mention: now I need this site to be protected and users need to login”.
Looking to the future, we predict a Partly Cloudy sky from zenith to horizon — i.e. we see the Hybrid Cloud as the research outfit’s default state. This immediately raises the questions: when to host in the Cloud, when to host on-prem, and how to migrate workloads between the two. Will on-premise become more cloudy now that platforms such as Kubernetes provide standard interfaces to all three Hyperscalers? (AWS, Azure & Google). How will we help our researchers meet the developing standards for reproducible software stacks?
Speakers are drawn entirely from our peers: infrastructure engineers, product managers, and management supporting research, offering their perspectives on these challenges. We structure the conference as follows: Welcome dinner & networking Thursday evening. On Friday, speakers in the morning, small-group break-out after lunch, finishing with a plenary session in which we synthesize what we have learned.