About Partly Cloudy
Scientific research faces challenges when it looks to the Cloud. Yes, elasticity is attractive and scaling beyond what we can host on-prem is tantalizing. However, Cloud economics dictate that increased resource consumption incurs increased costs, which suits the retail world neatly: more people buying your widgets leads to more revenue which pays the growing Cloud bill. But consumption of compute / network / storage in research tends to be poorly correlated with a revenue stream; this mismatch acts as an early barrier to leveraging the Cloud to support research activities.
Retooling our expertise to move from traditional programming techniques to Cloud-native paradigms (e.g. POSIX file semantics to object storage manipulation) requires retraining both IT and scientific staff. The Cloud comes with security defaults which are hard to replicate on-prem: how will we evolve our security models, handle compliance, and communicate risk in the Hybrid world? Capping costs and charge-backs schemes require evolution. And of course, the heterogeneity of the Hybrid Cloud user base requires additional flexibility: what are the challenges here?
Researchers vary in their financial resources — some can afford to trade $$ for time-to-completion, bursting into the Cloud to reduce wall-time; others are conserving their $ and want to live within the constraints of the on-prem systems. Some researchers rely on data sets already stored in the Cloud; others produce petabytes of images using custom instruments housed at their institution — these profiles experience different cost / performance trade-offs when comparing on-prem and Cloud hosted resources.
For IT infrastructure staff, Hybrid Cloud pushes us to support dual-stacks for the foreseeable future, and pushes us to understand our costs more clearly, both on-prem and sky-based, so that we can more effectively trade-off where to host which effort.
Unlike startups or retailers, IT infrastructure in a research environment tends to support multiple and sometimes conflicting goals: stability for customer A, agility & speed for customer B, regulatory compliance for customer C. Also, we want to anticipate rapid changes in requirements: “Oh, I forgot to mention: now I need this site to be protected and users need to login”.
Looking to the future, we predict a Partly Cloudy sky from zenith to horizon — i.e. we see the Hybrid Cloud as the research outfit’s default state. This immediately raises the questions: when to host in the Cloud, when to host on-prem, and how to migrate workloads between the two. Will on-premise become more cloudy now that platforms such as Kubernetes provide standard interfaces to all three Hyperscalers? (AWS, Azure & Google). How will we help our researchers meet the developing standards for reproducible software stacks?
Speakers are drawn entirely from our peers: infrastructure engineers, product managers, and management supporting research, offering their perspectives on these challenges. We structure the conference as follows: Welcome dinner & networking Thursday evening. On Friday, speakers in the morning, small-group break-out after lunch, finishing with a plenary session in which we synthesize what we have learned.