Elastic Serverless Platform for High Throughput Computing Scientific Application

Año de inicio 2019
Organismo financiador EGI FOUNDATION
Tipo de proyecto APOYO COMP. TRANSF. Y DIFUSIÓN
Responsable científico Moltó Martínez Germán
Resumen This project will create a prototype to automatically deploy on-demand a computing platform to provide Functions-as-a-Service (FaaS) computing to execute scientific applications on EGI Federated Cloud. The platform will integrate open-source components and new developments to: i) deploy an elastic Kubernetes cluster whose number of nodes can grow and shrink; ii) support an on-premises Functions-as-a -Service manager (e.g. OpenFaaS); iii) integrate data stage in and stage out capabilities so that the functions can process from, and store data in, an external storage space. The platform will demonstrate how scientific communities can create highly scalable HTTP endpoints to expose an application and facilitate the execution of fileprocessing workloads. A programming model will be adopted to create highlyparallel event-driven file-processing serverless applications that execute on customized runtime environments provided by Docker containers and run on the elastic FaaS platform. This prototype will evolve technology at TRL3 developed to create the SCAR framework, which supports the execution of Docker containers on AWS Lambda, into TRL6 applied to the context of the EGI Federated Cloud. The following EGI Services and components from the EOSC-Hub catalogue, are expected to be partially integrated in this prototype: i) EGI Applications on Demand (particularly EC3), to deploy an elastic Kubernetes cluster; ii) lnfrastructure Manager, to provision and configure the required virtual infrastructure; iii) an elasticity plugin for CLUES to secale the Kubernetes cluster and iv) EGI DataHub (OneData), to provide data handling capabilities between the user and the functions runtime space, to the extent that this integration is possible