What can containers not do that you wish they could? What features would you like to see?
2022
common X forwarding, finer grained platform differentiation (OCI WG), reuse of layers independent of layer rank
Mount inside a container (this is probably out of scope for containers in HPC)
Better MPI/PMIx support
reduction in complexity. using podman to create a simple jail isn't simple (should just use a jail or chroot tho lol)
Vulnerability scanning across the board. Generating an SBOM.
A better distribution story?
More attention to co-processor integration
make applications portable, make results reliably repeatable
Performance for specific architectures
Better support for rootless in shared clusters with appropriate networking and user namespaces. Better support for rootless builds.
Hardware isolation. HPC ecosystem needs to use singularity more for background processes, services (loghosts, web, open ondemand, etc), needs to be used to sequester users into individual or lab based clusters similar to virtuals (good for security), why isn't each user login a singularity container?
Security vulnerability reporting native to the container at build time; the equivalent of running "dnf -n update" or "apt list --upgradable" within the container when the container is launched and on a scheduled basis, and reporting the results to an address defined when the container was created.
Security built in to easily identify if a container has been patched or scanned
Easier integration with different CPU architectures and better performance especially for Docker.
I've been able to do everything I can do on bare metal in a container. I recently created a remote desktop container which is one of the last things I wish I was able to do.
Remote storage on startup (think, samba mount on start)
Proper pass through of sssd user ids
docker with normal user permission
I don't have enough experience to know how to answer this question...With my ex-CIO/IT manager hat on, I have to give the first answer to all such questions: Can security of containers be improved?
It's sort of done but would be nice to have better support for docker-compose style workloads. Any sort of container networking for singularity/apptainer would be nice.
Run Singularity/Apptainer images locally similarly to Docker Desktop.
Automate clean-up of the image following build
Better compatibility
Schedulers like Slurm need a better ability to deploy a collection of containers with private, encrypted container networking. Of course Kubernetes could be used, but that requires a very different HPC architecture than traditional HPC.
Possibly more tutorials and simplicification of using containers
Help to cross the OS platform automatically
Be reproducible in perPettit which they are far from
Generate Dockerfiles
Work without root permissions, that's the reason we use apptainer and not docker
See my previous answer.
It'd be great if we could easily combine containers !
I've had customers wish the ability to distribute containers that are usable by an end user but otherwise not modifiable/accessible. Basically a binary that can't be distributed outside of the container.
Easy way to import the MPI implementation from the HPC center, which is typically tuned to that center.
Easy "debug mode/tools" - you can always `exec` into a container, but then there's a vastly different set of software installed (might not have `curl`, and so on). In k8s environments the concept is easier (create a new container in the pod with all your tools, share namespaces & mounts) but still not perfect.
Be as fast as native baremetal applications.
I'd like to see checkpoint-migration-restore with production-ready integration with as many container technologies and schedulers as possible.
The CNCF and HPC facilities should take a close look at docker-nvidia-glx-desktop and docker-nvidia-egl-desktop, as well as selkies-gstreamer for graphical containers in unprivileged clusters.
Most HPC clusters and NSF-funded infrastructure are unprivileged and utilizing hardware-accelerated GUIs within a container was historically hard to perform.
Most HPC clusters and NSF-funded infrastructure are unprivileged and utilizing hardware-accelerated GUIs within a container was historically hard to perform.
Easily access data on a broad range of systems, be truly portable.
Seamless interaction without careful setup
Currently they do all I need
Out of the box infiniband, id management, out of the box GPU support.
Magically identify there system dependences, magically deal with version skews on the system
host mpi version conflicts
Provide a quick and standardized metadata/provenance report, to be used in other provenance formats like RO-Crate
sudo inside container
Docker and user identity is still a headache, but this is fine in singularity/apptainer
We have had great success with our researchers able to get going when our environment may not support all the requirements
Having something like the ONBUILD instructions in the Docker container format be supported by the OCI container format
Drivers! (Yes I know)
improvement in regards to graphics
Nesting containers
Not sure yet. But need to chain multiple containers in a pipeline to support various researcher workflows (haven't done this yet and hoping its possible)
Support for Linux device tree validation - would like containers built for specific architectures and accelerators (rdma nic, gpu, etc). This feature could be a serialization of the device tree file that gets validated with a string comparison on container initialization, or maybe a file with linear fields for different features that have hashed values which are validated against the machine running the image so that the container runtime could provide useful errors (architectures dont match, etc). Users could pick and choose what information to bake into the container so as little as architecture (x86, arm, riscv, etc) or as much as accelerator (gpu, rdma nic, etc)