🎙️ Discussion

Container Metadata


Table of contents

  1. What are the challenges that we are trying to solve?
  2. What ideas do we have for how to address these challenges?
  3. How do these ideas break down into…
    1. Already existing OCI specs?
    2. A new project (CNCF or OCI)
  4. Questions?
  5. What pre-existing work or software can support these ideas?
  6. Relevant OCI Issues

What are the challenges that we are trying to solve?

  • Containers are not able to provide enough metadata about binaries / libraries inside to assess compatibility with a host.
  • Although there is a limited ability to parse high level container metadata, details about the contents of containers does not come in any standard format that can be easily parsed. HPC developers often must parse build files, scripts, image layers, etc. (when available) to determine the contents.
  • Configuration of software and environment variables can affect reproducibility but are rarely captured in metadata.

Containers can seem like black boxes. But for running a container on HPC, we need to know about:

  1. attributes of the applications and binaries that are contained inside
  2. what requirements might be for running (e.g. hardware, memory, disk space)
  3. potentially other information that could be used by a resource manager or scheduler to know if it can run a container using this image.

What ideas do we have for how to address these challenges?

TODO add ideas from papers below?

How do these ideas break down into…

Already existing OCI specs?

A new project (CNCF or OCI)

Questions?

What pre-existing work or software can support these ideas?

Existing technologies that address parts of these ideas (incompletely):

  • Docker provides Object labels that are optional for images, containers, volumes, and more (such as the LABEL command within the Dockerfile).
  • Singularity definition files provide a few relevant sections, such as label and help. More Singularity metadata discussion here.

The following works are relevant:

Relevant OCI Issues