Match existing images and base images in ScanCode.io

[pombredanne]

When scanning an image I would like to match to know existing images, and layers, and in particular base images. For this we need to:

• PurlDB: index known container images layers, and base images Index all the container images, in particular base layer for popular images purldb#434
• MatchCode: match existing layers to an index of images
• SCIO: create pipe and pipelines to match container layers and base images (this issue)

The purpose of this is that when we want to figure out vulnerabilities, we can first work on the base images, patch the vulnerabilities, and then reuse that patched base image throughout.

On the scanning side, we can then identify quickly if the image we have is patched or not, without having to review the vulnerabilities of possibly thousand packages with million vulnerabilities across thousand repos.

The base images should be collected for indexing from the public docker hub registry or other public registries or a private registry, possibly filtered only on the base images.

These are related issues and code:

• SCIO: Lookup/fetch and match base image for windows containers #1189
• Treat each layer in a docker images as a package of its own.  #661
• In alpine container where the first layer is not clearly alpine, we do not detect the OS #1159 (because if we can match we can deal with missing metadata)
• https://github.com/aboutcode-org/purldb/blob/112f4f244350b5c6a23d7eaf6dc9bf41e97ac893/minecode/miners/dockerhub.py#L169

And @Hritik14 's :

• https://github.com/supplyshield/supplyshield/blob/master/libinv/scanners/image_scanner/base_image.py