Image caching support for copied local files

[mthalman]

Image Builder's image caching support has the ability to determine that a Dockerfile should be rebuilt if its checksum has changed since it was last published. This checksum comparison is strictly limited to the Dockerfile, however. It doesn't handle cases where the Dockerfile is configured to copy in local files that are operated upon. Any change to those files (and not the Dockerfile) wouldn't trigger the Dockerfile to be rebuilt. There are such cases of Dockerfiles copying in local files in the buildtools-prereqs repo (example).

Image Builder should be updated to account for such file changes so that the Dockerfile can be rebuilt.

I see two options to support this:

• Update the existing image caching implementation to support other arbitrary files beyond the Dockerfile. This could be hard to maintain because the checksums for those would need to be stored in the image info JSON file which means Image Builder needs to know which files to track. So we'd either need to statically list those files the Dockerfile depends on in the manifest or something OR Image Builder would have to dynamically derive those files by parsing the Dockerfile. Rather than parsing the Dockerfile to see what gets copied, we could have Image Builder just scan the directory (recursively) where the Dockerfile is located and track all those files.
• Make use of the Docker BuildKit's caching feature (--cache-from, --cache-to). BuildKit isn't yet supported for Windows in current versions of Docker though.

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[lbussell]

[Triage] We could also consider supporting Docker's caching on linux only, and not support any file-based caching on Windows (continue using our old solution on windows until BuildKit is supported).