How to handle keg-recipes in other projects

[schaefi]

We provide an image definition tree in a git repo here https://github.com/SUSE-Enceladus/keg-recipes

Projects that uses keg needs to do a local checkout of the data to run keg and would be impacted by any change of the repo.
I suggest we package up the recipes such that a specific state of them comes with a specific version

[rjschwei]

This is a difficult topic that we touched on in conversation in the first PR.

There is clearly the "problem" that a user that generates image descriptions with keg points at a description tree that does not contain the version of the descriptions the user is interested in. We could mitigate this by forcing the tree we are pointing to be the latest, but then the description tree may not be maintained in git and as such we'd have to carry some plugin mechanism to deal with various SCM systems. Not ideal. Also forcing the latest has the issue that a user that is paying attention and wants to generate descriptions at a given level will have to jump through extra hoops. Bottom line a "force the latest" approach would favor the lazy over those that pay attention.

Tying an image description tree version to a specific version of keg may make things more cumbersome. Of course if the format of the description tree changes in a way that requires keg code changes then we'd have to do this. Maybe the description tree should have a "version" file at the top level and keg could have an appropriate check and if keg doesn't understand the version string tit could complain. But again this only comes into play if we end up changing the structure of the description tree at some point in the future.

I am skeptical of creating a "package" either py or rpm of the data as it creates a process step for, at this point, no apparent gain. Then again we have not decided if there should be a "release" process for keg-recipes, as a matter of fact we've not discussed any process around the description tree and that is a conversation we should probably have before we pick this topic up.

[schaefi]

Thanks for sharing your thoughts

as a matter of fact we've not discussed any process around the description tree and that is a conversation we should probably have before we pick this topic up.

yes I agree it would be good to have this conversation

Maybe we can base the conversation on a real use case. So when the first version of keg is out and usable I want to use it to manage the Azure LI/VLI image descriptions. For this purpose my current process and thinking is as follows:

• Go to the checkout of the azure-li-services git tree and fetch the keg-recipes
  git clone [email protected]:SUSE-Enceladus/keg-recipes.git (cloning a git in a git is possible)

• Add keg-recipes/ to be ignored in .gitignore by azure-li-services

• Write a little helper script in helper/ that creates me all Azure LI/VLI image descriptions by calling keg and
  store them to images/ at the moment the data in images/ is maintained information, but will then turn into keg
  generated information. Therefore place a README file in image/ telling that all data here is Keg generated and should
  not be touched

• From there create pull requests on review like we did before such that there is always a review of the generated image
  descriptions

• If a change on the image descriptions is needed it must be done in the keg-recipes git repo and also reviewed by a PR.
  The little helper script in azure-li-services must call a git pull prior calling keg

  ==> and here my problem starts. A pull from the keg-recipes brings in everything and could impact the generated results.
  There will always be a review of generated data, thus I'm not too concerned but you basically trust data from some main
  tree of some repo

My idea in this regard was that we could have a release process of the data in keg-recipes only for the data which is completely isolated and not related to a specific use case. A user can then install e.g keg-recipes-base and implement its individual tree according to the use case. That way the keg image definition data for the Azure LI/VLI project could live in the git project associated with the project. This would basically question the one repo to hunt them all approach.

So I have no real solution either but was thinking based on a real use case how to make use of keg-recipes in the best way

[gmoro]

Hello @rjschwei @jgleissner

On this topic, I started a new repository https://github.com/openSUSE/keg-recipes-minimal and this will be the base for all ALP images and Minimal-VM images, we have been talking in how to best organise this as well, so I would like to revive the conversation.
Initially we thought about just one repository, but that might became over complex specially if we merge the public cloud images on it.
Still we want to have at least a base set of recipes that can define what SLES and ALP would be and that is as fine grained as possible for the different formats and architectures we will support.
I will not throw all the ideas here at once, can I create a shared document where we can discuss that? Where would you like it? (For now we are collecting info on Confluence, but we should do that in the open)