Simplify TeachOpenCADD's environment setup

[dominiquesydow]

Installing TeachOpenCADD has become quite complex due to the many dependencies across the different talktorial topics which cover a wide range of CADD applications.

For example, if you are interested only e.g. in the RDKit-heavy notebooks (T001-7), you install (if using our env file) all the dependencies from the other notebooks even if you won't use them.

The current setup is cumbersome when:

1. Using the Binder integration --> the environment setup can take ages
2. Working with our CI --> again, installation takes up to 45 min I think & all notebooks (even those without any changes) are currently tested with every push to a PR

On the plus side: When installing TeachOpenCADD locally, the environment setup is fairly fast with mamba in my experience.

Possible solutions:

• Provide environments per notebook
  • This is painful to maintain I think (for example, if we need to pin a package like nglview, we would need to update all related notebook envs)
• Look into git submodules --- thanks @greglandrum for the tip
  • This would get rid of problem 1 and 2 but might be more difficult to understand for external contributors and I need to check if this is an issue for our website.
• To fix the problem 2, we could tell the CI to only check changed notebooks when running in PRs (not when running our scheduled CI on master) --- thanks @richardjgowers for the tip

[dominiquesydow]

Regarding the easier and faster Binder setup for TeachOpenCADD, I might have found a solution that we can provide rather quickly: Separate environment (either per-notebook or per-notebook-collection*) from content using nbgitpuller:
https://mybinder.readthedocs.io/en/latest/howto/external_binder_setup.html

• Content: As before https://github.com/volkamerlab/teachopencadd
• Environment: Use separate repo, e.g. https://github.com/dominiquesydow/teachopencadd-binder-t001

Use this website to generate the URLs for separate content/environment Binder setups:
https://jupyterhub.github.io/nbgitpuller/link?tab=binder

Example for T001 - Binder URL:
https://mybinder.org/v2/gh/dominiquesydow/teachopencadd-binder-t001/main?urlpath=git-pull%3Frepo%3Dhttps%253A%252F%252Fgithub.com%252Fvolkamerlab%252Fteachopencadd%26urlpath%3Dtree%252Fteachopencadd%252Fteachopencadd%252Ftalktorials%252FT001_query_chembl%252Ftalktorial.ipynb%26branch%3Dmaster

More notes on how Binder setups work: https://discourse.jupyter.org/t/how-to-reduce-mybinder-org-repository-startup-time/4956 --- e.g. important for TeachOpenCADD:

Use two repositories: one for the environment, one for your content - many people change their content much more often than they change the environment needed for it. However, Binder will re-build the environment for any changes to a repository. A hack to get around this is to define an “environment repository” that Binder builds, and use a hook to pull in new content at launch from a “content repository”. This means that your “environment repository” changes less-often, which should result in fewer new builds and reduced launch times. See the instructions in this post to get started.

*collections could be

• Cheminformatics, T001-T010, https://projects.volkamerlab.org/teachopencadd/talktorials.html#edition-2019-jcim
• DL, T021+22
• MD, T019+20
• SBDD pipeline, T011-T018
• Kinase similarity, T023-T028