Design for tools like node and python that hack their own installation

[hohwille]

As a general best-practice I would expect a tool to allow a pristine installation. That is the tool gets installed in some installation folder and then nothing is changes for this installation (while the version of the installation remains the same).

Based on this concept we create the software repository in IDEasy. For every tool we create a folder with a sub-folder for every version of that tool that gets installed. Then we can simply set/change/remove a symbolic link to install, upgrade/downgrade or remove a tool from a project. As a benefit of this principle multiple IDEasy projects can share the installation of a tool if they also use the same version what allows to save disc-space. Also downloading, extracting, etc. does not have to be repeated and the installation and upgrade/downgrade is lightning fast if the desired version is also available in the software repository.

However, IDEasy wants to ensure the sandbox principle for projects. That is each project lives in isolation (like a docker container) and changes in project A do not effect project B (on the same machine). This implies that the installation of a software tool for a particular version remains pristine.

In general this is all given for most tools like java, dotnet, etc. and working smooth.
A challenge are tools that support plugins as these are typically "installed" inside the installation itself. However, if project A installs plugin P1 for tool T then project B should not automatically get plugin P1 if it also uses tool T (in the same version).
We solved this by a dedicated plugins folder inside the projects and relocation of the plugins for a tool into such dedicated project specific folder via some launch parameter or configuration. Beside some edge-cases (see #224) this is also solved and working.

However, we have two famous tools that completely violate this principle and cause severe issues:

1. node (aka node.js)
2. pyhton

node

Inside a node installation all software that is based on node is installed on top manipulating the existing installation.
The most prominent aspect is already npm that comes with node. If the project P1 and P2 both want to use the latest version of node they share the installation in the software repository behind their same symbolic link. Now if P1 and P2 have different requirements for the version of npm we have a severe conflict as if P1 installs npm version V1 it will do this inside the installation that is also used by P2 and therefore have such side effect. If P2 instead needs a different npm version V2 it will override it also for P1 on installation and vice versa.
If even gets worse since npm install -g «tool» also installs any other tool globally (-g) into that installation causing the same side effects.

See also #324

python

With python we more or less have the same story as with node. What is npm for node we have with pip for python and installing pip and other python based tools independent from the python installation itself trying to keep it pristine have failed.
Here we already have various tickets indicating the problem:

• devon python pip not working ide#839
• Add version handling to commandlet pip ide#1170
• Integrate pip into python commandlet ide#1314

comparison

A great design can be seen with java or dotnet that bring their own runtime and then independent of that other tools depending on it can be installed elsewhere and as long as they know where to find such runtime (via env variables like PATH or JAVA_HOME, etc.) all is working fine. However, since we are not the makers of nodeorpython` we cannot change this "flawed design" and it seems so fundamental that there is almost no chance that these projects are likely to consider our plea.

proposed solution

As a solution, I would suggest to add some protected boolean isIgnoreSoftwareRepo() to LocalToolCommandlet that by default returns false. Then we can change the implementation in LocalToolCommandlet such that if the new method returns true we then do not delegate to installInRepo method on installation and also installInRepo method will fail. Instead the install method will then install the software directly in the software folder.
This will have the following limitations if that feature is activated (may be more, not complete):

• no workaround for bug Prevent windows file lock error when installing tool update #264
• no support for dependencies in conflicting versions to such tool (Support for tool dependencies #131) possible (a project cannot have two tools that require different/incompatible versions of node or python).
• installation/upgrading is not lightning fast
• wast of disk-space if two projects use the same tool in the same version

We will first create a PR implementing this change/feature to LocalToolCommandlet.

However, then we can solve the problem:

• node commandlet is overriding isIgnoreSoftwareRepo() to true
• npm commandlet is a simple delegation to npm install -g resolveVersion($NPM_VERSION) (simple delegate similar to kubectl - see Implement ToolCommandlet for Kubernetes #81). Like in devonfw-ide we will have no software/npm folder but have npm inside software/node.
• python commandlet is overriding isIgnoreSoftwareRepo() to true
• pip commandlet works like npm or kubectl as a simple wrapper on top of python commandlet. We will have to install pip inside python and not in its own software/pip folder.

Here we will create a separate PR for node/npm (e.g. in the context of #324) based on the PR for LocalToolCommandlet and this story itself.
Also we can then also start implementing PR for #38 and another one for pip (story needs to be created).

[hohwille]

I have finally created this documentation page:
https://github.com/devonfw/IDEasy/blob/main/documentation/tool-vendor-plea.adoc
This has been discussed as an idea ~2-3 years ago in the team but never happened.
In the future when we post in python, node, ... or create bugs or feature requests, we can link to that page.