Missing installation requirements

[KeithMnemonic]

The keg documentation only mentions needing kiwi installed but it also requires gcc and the python3X-devel package. Without these you get the following errors when installing from pip

    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -DOPENSSL_LOAD_CONF -fwrapv -fno-semantic-interposition -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -fPIC -D_XATTR_VERSION="0.7.2" -D_XATTR_AUTHOR="Iustin Pop" -D_XATTR_EMAIL="[email protected]" -I/home/kberger/venv/azure-img-utils/include -I/usr/include/python3.8 -c xattr.c -o build/temp.linux-x86_64-3.8/xattr.o -Wall -Werror -Wsign-compare
    unable to execute 'gcc': No such file or directory
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/kberger/venv/azure-img-utils/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-wls8jmdr/pyxattr_c3d5e81c037b4c23ba1fc0e8f311474e/setup.py'"'"'; __file__='"'"'/tmp/pip-install-wls8jmdr/pyxattr_c3d5e81c037b4c23ba1fc0e8f311474e/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-zj15wa79/install-record.txt --single-version-externally-managed --compile --install-headers /home/kberger/venv/azure-img-utils/include/site/python3.8/pyxattr Check the logs for full command output.

    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -DOPENSSL_LOAD_CONF -fwrapv -fno-semantic-interposition -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -fPIC -D_XATTR_VERSION="0.7.2" -D_XATTR_AUTHOR="Iustin Pop" -D_XATTR_EMAIL="[email protected]" -I/home/kberger/venv/azure-img-utils/include -I/usr/include/python3.8 -c xattr.c -o build/temp.linux-x86_64-3.8/xattr.o -Wall -Werror -Wsign-compare
    xattr.c:25:10: fatal error: Python.h: No such file or directory
       25 | #include <Python.h>
          |          ^~~~~~~~~~
    compilation terminated.
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/kberger/venv/azure-img-utils/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-onfbf_zq/pyxattr_1da0775ed69e4cd9b809b6864428a9a2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-onfbf_zq/pyxattr_1da0775ed69e4cd9b809b6864428a9a2/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-jupk5nnu/install-record.txt --single-version-externally-managed --compile --install-headers /home/kberger/venv/azure-img-utils/include/site/python3.8/pyxattr Check the logs for full command output.

[jgleissner]

There's no C code in keg. It could be a module from the dependency chain that pip tries to install.

[KeithMnemonic]

regardless, you get an error if you follow the docs unless you already have gcc installed.

[jgleissner]

Sorry for leaving this for so long. Yes, it appears something in the dependency chain (pyxattr by the looks of it) requires compilation for installation through pip, so you'll need gcc and python-devel unless you have it pyxattr installed locally. The README does state that you need to have Python development headers installed. It doesn't mentioned gcc explicitly, but that's easily added of course.

[KeithMnemonic]

I found a few others as well that were needed while playing with keg

zypper in jing
pip install anymarkup

[jgleissner]

Both are used by kiwi not keg directly, so you would expect them to be installed automatically in the tox env. Maybe not jing, this might be optional.

[KeithMnemonic]

I ran into needs them before ever using kiwi. I did not think kiwi was a hard requriment if you are only building the kiwi files and not using kiwi to build the image.

[jgleissner]

Keg uses bits of kiwi to validate the generated description, so it is a hard requirement.

[KeithMnemonic]

hmm, mine runs without kiwi installed.

 keg --recipes-root ../keg-recipes --dest-dir /tmp sles/ondemand/15-sp3 -f
[ INFO    ]: 11:31:47 | Using KIWI schema: vm
[ INFO    ]: 11:31:47 | Support for multiple markup descriptions available
[ WARNING ]: 11:31:47 | header template config_sh_header.templ missing, using fallback header

[jgleissner]

It should be installed under .tox/ at least. The "Support for multiple ..." log message comes from kiwi.

[jgleissner]

I guess this can be closed, or is there anything that needs fixing from your POV @KeithMnemonic ?

[KeithMnemonic]

I do not think you should close it until all of these extra bits are documented. I will try on a fresh system and then list anything needed that was not found in the tox env.

[KeithMnemonic]

Using the current readme https://github.com/SUSE-Enceladus/keg#readme it does not provide correct requirements to develop with keg. The readme only mentions

"Make sure the Python development headers are installed (e.g. python36-devel)"

The understood expectation of a README is to provide correct steps to begin using/developing the project.

I have fresh system with only python3-devel installed

simply running tox -devel , I get this error

`ERROR: Command errored out with exit status 1: /home/vagrant/repos/github/keg/.tox/3/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-r8q93rz3/pyxattr_b09b4c8f9c4248ec86e00addac8efd92/setup.py'"'"'; file='"'"'/tmp/pip-install-r8q93rz3/pyxattr_b09b4c8f9c4248ec86e00addac8efd92/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-b7f6ipis/install-record.txt --single-version-externally-managed --compile --install-headers /home/vagrant/repos/github/keg/.tox/3/include/site/python3.6/pyxattr Check the logs for full command output.

ERROR: could not install deps [-r.virtualenv.dev-requirements.txt]; v = InvocationError('/home/vagrant/repos/github/keg/.tox/3/bin/pip install -r.virtualenv.dev-requirements.txt (see /home/vagrant/repos/github/keg/.tox/3/log/devel-1.log)', 1)
______________________________________________________________ summary ______________________________________________________________
ERROR: devel: could not install deps [-r.virtualenv.dev-requirements.txt]; v = InvocationError('/home/vagrant/repos/github/keg/.tox/3/bin/pip install -r.virtualenv.dev-requirements.txt (see /home/vagrant/repos/github/keg/.tox/3/log/devel-1.log)', 1)`

If I install gcc, the tox envs runs ok. If you continue follow the steps in the readme, everything runs as expected.

The next set of missing requirements is found when you go to build the docs as listed in the Makefile. make must also be installed on the system. While this may seem obvious, if you are using a VM or container to build a throw away keg dev env, you need to ensure these are present. Once make is installed, if you go into the doc directory and try to build the docs you get errors depending on which target you are using. For example:

make latexpdf : make[1]: latexmk: Command not found
make info: make[1]: makeinfo: Command not found

Also some of the build targets in the makefile show "not registered" errors

make doctest
make coverage
make spell

Sphinx error:
Builder name not registered or available through entry point
make: *** [Makefile:171: coverage] Error 2

where is [doctest, coverage, spell]

I logged a new issue on this.

To Summarize, the current README should list the following as additional requirements.

gcc
make
latexmkm
makeinfo

Also if developing on python3.8, you would not install python36-devel. I think the "alternative" python3-devel enables zypper to install the correct devel package for the installed python version. If only python3.6 is supported then this should be clearly stated in the README