Import error when using proposal on Linux kernels < 4.11

[Jean1995]

When trying to import the proposal python module on systems with Linux kernels that are older than 4.11 after installing PROPOSAL using conan, for example using pip install proposal, the import error
proposal.cpython-37m-x86_64-linux-gnu.so: undefined symbol: statx
is raised.

The reason for this is that PROPOSAL uses cubic_interpolation, which per default uses boost filesystem version 1.75. However, version 1.75 of boost introduced the usage of statx, which has been introduced in Linux kernel 4.11 (see boost filesystem release history).

From this issue, I conclude that the boost binaries provided by conan have been compiled on a newer kernel using statx. If you now install PROPOSAL on a Linux kernel older than 4.11, it uses the binaries compiled on a different kernel, but they are incompatible.

The current quick fix is probably to install the library cubic_interpolation either without conan by providing a boost installation, or by using conan but forcing conan to build the boost dependency from source (this can be done by using the --build option of conan install, see here for more details).
Afterwards, PROPOSAL can be installed using this custom installation of cubic_interpolation.

Long-term solution?

I am not sure how to fix this problem. I don't think it is a good solution to drop the support for older Linux kernels, since many clusters are still using them. I also don't think that we should downgrade the boost dependency of cubic_interpolation to 1.74 since at one point in the future, cubic_interpolation might want to use newer features of boost.

For now, believe that this is basically a conan issue. Maybe we should open an issue there, since they are providing the "wrong" boost binaries. But I am not sure whether my conclusion is correct.

[Jean1995]

Apparently, there are also other incarnations of this error.

Building on a Linux Kernel 3.10 worked fine, but when trying to create interpolation tables, python throws the runtime error:

RuntimeError: boost::filesystem::status: Function not implemented: "/xxx/tables/dndx_xxxxxx.txt"

One "hack" to fix this problem is to change the conanfile.py prior to installing, so that boost/1.74 is used instead of boost/1.75:

    def requirements(self):
        self.requires("cubicinterpolation/0.1.4")
        self.requires("spdlog/1.8.2")
        self.requires("nlohmann_json/3.9.1")
+       self.requires("boost/1.74.0")
        if self.options.with_python:
            self.requires("pybind11/2.6.2")
        if self.options.with_testing:
-           self.requires("boost/1.75.0")
            self.requires("gtest/1.10.0")
        if self.options.with_documentation:
            self.requires("doxygen/1.8.20")

[Jean1995]

When using conan to install the dependencies, the requirements can be overwritten using --require-override.

This means that one can call
conan install .. --require-override=boost/1.72.0
to build PROPOSAL on old Linux kernels.

For pip, we might automatize this by checking inside setup.py whether the Linux kernel is outdated (although I am not sure yet how to do this reliably) and, if necessary, call the --require-override keyword.

[Jean1995]

It sounds like the workaround in the conan call proposed here should work for people who want to use conan.

So something like

conan install .. -o with_python=True -o boost:extra_b2_flags=define= BOOST_FILESYSTEM_DISABLE_STATX	

[Jean1995]

I have added a possible approach to fix this issue by providing a step-by-step installing instruction on this wiki page:

https://github.com/tudo-astroparticlephysics/PROPOSAL/wiki/Installing-PROPOSAL-on-a-Linux-kernel---4.11

[Jean1995]

We just found a case where the kernel was 4.19.262, but importing proposal still failed with an identical error.
However, the same fix found under this link worked.