Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use mold in nix environments #1564

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

use mold in nix environments #1564

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
946c73a
use mold in nix environments
tbro Jun 7, 2024
c5f8b7a
Install mold from Nix; disable on MacOS
jbearer Jun 7, 2024
7f1d4d3
make rustDeps available globally
tbro Jul 23, 2024
a190910
get gcc version
tbro Jul 23, 2024
6d071ed
add env var for linker clang
tbro Jul 23, 2024
f9c2611
improve comment
tbro Jul 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/contracts.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,11 @@ jobs:
git diff

- name: Build diff-test
# workaround for `gcc: error: unrecognized command-line option '-fuse-ld=/nix/store/b7hb6xpazdnidp0s87fain2im6lk4f4r-mold-2.30.0/bin/mold'`.
# my best guess is that a nested dependency of `libc` crate requires an antiquated version of gcc
# https://github.com/EspressoSystems/espresso-sequencer/actions/runs/10062088424/job/27813743953
env:
CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_LINKER: "clang"
run: |
nix develop --accept-flake-config -c cargo build --locked --bin diff-test --release

Expand Down
60 changes: 28 additions & 32 deletions flake.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,11 +46,20 @@
}:
flake-utils.lib.eachDefaultSystem (system:
let
# Mold linker is faster, but is not supported on MacOS.
moldLinker = !pkgs.stdenv.isDarwin;
rustDeps = [];
# node=error: disable noisy anvil output
RUST_LOG = "info,libp2p=off,isahc=error,surf=error,node=error";
RUST_BACKTRACE = 1;
ASYNC_FLAGS = " --cfg async_executor_impl=\"async-std\" --cfg async_channel_impl=\"async-std\" ";
RUSTFLAGS = "${ASYNC_FLAGS} --cfg hotshot_example";
LINKER_FLAGS =
if moldLinker then
"-C link-arg=-fuse-ld=${pkgs.mold}/bin/mold"
else
""
;
RUSTFLAGS = "${ASYNC_FLAGS} --cfg hotshot_example ${LINKER_FLAGS}";
RUSTDOCFLAGS = ASYNC_FLAGS;
# Use a distinct target dir for builds from within nix shells.
CARGO_TARGET_DIR = "target/nix";
Expand Down Expand Up @@ -182,16 +191,23 @@
exec ${pkgs.nixFlakes}/bin/nix --experimental-features "nix-command flakes" "$@"
'';
solc = pkgs.solc-bin.latest;
in
mkShell (rustEnvVars // {
buildInputs = [
# Rust dependencies
stdenv =
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is stdenv meant to be passed to mkShell?

I get compilation errors on NixOS currently (and also if i pass stdenv to mkShell).

  = note: gcc: error: unrecognized command-line option '-fuse-ld=/nix/store/b7hb6xpazdnidp0s87fain2im6lk4f4r-mold-2.30.0/bin/mold'

So it looks like the devShell still brings in gcc and that's used for compilation.

I can compile with this change

diff --git a/flake.nix b/flake.nix
index 66d29e7a..7e0872a5 100644
--- a/flake.nix
+++ b/flake.nix
@@ -205,7 +205,8 @@
             jq
           ] ++ lib.optionals moldLinker [ mold ];
         in
-        mkShell (rustEnvVars // {
+        stdenv.mkDerivation (rustEnvVars // {
+          name = "shell";
           buildInputs = [
             stableToolchain

However linking still seems fairly slow. What kind of speedup do you get?

Copy link
Contributor Author

@tbro tbro Jul 24, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is stdenv meant to be passed to mkShell?

I get compilation errors on NixOS currently (and also if i pass stdenv to mkShell).

  = note: gcc: error: unrecognized command-line option '-fuse-ld=/nix/store/b7hb6xpazdnidp0s87fain2im6lk4f4r-mold-2.30.0/bin/mold'

Sorry my nix skills are very thin, so I'm probably doing something wrong. The gcc error is the same that caused me to add the env var. I'm unsure what configuration is causing gcc to be selected in some cases and not others. Possibly the issue is related to the gcc version.

So it looks like the devShell still brings in gcc and that's used for compilation.

However linking still seems fairly slow. What kind of speedup do you get?

It is noticeably faster for me. It isn't very straightforward to get real figures, because the biggest speedups are on incremental changes and modifying RUSTFLAGS will trigger a recompile of many things.

Copy link
Collaborator

@sveitser sveitser Jul 25, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I think works quite well and is somewhat realistic for development is to run a full build, then touch sequencer/src/lib.rs and then run the build again and time it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

#!/usr/bin/env bash

function build() {
    echo "building..."
    cargo clippy --all-features
}

build
touch sequencer/src/lib.rs
echo "timing re-build ..."
time build

I tried with this script and for both the new shell and the old shell I get 4-5 seconds real time.

Copy link
Collaborator

@sveitser sveitser Jul 25, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the linker is actually not invoked when running cargo clippy. If I make this change

diff --git a/flake.nix b/flake.nix
index 66d29e7a..5015ff00 100644
--- a/flake.nix
+++ b/flake.nix
@@ -55,7 +55,7 @@
       ASYNC_FLAGS = " --cfg async_executor_impl=\"async-std\" --cfg async_channel_impl=\"async-std\" ";
       LINKER_FLAGS =
         if moldLinker then
-          "-C link-arg=-fuse-ld=${pkgs.mold}/bin/mold"
+          "-C link-arg=-fuse-ld=${pkgs.mold}/bin/mold-blah"
         else
           ""
       ;

things still work.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also the shell still doesn't seem to be using mold for linking in my case 🤔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe you can confirm like so:

readelf -p .comment target/debug/sequencer

String dump of section '.comment':
  [     0]  **mold** 2.31.0 (20fa8d56f5e0c47d1f4bbf7b829c12d3f43298e1; compatible with GNU ld)
  [    4f]  rustc version 1.78.0 (9b00956e5 2024-04-29)
  [    7b]  GCC: (Debian 12.2.0-14) 12.2.0

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the linker is actually not invoked when running cargo clippy. If I make this change

Cargo clippy evokes rustc w/ linker arguments... I think some linking would have to go on since you are compiling a new binary... but I'm really not clear on the internals.

Copy link
Contributor Author

@tbro tbro Jul 25, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe if you want to get a better idea if it mold is useful for you, start by removing nix from the equation. Then we can figure out how to fix my terrible nix code :/.

if moldLinker then
pkgs.stdenvAdapters.useMoldLinker pkgs.clangStdenv
else
pkgs.stdenv
;
rustDeps = [
pkg-config
openssl
curl
protobuf # to compile libp2p-autonat
stableToolchain
jq
] ++ lib.optionals moldLinker [ mold ];
in
mkShell (rustEnvVars // {
buildInputs = [
stableToolchain

# Rust tools
cargo-audit
Expand Down Expand Up @@ -222,7 +238,9 @@
solhint
(python3.withPackages (ps: with ps; [ black ]))
yarn
] ++ lib.optionals stdenv.isDarwin
]
++ rustDeps
++ lib.optionals stdenv.isDarwin
[ darwin.apple_sdk.frameworks.SystemConfiguration ]
++ lib.optionals (!stdenv.isDarwin) [ cargo-watch ] # broken on OSX
;
Expand All @@ -248,29 +266,14 @@
};
in
mkShell (rustEnvVars // {
buildInputs = [
# Rust dependencies
pkg-config
openssl
curl
protobuf # to compile libp2p-autonat
toolchain
];
buildInputs = [ toolchain ] ++ rustDeps;
});
devShells.coverage =
let
toolchain = pkgs.rust-bin.nightly.latest.minimal;
in
mkShell (rustEnvVars // {
buildInputs = [
# Rust dependencies
pkg-config
openssl
curl
protobuf # to compile libp2p-autonat
toolchain
grcov
];
buildInputs = [ toolchain grcov ] ++ rustDeps;
CARGO_INCREMENTAL = "0";
shellHook = ''
RUSTFLAGS="$RUSTFLAGS -Zprofile -Ccodegen-units=1 -Cinline-threshold=0 -Clink-dead-code -Coverflow-checks=off -Cpanic=abort -Zpanic_abort_tests -Cdebuginfo=2"
Expand All @@ -285,14 +288,7 @@
};
in
mkShell (rustEnvVars // {
buildInputs = [
# Rust dependencies
pkg-config
openssl
curl
protobuf # to compile libp2p-autonat
stableToolchain
];
buildInputs = [ stableToolchain ] ++ rustDeps;
});
});
}
2 changes: 2 additions & 0 deletions scripts/show-toolchain-versions
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ echo 'rustc --version: ' $(rustc --version)
echo 'rustfmt --version: ' $(rustfmt --version)
echo 'cargo fmt --version: ' $(cargo fmt --version)
echo 'cargo clippy --version: ' $(cargo clippy --version)
echo 'gcc --version: ' $(gcc --version)
echo 'gcc -v: ' $(gcc -V)
echo

forge --version
Expand Down
Loading