Buildin on RISC-V #762

New Issue

Open

opened 2024-03-07 09:50:28 +00:00 by veretcle · 5 comments

veretcle commented

2024-03-07 09:50:28 +00:00

Hi,

I’m one of the lucky ones to have a RISC-V server by Scaleway. I tried to compile garage for it (directly on master), and ran into an issue regarding libsodium-sys compilation:

error: failed to run custom build command for `libsodium-sys v0.2.7`

Caused by:
  process didn't exit successfully: `/home/ubuntu/garage/target/debug/build/libsodium-sys-009635d90f5cb2f9/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=SODIUM_LIB_DIR
  cargo:rerun-if-env-changed=SODIUM_SHARED
  cargo:rerun-if-env-changed=SODIUM_USE_PKG_CONFIG
  cargo:rerun-if-env-changed=SODIUM_DISABLE_PIE
  OPT_LEVEL = Some("0")
  TARGET = Some("riscv64gc-unknown-linux-gnu")
  HOST = Some("riscv64gc-unknown-linux-gnu")
  cargo:rerun-if-env-changed=CC_riscv64gc-unknown-linux-gnu
  CC_riscv64gc-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CC_riscv64gc_unknown_linux_gnu
  CC_riscv64gc_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CC
  HOST_CC = None
  cargo:rerun-if-env-changed=CC
  CC = None
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some("true")
  CARGO_CFG_TARGET_FEATURE = Some("a,c,m")
  cargo:rerun-if-env-changed=CFLAGS_riscv64gc-unknown-linux-gnu
  CFLAGS_riscv64gc-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CFLAGS_riscv64gc_unknown_linux_gnu
  CFLAGS_riscv64gc_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CFLAGS
  HOST_CFLAGS = None
  cargo:rerun-if-env-changed=CFLAGS
  CFLAGS = None
  checking build system type... riscv64-unknown-linux-gnu
  checking host system type...
  --- stderr
  Invalid configuration `riscv64gc-unknown-linux-gnu': machine `riscv64gc-unknown' not recognized
  configure: error: /bin/bash build-aux/config.sub riscv64gc-unknown-linux-gnu failed
  thread 'main' panicked at /home/ubuntu/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libsodium-sys-0.2.7/build.rs:257:9:

  Failed to configure libsodium using cd "/home/ubuntu/garage/target/debug/build/libsodium-sys-4ead82f45f2702a9/out/source/libsodium" && CC="cc" CFLAGS="-O0 -ffunction-sections -fdata-sections -fPIC -gdwarf-4 -fno-omit-frame-pointer -march=rv64gc -mabi=lp64d -mcmodel=medany -Wall -Wextra" "/home/ubuntu/garage/target/debug/build/libsodium-sys-4ead82f45f2702a9/out/source/libsodium/configure" "--prefix=/home/ubuntu/garage/target/debug/build/libsodium-sys-4ead82f45f2702a9/out/installed" "--libdir=/home/ubuntu/garage/target/debug/build/libsodium-sys-4ead82f45f2702a9/out/installed/lib" "--host=riscv64gc-unknown-linux-gnu" "--enable-shared=no"
  CFLAGS=-O0 -ffunction-sections -fdata-sections -fPIC -gdwarf-4 -fno-omit-frame-pointer -march=rv64gc -mabi=lp64d -mcmodel=medany -Wall -Wextra
  CC=cc


  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

By poking around, I found this PR that fixes the issue, but on a abandoned crate. The maintained version of this crate seems to be used for compiling libsodium but not libsodium-sys, forcing to play around inside Cargo cache to fix the issue.

I haven’t found a proper way to build this, hence the opening of this issue. Is there a way to update this dependency properly?

Hi, I’m one of the lucky ones to have a RISC-V server by Scaleway. I tried to compile garage for it (directly on master), and ran into an issue regarding `libsodium-sys` compilation: ``` error: failed to run custom build command for `libsodium-sys v0.2.7` Caused by: process didn't exit successfully: `/home/ubuntu/garage/target/debug/build/libsodium-sys-009635d90f5cb2f9/build-script-build` (exit status: 101) --- stdout cargo:rerun-if-env-changed=SODIUM_LIB_DIR cargo:rerun-if-env-changed=SODIUM_SHARED cargo:rerun-if-env-changed=SODIUM_USE_PKG_CONFIG cargo:rerun-if-env-changed=SODIUM_DISABLE_PIE OPT_LEVEL = Some("0") TARGET = Some("riscv64gc-unknown-linux-gnu") HOST = Some("riscv64gc-unknown-linux-gnu") cargo:rerun-if-env-changed=CC_riscv64gc-unknown-linux-gnu CC_riscv64gc-unknown-linux-gnu = None cargo:rerun-if-env-changed=CC_riscv64gc_unknown_linux_gnu CC_riscv64gc_unknown_linux_gnu = None cargo:rerun-if-env-changed=HOST_CC HOST_CC = None cargo:rerun-if-env-changed=CC CC = None cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS CRATE_CC_NO_DEFAULTS = None DEBUG = Some("true") CARGO_CFG_TARGET_FEATURE = Some("a,c,m") cargo:rerun-if-env-changed=CFLAGS_riscv64gc-unknown-linux-gnu CFLAGS_riscv64gc-unknown-linux-gnu = None cargo:rerun-if-env-changed=CFLAGS_riscv64gc_unknown_linux_gnu CFLAGS_riscv64gc_unknown_linux_gnu = None cargo:rerun-if-env-changed=HOST_CFLAGS HOST_CFLAGS = None cargo:rerun-if-env-changed=CFLAGS CFLAGS = None checking build system type... riscv64-unknown-linux-gnu checking host system type... --- stderr Invalid configuration `riscv64gc-unknown-linux-gnu': machine `riscv64gc-unknown' not recognized configure: error: /bin/bash build-aux/config.sub riscv64gc-unknown-linux-gnu failed thread 'main' panicked at /home/ubuntu/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libsodium-sys-0.2.7/build.rs:257:9: Failed to configure libsodium using cd "/home/ubuntu/garage/target/debug/build/libsodium-sys-4ead82f45f2702a9/out/source/libsodium" && CC="cc" CFLAGS="-O0 -ffunction-sections -fdata-sections -fPIC -gdwarf-4 -fno-omit-frame-pointer -march=rv64gc -mabi=lp64d -mcmodel=medany -Wall -Wextra" "/home/ubuntu/garage/target/debug/build/libsodium-sys-4ead82f45f2702a9/out/source/libsodium/configure" "--prefix=/home/ubuntu/garage/target/debug/build/libsodium-sys-4ead82f45f2702a9/out/installed" "--libdir=/home/ubuntu/garage/target/debug/build/libsodium-sys-4ead82f45f2702a9/out/installed/lib" "--host=riscv64gc-unknown-linux-gnu" "--enable-shared=no" CFLAGS=-O0 -ffunction-sections -fdata-sections -fPIC -gdwarf-4 -fno-omit-frame-pointer -march=rv64gc -mabi=lp64d -mcmodel=medany -Wall -Wextra CC=cc note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace ``` By poking around, I found this [PR](https://github.com/sodiumoxide/sodiumoxide/pull/474/files) that fixes the issue, but on a abandoned crate. The [maintained version of this crate](https://github.com/Kuska-ssb/sodiumoxide/blob/extra/libsodium-sys/build.rs) seems to be used for compiling `libsodium` but not `libsodium-sys`, forcing to play around inside Cargo cache to fix the issue. I haven’t found a proper way to build this, hence the opening of this issue. Is there a way to update this dependency properly?

lx commented

2024-03-07 17:59:19 +00:00

Owner

Related: #127

At some point we might be interested in porting the code to ssb-handshake + ssb-boxstream which are implemented using Rust crypto primitives.

Note that at this point, SSB packages from sunrise-choir seem to have less activity than those from kuska-ssb (the one we are using).

In the meantime, have you tried compiling just the libsodium-sys crate on RISC-V using the patch you found? If you're able to port the patch to the maintained version we might be able to do PRs on kuska-sodiumoxide and kuska-handhake to be able to integrate those changes with Garage.

Related: #127 At some point we might be interested in porting the code to [`ssb-handshake`](https://github.com/sunrise-choir/ssb-handshake) + [`ssb-boxstream`](https://github.com/sunrise-choir/ssb-boxstream) which are implemented using Rust crypto primitives. Note that at this point, SSB packages from [sunrise-choir](https://github.com/sunrise-choir) seem to have less activity than those from [kuska-ssb](https://github.com/Kuska-ssb) (the one we are using). In the meantime, have you tried compiling just the libsodium-sys crate on RISC-V using the patch you found? If you're able to port the patch to the maintained version we might be able to do PRs on kuska-sodiumoxide and kuska-handhake to be able to integrate those changes with Garage.

veretcle commented

2024-03-10 07:59:59 +00:00

Author

In the meantime, have you tried compiling just the libsodium-sys crate on RISC-V using the patch you found? If you're able to port the patch to the maintained version we might be able to do PRs on kuska-sodiumoxide and kuska-handhake to be able to integrate those changes with Garage.

I was able to compile it inside libsodium-sys via the patch referenced. What I really don’t understand is why this outdated library is still used when compiling kuska-sodiumoxide. Shouldn’t it use the version of libsodium-sys shipped with kuska-sodiumoxide?

Anyway, I was able to adapt the patch and compile kuska-sodiumoxide on RISC-V (see attachment) but this does not seem to be the version used within garage so…

> In the meantime, have you tried compiling just the libsodium-sys crate on RISC-V using the patch you found? If you're able to port the patch to the maintained version we might be able to do PRs on kuska-sodiumoxide and kuska-handhake to be able to integrate those changes with Garage. I was able to compile it inside `libsodium-sys` via the patch referenced. What I really don’t understand is why this outdated library is still used when compiling `kuska-sodiumoxide`. Shouldn’t it use the version of `libsodium-sys` shipped with `kuska-sodiumoxide`? Anyway, I was able to adapt the patch and compile `kuska-sodiumoxide` on RISC-V (see attachment) but this does not seem to be the version used within garage so…

libsodium-sys-build.rs.patch

733 B

lx commented

2024-03-11 09:55:04 +00:00

Owner

I was able to compile it inside libsodium-sys via the patch referenced. What I really don’t understand is why this outdated library is still used when compiling kuska-sodiumoxide. Shouldn’t it use the version of libsodium-sys shipped with kuska-sodiumoxide?

As I understand it:

kuska-handshake has this line in their cargo.toml :
```
sodiumoxide = { version = "0.2.5-0", package = "kuska-sodiumoxide" }
```
So they will import the package kuska-sodiumoxide from crates.io
The source for kuska-sodium is in this repo, and in their cargo.toml they have this line:
```
libsodium-sys = { version = "0.2.4", path = "libsodium-sys" }
```
This means that if you clone the repo of kuska-sodiumoxide and do a cargo build, it will use the libsodium-sys that is in that repo from the libsodium-sys folder. However, once that crate is published on crates.io, and it is imported in other apps, it will use the official libsodium-sys crate, version 0.2.4 or newer. Garage is locked to the current version which is 0.2.7 on crates.io, so that is the one that is used when building Garage.

In conclusion, to fix RISC-V, you need to patch the build script in the upstream libsodium-sys repo (which is archived, so good luck with that) and get them to publish a new version on crates.io which we can then use in Garage by updating our lock file.

Since upstream libsodium-sys is probably not going to update anything, other options include 1/ publishing a forked libsodium-sys on crates.io, possibly from the kuska-ssb sodiumoxide repo, or 2/ rewriting the code to not use libsodium at all.

I agree that depending on an unmaintained crate is a big problem and I might put some energy in fixing this situation in the comming weeks.

> I was able to compile it inside libsodium-sys via the patch referenced. What I really don’t understand is why this outdated library is still used when compiling kuska-sodiumoxide. Shouldn’t it use the version of libsodium-sys shipped with kuska-sodiumoxide? As I understand it: 1. kuska-handshake has this line in their `cargo.toml` : ``` sodiumoxide = { version = "0.2.5-0", package = "kuska-sodiumoxide" } ``` So they will import the package [kuska-sodiumoxide](https://crates.io/crates/kuska-sodiumoxide) from crates.io 2. The source for kuska-sodium is in [this repo](https://github.com/Kuska-ssb/sodiumoxide/tree/extra), and in their `cargo.toml` they have this line: ``` libsodium-sys = { version = "0.2.4", path = "libsodium-sys" } ``` This means that if you clone the repo of kuska-sodiumoxide and do a `cargo build`, it will use the libsodium-sys that is in that repo from the libsodium-sys folder. However, once that crate is published on crates.io, and it is imported in other apps, it will use the official [`libsodium-sys` crate](https://crates.io/crates/libsodium-sys), version 0.2.4 or newer. Garage is locked to the current version which is `0.2.7` on crates.io, so that is the one that is used when building Garage. In conclusion, to fix RISC-V, you need to patch the build script in [the upstream libsodium-sys repo](https://github.com/sodiumoxide/sodiumoxide/tree/master/libsodium-sys) (which is archived, so good luck with that) and get them to publish a new version on crates.io which we can then use in Garage by updating our lock file. Since upstream libsodium-sys is probably not going to update anything, other options include 1/ publishing a forked libsodium-sys on crates.io, possibly from the kuska-ssb sodiumoxide repo, or 2/ rewriting the code to not use libsodium at all. I agree that depending on an unmaintained crate is a big problem and I might put some energy in fixing this situation in the comming weeks.

👍 1

lx commented

2024-03-11 09:59:50 +00:00

Owner

Note that this crate seems to be a maintaned version of libsodium-sys : https://crates.io/crates/libsodium-sys-stable.

We might be able to convince the kuska people to migrate kuska-sodiumoxide to that binding instead of the deprecated one.

Note that this crate seems to be a maintaned version of libsodium-sys : https://crates.io/crates/libsodium-sys-stable. We might be able to convince the kuska people to migrate kuska-sodiumoxide to that binding instead of the deprecated one.

lx commented

2024-03-11 11:03:30 +00:00

Owner

Made a PR on kuska-sodiumoxide : https://github.com/Kuska-ssb/sodiumoxide/pull/1

👍 1