Segfault from jll package: Assertion `(((jl_datatype_t*)(dt))->layout->nfields) == 1' failed

[biona001]

This is followup to #394, which is now failing at a different stage due to upgrading CxxWrap.jl and libcxxwrap_julia_jll.jl (now their versions are v0.14.2 and v0.11.0 respective)

The problem

I wrote a Julia wrapper for the ghostbasil C++ library, and it works fine with a local installation of CxxWrap.jl and libcxxwrap_julia_jll.jl.

• Here is my CMakeLists.txt
• Here is my wrapper code

However, when I tried to make a jll package for this wrapper, the jll package can be loaded but segfaults when I try to run things, even though it built successfully (tested on x86_64-linux-gnu).

Do you have any insights/suggestions? It is mysterious to me that the wrapper code works locally, but not within a jll package. The error message is not very meaningful to me:

julia: /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/local/include/jlcxx/type_conversion.hpp:668: jlcxx::BoxedValue jlcxx::boxed_cpp_pointer(T*, jl_datatype_t*, bool) [with T = std::vector; jl_datatype_t = _jl_datatype_t]: Assertion `jl_is_concrete_type((jl_value_t*)dt)' failed

MWE:

# this is the jll package I wrote
pkg> add /~https://github.com/biona001/ghostbasil_jll.jl

# load the wrapper 
julia> using CxxWrap
julia> module ghostbasil
           using CxxWrap
           using ghostbasil_jll

           @wrapmodule(ghostbasil_jll.get_libghostbasil_wrap_path)

           function __init__()
               @initcxx
           end

           # wrapper exports only 1 function `block_group_ghostbasil`
           function block_group_ghostbasil(
               Ci::Matrix{Float64}, 
               Si_scaled::Matrix{Float64}, 
               r::Vector{Float64}, 
               lambda_path::Vector{Float64};
               m::Int = 5,
               max_n_lambdas::Int = 100,
               lambdas_iter::Int = 5,
               use_strong_rule::Bool = false,
               do_early_exit::Bool = true,
               delta_strong_size::Int = 500,
               max_strong_size::Int = (m+1)*size(Ci, 1),
               max_n_cds::Int = 100000,
               thr::Float64 = 1e-7,
               min_ratio::Float64 = 1e-2,
               n_threads::Int = 1,
               )
               # check for errors
               p = size(Ci, 1)
               size(Ci) == size(Si_scaled) || error("Expected size(Ci) == size(Si_scaled)")
               length(r) == (m+1)*p || error("Expected length(r) == $((m+1)*p)")
               issorted(lambda_path, rev=true) || 
                   error("lambda_path should be sorted from largest to smallest")
               isapprox(Ci, Ci', atol=1e-8) || error("Ci is not symmetric")
               isapprox(Si_scaled, Si_scaled', atol=1e-8) || 
                   error("Si_scaled is not symmetric")
               (m > 0) && (max_n_lambdas > 0) && (lambdas_iter > 0) && 
                   (max_n_cds > 0) && (thr > 0) && (min_ratio > 0) || 
                   error("Expected m, max_n_lambdas, lambdas_iter, max_n_cds, thr, min_ratio all to be > 0")
               
               return block_group_ghostbasil(Ci, Si_scaled, r, lambda_path, 
                   m, p, max_n_lambdas, lambdas_iter, use_strong_rule, do_early_exit, 
                   delta_strong_size, max_strong_size, max_n_cds, thr, min_ratio, n_threads)
           end

           export block_group_ghostbasil
       end

# simulate data
p = 10
x = randn(p, p)
y = randn(p, p)
Ci = x'*x
Si = y'*y
r = randn(6p)
lambda_path = [0.1, 0.05, 0.01, 0.001]

# call wrapper
julia> beta_i = ghostbasil.block_group_ghostbasil(Ci, Si, r, lambda_path)
julia: /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/local/include/jlcxx/type_conversion.hpp:669: jlcxx::BoxedValue<T> jlcxx::boxed_cpp_pointer(T*, jl_datatype_t*, bool) [with T = std::vector<double>; jl_datatype_t = _jl_datatype_t]: Assertion `(((jl_datatype_t*)(dt))->layout->nfields) == 1' failed.

signal (6): Aborted
in expression starting at REPL[14]:1
gsignal at /lib64/libc.so.6 (unknown line)
abort at /lib64/libc.so.6 (unknown line)
__assert_fail_base at /lib64/libc.so.6 (unknown line)
__assert_fail at /lib64/libc.so.6 (unknown line)
_ZN5jlcxx17boxed_cpp_pointerISt6vectorIdSaIdEEEENS_10BoxedValueIT_EEPS5_P14_jl_datatype_tb at /home/groups/sabatti/.julia/artifacts/42711b370b2d26640bb4347cbfd0d5bdbe5db663/lib/libghostbasil_wrap.so (unknown line)
_ZN5jlcxx6detail11CallFunctorISt6vectorIdSaIdEEJNS_8ArrayRefIdLi2EEES6_NS5_IdLi1EEES7_lmmmbbmmmddmEE5applyEPKvP10jl_array_tSC_SC_SC_lmmmbbmmmddm at /home/groups/sabatti/.julia/artifacts/42711b370b2d26640bb4347cbfd0d5bdbe5db663/lib/libghostbasil_wrap.so (unknown line)
block_group_ghostbasil at /home/groups/sabatti/.julia/packages/CxxWrap/5IZvn/src/CxxWrap.jl:624
#block_group_ghostbasil#1 at ./REPL[5]:41
block_group_ghostbasil at ./REPL[5]:11
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
do_call at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/interpreter.c:126
eval_value at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/interpreter.c:215
eval_stmt_value at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/interpreter.c:166 [inlined]
eval_body at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/interpreter.c:612
jl_interpret_toplevel_thunk at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/interpreter.c:750
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:906
jl_toplevel_eval_flex at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:850
ijl_toplevel_eval_in at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/toplevel.c:965
eval at ./boot.jl:368 [inlined]
eval_user_input at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/REPL/src/REPL.jl:151
repl_backend_loop at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/REPL/src/REPL.jl:247
start_repl_backend at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/REPL/src/REPL.jl:232
#run_repl#47 at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/REPL/src/REPL.jl:369
run_repl at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/REPL/src/REPL.jl:355
jfptr_run_repl_64164.clone_1 at /share/software/user/open/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
#967 at ./client.jl:419
jfptr_YY.967_37865.clone_1 at /share/software/user/open/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
jl_f__call_latest at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/builtins.c:774
#invokelatest#2 at ./essentials.jl:729 [inlined]
invokelatest at ./essentials.jl:726 [inlined]
run_main_repl at ./client.jl:404
exec_options at ./client.jl:318
_start at ./client.jl:522
jfptr__start_58474.clone_1 at /share/software/user/open/julia/1.8.4/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
ijl_apply_generic at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/gf.c:2559
jl_apply at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
true_main at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jlapi.c:575
jl_repl_entrypoint at /cache/build/default-aws-shared0-3/julialang/julia-release-1-dot-8/src/jlapi.c:719
main at julia (unknown line)
__libc_start_main at /lib64/libc.so.6 (unknown line)
unknown function (ip: 0x401098)
Allocations: 14248817 (Pool: 14235892; Big: 12925); GC: 19
Aborted

Possible reason: incorrect build_tarballs.jl

I am new to BinaryBuilder.jl, so perhaps my build_tarballs.jl is problematic? However it seems to build successfully, so I feel it is not the problem

# Note that this script can accept some limited command-line arguments, run
# `julia build_tarballs.jl --help` to see a usage message.
using BinaryBuilder, Pkg

name = "ghostbasil"
version = v"0.0.4"

# Collection of sources required to complete build
sources = [
    GitSource("/~https://github.com/biona001/ghostbasil.git", "5a7121542d39ac4439b366d0223cd117f4211681")
]

# Bash recipe for building across all platforms
script = raw"""
cd $WORKSPACE/srcdir
mkdir -p ghostbasil/julia/build
cd ghostbasil/julia/build

cmake \
    -DJulia_PREFIX=$prefix \
    -DCMAKE_INSTALL_PREFIX=$prefix \
    -DCMAKE_FIND_ROOT_PATH=$prefix \
    -DCMAKE_TOOLCHAIN_FILE=${CMAKE_TARGET_TOOLCHAIN} \
    -DEigen3_DIR=$prefix/share/eigen3/cmake \
    -DCMAKE_BUILD_TYPE=Release \
    ../

make
make install

# install license
install_license $WORKSPACE/srcdir/ghostbasil/R/LICENSE.md
"""

# These are the platforms we will build for by default, unless further
# platforms are passed in on the command line
platforms = [
    Platform("x86_64", "linux"; libc = "glibc"),
    Platform("aarch64", "linux"; libc = "glibc"),
    Platform("powerpc64le", "linux"; libc = "glibc"),
    Platform("x86_64", "linux"; libc = "musl"),
    Platform("aarch64", "linux"; libc = "musl")
]
platforms = expand_cxxstring_abis(platforms)

# The products that we will ensure are always built
products = [
    LibraryProduct("libghostbasil_wrap", :libghostbasil_wrap)
]

# Dependencies that must be installed before this package can be built
dependencies = [
    Dependency("libcxxwrap_julia_jll"),
    BuildDependency("Eigen_jll"),
    BuildDependency("libjulia_jll")
]

# Build the tarballs, and possibly a `build.jl` as well.
build_tarballs(ARGS, name, version, sources, script, platforms, products, dependencies; julia_compat="1.6", preferred_gcc_version = v"7.1.0")

Again, any insights or suggestions would be highly appreciated. Thank you very much for your time.

[barche]

I think the problem is indeed with the build script, packages based on libcxxwrap_julia need to be built against specific versions of libjulia, probably here you are loading a JLL that was compiled for a different Julia version than the one you are running (I don't know which one it picks by default). You can check an example here: /~https://github.com/JuliaPackaging/Yggdrasil/blob/master/L/libsingular_julia/build_tarballs.jl

Use of the libjulia_platforms function is what ensures that the Julia version is properly encoded into the platform.

[biona001]

Thank you for the tip!

I can confirm that after changing the platform variable to

platforms = [
    Platform("x86_64", "linux"; libc = "glibc", julia_version=v"1.9.0"),
    Platform("aarch64", "linux"; libc = "glibc", julia_version=v"1.9.0"),
    Platform("powerpc64le", "linux"; libc = "glibc", julia_version=v"1.9.0"),
    Platform("x86_64", "linux"; libc = "musl", julia_version=v"1.9.0"),
    Platform("aarch64", "linux"; libc = "musl", julia_version=v"1.9.0")
]

the jll package indeed works on Julia 1.9.0 but still segfaults on Julia 1.9.3.

If I do

platforms = [
    Platform("x86_64", "linux"; libc = "glibc", julia_version=v"1.9.0"),
    Platform("aarch64", "linux"; libc = "glibc", julia_version=v"1.9.0"),
    Platform("powerpc64le", "linux"; libc = "glibc", julia_version=v"1.9.0"),
    Platform("x86_64", "linux"; libc = "musl", julia_version=v"1.9.0"),
    Platform("aarch64", "linux"; libc = "musl", julia_version=v"1.9.0"),
    Platform("x86_64", "linux"; libc = "glibc", julia_version=v"1.9.3"),
    Platform("aarch64", "linux"; libc = "glibc", julia_version=v"1.9.3"),
    Platform("powerpc64le", "linux"; libc = "glibc", julia_version=v"1.9.3"),
    Platform("x86_64", "linux"; libc = "musl", julia_version=v"1.9.3"),
    Platform("aarch64", "linux"; libc = "musl", julia_version=v"1.9.3")
]

then the jll package runs in both Julia v1.9.0 and 1.9.3.