Unexplainable seg-fault on compute pipeline creation

[SiebenCorgie]

Hi,
I am currently trying to write a real time ray traced shadows (the hype is real) via vulkano.
So far everything works out great, but currently I am experiencing a strange bug.

When does it happen?

When Vulkano tries to create compute pipeline a seg-fault happens. RUST_BACKTRACE=1 returns nothing on that matter.

Since no Spirv compilation error happens I tried to find the "wrong" code by removing everything in the shader and then inserting lines till it crashes.
My compute pipeline creation code looks like this:

        let dispatch_dims = [
            (image_dims[0] / SHADOWTHREADSIZE[0]) + 1,
            (image_dims[1] / SHADOWTHREADSIZE[1]) + 1,
            1
        ];

        //Now start the point and spot light compute shader
        let point_desc = self.descriptor_pool_else.next()
        .add_sampled_image(
            position_img, self.gbuffer_sampler.clone()
        ).expect("failed to add position image to shadow pass")
        .add_sampled_image(
            normal_img, self.gbuffer_sampler.clone()
        ).expect("failed to add normal image to shadow pass")
        .add_image(shadow_img)
        .expect("failed to add target shadow image to shadow pass")
        .add_buffer(bvh_buffer)
        .expect("failed to add bvh buffer to compute pass")
        .add_buffer(vertex_buffer)
        .expect("failed to add vertex buffer to compute pass")
        .add_image(debug_img)
        .expect("failed to add debug image!")
        .build().expect("failed to build bvh buffer");


        let point_light_desc = light_sys.get_light_descriptorset(1, self.pipeline_else.clone());
        //Finally dispatch the last shadow pass
        let final_cb = command_buffer.dispatch(
            dispatch_dims, self.pipeline_else.clone(),
            (point_desc, point_light_desc), ()
        ).expect("failed to start point light shadows compute shader");

The whole module can be found here, WARNING spaghetti code :)
The whole shader can be found here: https://gitlab.com/Siebencorgie/jakar-engine/blob/master/data/shader/r_shadow_else.comp

However, its mostly a copy of the r_shadow_dir.comp shader, which works perfectly fine so far.
The only changes are that I ray trace not "AnyHit" but "ClosestHit" (since it is for point lights and later spot lights).
Anyways the seg-fault only happens when I write to the current_hit:

///Traces a ray through the scene, returns if there was any hit.
TriangleHit ClosestHit(Ray ray){

  uint top_bvh_buffer_length = u_bvh.nodes.length();
  TriangleHit current_hit = TriangleHit(vec2(0.0,0.0), infinity, 0);
  //Start tracing at root
  int idx = 0;

  while(idx != -1){
    if (idx == -1 || idx > (top_bvh_buffer_length-1)){
      break;
    }
    int current_idx = idx;
    idx++;
    BvhNode node = u_bvh.nodes[current_idx];

    //Check if we intersect
    if (aabb_intersects(node.aabb_min.xyz, node.aabb_max.xyz, ray)){
      if (node.isLeaf == 1){
        TriangleHit leafhit = ray_intersect_mesh(ray, node.left, node.right);
        if (leafhit.is_hit == 1 && leafhit.tvec > MINHITDISTANCE && leafhit.tvec < current_hit.tvec){
          current_hit = leafhit; //<- If I remove this it works! --------------------------------------------------
        }
      }
    }else{
      //We can advance to the escape idx
      idx = node.esc_idx;
    }
  }
  return current_hit;
}

Question

My Question now is, has anyone an idea why this happens? Could this be some "over optimizing" by the glsl -> SPIRV compiler?

If you are reading this and you have a graphics card other than AMD vega 56 on Linux/Windows and some time on your hand.
Could you please:

1. download the full source code from here: https://gitlab.com/Siebencorgie/jakar-engine/tree/master
2. Download some simple gltf model (for instance this one: /~https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0/Lantern)
3. put it into examples/simple_scene/
4. correct the path on line 86 in examples/simple_scene/main.rs
5. run the example with cargo run --example simple --release

If I can provide any more detail I'd be happy to assist. I tried everything I could think of, including changing every return of a struct to a inout value like here and deleting the whole function and rewriting it.

-Siebencorgie

[rukai]

I havent looked into your code at all, but maybe it's a bug in the shader compiler? Try using nlordell's branch switching to shaderc-rs #947

[SiebenCorgie]

Hi,
I tried using his shaderc branch, but the segfault still happens.
I tried to get a stack trace via valgrid which gives me the following events:

==10646== Invalid read of size 4
==10646==    at 0x8095DC3: ??? (in /usr/lib/amdvlk64.so)
==10646==    by 0x8093D58: ??? (in /usr/lib/amdvlk64.so)
==10646==    by 0x939461D: ??? (in /usr/lib/amdvlk64.so)
==10646==    by 0x8079D57: ??? (in /usr/lib/amdvlk64.so)
==10646==    by 0x8066493: ??? (in /usr/lib/amdvlk64.so)
==10646==    by 0x806743E: ??? (in /usr/lib/amdvlk64.so)
==10646==    by 0x805D4FC: ??? (in /usr/lib/amdvlk64.so)
==10646==    by 0x80374E5: ??? (in /usr/lib/amdvlk64.so)
==10646==    by 0x27BBBB: <vulkano::pipeline::compute_pipeline::ComputePipeline<()>>::new (in /home/user/Code/Rust/engine/jakar-engine/simple)
==10646==    by 0x22B233: jakar_engine::render::raytracing::shadow::ShadowRayTracing::new (in /home/user/Code/Rust/engine/jakar-engine/simple)
==10646==    by 0x361CE4: jakar_engine::render::raytracing::RayTracing::new (in /home/user/Code/Rust/engine/jakar-engine/simple)
==10646==    by 0x284CC0: <jakar_engine::render::render_builder::RenderBuilder as jakar_engine::render::renderer::BuildRender>::build (in /home/user/Code/Rust/engine/jakar-engine/simple)
==10646==  Address 0x20 is not stack'd, malloc'd or (recently) free'd
==10646== 
==10646== 
==10646== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==10646==  Access not within mapped region at address 0x20
==10646==    at 0x8095DC3: ??? (in /usr/lib/amdvlk64.so)
==10646==    by 0x8093D58: ??? (in /usr/lib/amdvlk64.so)
==10646==    by 0x939461D: ??? (in /usr/lib/amdvlk64.so)
==10646==    by 0x8079D57: ??? (in /usr/lib/amdvlk64.so)
==10646==    by 0x8066493: ??? (in /usr/lib/amdvlk64.so)
==10646==    by 0x806743E: ??? (in /usr/lib/amdvlk64.so)
==10646==    by 0x805D4FC: ??? (in /usr/lib/amdvlk64.so)
==10646==    by 0x80374E5: ??? (in /usr/lib/amdvlk64.so)
==10646==    by 0x27BBBB: <vulkano::pipeline::compute_pipeline::ComputePipeline<()>>::new (in /home/user/Code/Rust/engine/jakar-engine/simple)
==10646==    by 0x22B233: jakar_engine::render::raytracing::shadow::ShadowRayTracing::new (in /home/user/Code/Rust/engine/jakar-engine/simple)
==10646==    by 0x361CE4: jakar_engine::render::raytracing::RayTracing::new (in /home/user/Code/Rust/engine/jakar-engine/simple)
==10646==    by 0x284CC0: <jakar_engine::render::render_builder::RenderBuilder as jakar_engine::render::renderer::BuildRender>::build (in /home/user/Code/Rust/engine/jakar-engine/simple)
==10646==  If you believe this happened as a result of a stack
==10646==  overflow in your program's main thread (unlikely but
==10646==  possible), you can try to increase the size of the
==10646==  main thread stack using the --main-stacksize= flag.
==10646==  The main thread stack size used in this run was 8388608.
==10646== 
==10646== HEAP SUMMARY:
==10646==     in use at exit: 73,827,915 bytes in 393,837 blocks
==10646==   total heap usage: 2,088,175 allocs, 1,694,349 frees, 648,476,882 bytes allocated
==10646== 
==10646== 28 bytes in 1 blocks are possibly lost in loss record 577 of 6,498
==10646==    at 0x483777F: malloc (vg_replace_malloc.c:299)
==10646==    by 0x63C92B1: ??? (in /usr/lib/libxcb-dri3.so.0.0.0)
==10646==    by 0x63C94E4: ??? (in /usr/lib/libxcb-dri3.so.0.0.0)
==10646==    by 0x57E9C48: ???
==10646==    by 0x4010549: call_init.part.0 (in /usr/lib/ld-2.28.so)
==10646==    by 0x4010649: _dl_init (in /usr/lib/ld-2.28.so)
==10646==    by 0x4014532: dl_open_worker (in /usr/lib/ld-2.28.so)
==10646==    by 0x4A13E76: _dl_catch_exception (in /usr/lib/libc-2.28.so)
==10646==    by 0x4013DFE: _dl_open (in /usr/lib/ld-2.28.so)
==10646==    by 0x4893159: ??? (in /usr/lib/libdl-2.28.so)
==10646==    by 0x4A13E76: _dl_catch_exception (in /usr/lib/libc-2.28.so)
==10646==    by 0x4A13F12: _dl_catch_error (in /usr/lib/libc-2.28.so)

I also doubled the stack size but no changes.
I wont have access to my code for the next week, so It will be difficult to try new stuff, but after that I try to find a computer with a Nvidia card and test my code there. Maybe its a driver problem. But if so it would be strange because I already changed from RADV to AMDVLK.

[nicokoch]

Have you tried enabling validation layers?

[SiebenCorgie]

Oh, no I totally forgot about those. I'll do as soon as I come home!

[SiebenCorgie]

Okay so I activated the lunarG layers and this is the output:

Thread 0, Frame 0:
vkCreateDescriptorSetLayout(device, pCreateInfo, pAllocator, pSetLayout) returns VkResult VK_SUCCESS (0):
    device:                         VkDevice = 0x7f811c6cfb40
    pCreateInfo:                    const VkDescriptorSetLayoutCreateInfo* = 0x7ffe6343a6b0:
        sType:                          VkStructureType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_CREATE_INFO (32)
        pNext:                          const void* = NULL
        flags:                          VkDescriptorSetLayoutCreateFlags = 0
        bindingCount:                   uint32_t = 6
        pBindings:                      const VkDescriptorSetLayoutBinding* = 0x7ffe6343a728
            pBindings[0]:                   const VkDescriptorSetLayoutBinding = 0x7ffe6343a728:
                binding:                        uint32_t = 0
                descriptorType:                 VkDescriptorType = VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER (1)
                descriptorCount:                uint32_t = 1
                stageFlags:                     VkShaderStageFlags = 32 (VK_SHADER_STAGE_COMPUTE_BIT | VK_SHADER_STAGE_ALL)
                pImmutableSamplers:             const VkSampler* = NULL
            pBindings[1]:                   const VkDescriptorSetLayoutBinding = 0x7ffe6343a740:
                binding:                        uint32_t = 1
                descriptorType:                 VkDescriptorType = VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER (1)
                descriptorCount:                uint32_t = 1
                stageFlags:                     VkShaderStageFlags = 32 (VK_SHADER_STAGE_COMPUTE_BIT | VK_SHADER_STAGE_ALL)
                pImmutableSamplers:             const VkSampler* = NULL
            pBindings[2]:                   const VkDescriptorSetLayoutBinding = 0x7ffe6343a758:
                binding:                        uint32_t = 2
                descriptorType:                 VkDescriptorType = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE (3)
                descriptorCount:                uint32_t = 1
                stageFlags:                     VkShaderStageFlags = 32 (VK_SHADER_STAGE_COMPUTE_BIT | VK_SHADER_STAGE_ALL)
                pImmutableSamplers:             const VkSampler* = UNUSED
            pBindings[3]:                   const VkDescriptorSetLayoutBinding = 0x7ffe6343a770:
                binding:                        uint32_t = 3
                descriptorType:                 VkDescriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER (7)
                descriptorCount:                uint32_t = 1
                stageFlags:                     VkShaderStageFlags = 32 (VK_SHADER_STAGE_COMPUTE_BIT | VK_SHADER_STAGE_ALL)
                pImmutableSamplers:             const VkSampler* = UNUSED
            pBindings[4]:                   const VkDescriptorSetLayoutBinding = 0x7ffe6343a788:
                binding:                        uint32_t = 4
                descriptorType:                 VkDescriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER (7)
                descriptorCount:                uint32_t = 1
                stageFlags:                     VkShaderStageFlags = 32 (VK_SHADER_STAGE_COMPUTE_BIT | VK_SHADER_STAGE_ALL)
                pImmutableSamplers:             const VkSampler* = UNUSED
            pBindings[5]:                   const VkDescriptorSetLayoutBinding = 0x7ffe6343a7a0:
                binding:                        uint32_t = 5
                descriptorType:                 VkDescriptorType = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE (3)
                descriptorCount:                uint32_t = 1
                stageFlags:                     VkShaderStageFlags = 32 (VK_SHADER_STAGE_COMPUTE_BIT | VK_SHADER_STAGE_ALL)
                pImmutableSamplers:             const VkSampler* = UNUSED
    pAllocator:                     const VkAllocationCallbacks* = NULL
    pSetLayout:                     VkDescriptorSetLayout* = 0xcd

Thread 0, Frame 0:
vkCreateDescriptorSetLayout(device, pCreateInfo, pAllocator, pSetLayout) returns VkResult VK_SUCCESS (0):
    device:                         VkDevice = 0x7f811c6cfb40
    pCreateInfo:                    const VkDescriptorSetLayoutCreateInfo* = 0x7ffe6343a6b0:
        sType:                          VkStructureType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_CREATE_INFO (32)
        pNext:                          const void* = NULL
        flags:                          VkDescriptorSetLayoutCreateFlags = 0
        bindingCount:                   uint32_t = 4
        pBindings:                      const VkDescriptorSetLayoutBinding* = 0x7ffe6343a728
            pBindings[0]:                   const VkDescriptorSetLayoutBinding = 0x7ffe6343a728:
                binding:                        uint32_t = 0
                descriptorType:                 VkDescriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER (7)
                descriptorCount:                uint32_t = 1
                stageFlags:                     VkShaderStageFlags = 32 (VK_SHADER_STAGE_COMPUTE_BIT | VK_SHADER_STAGE_ALL)
                pImmutableSamplers:             const VkSampler* = UNUSED
            pBindings[1]:                   const VkDescriptorSetLayoutBinding = 0x7ffe6343a740:
                binding:                        uint32_t = 1
                descriptorType:                 VkDescriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER (7)
                descriptorCount:                uint32_t = 1
                stageFlags:                     VkShaderStageFlags = 32 (VK_SHADER_STAGE_COMPUTE_BIT | VK_SHADER_STAGE_ALL)
                pImmutableSamplers:             const VkSampler* = UNUSED
            pBindings[2]:                   const VkDescriptorSetLayoutBinding = 0x7ffe6343a758:
                binding:                        uint32_t = 2
                descriptorType:                 VkDescriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER (7)
                descriptorCount:                uint32_t = 1
                stageFlags:                     VkShaderStageFlags = 32 (VK_SHADER_STAGE_COMPUTE_BIT | VK_SHADER_STAGE_ALL)
                pImmutableSamplers:             const VkSampler* = UNUSED
            pBindings[3]:                   const VkDescriptorSetLayoutBinding = 0x7ffe6343a770:
                binding:                        uint32_t = 3
                descriptorType:                 VkDescriptorType = VK_DESCRIPTOR_TYPE_UNIFORM_BUFFER (6)
                descriptorCount:                uint32_t = 1
                stageFlags:                     VkShaderStageFlags = 32 (VK_SHADER_STAGE_COMPUTE_BIT | VK_SHADER_STAGE_ALL)
                pImmutableSamplers:             const VkSampler* = UNUSED
    pAllocator:                     const VkAllocationCallbacks* = NULL
    pSetLayout:                     VkDescriptorSetLayout* = 0xce

Thread 0, Frame 0:
vkCreatePipelineLayout(device, pCreateInfo, pAllocator, pPipelineLayout) returns VkResult VK_SUCCESS (0):
    device:                         VkDevice = 0x7f811c6cfb40
    pCreateInfo:                    const VkPipelineLayoutCreateInfo* = 0x7ffe6343aa90:
        sType:                          VkStructureType = VK_STRUCTURE_TYPE_PIPELINE_LAYOUT_CREATE_INFO (30)
        pNext:                          const void* = NULL
        flags:                          VkPipelineLayoutCreateFlags = 0
        setLayoutCount:                 uint32_t = 2
        pSetLayouts:                    const VkDescriptorSetLayout* = 0x7ffe6343ac00
            pSetLayouts[0]:                 const VkDescriptorSetLayout = 0xcd
            pSetLayouts[1]:                 const VkDescriptorSetLayout = 0xce
        pushConstantRangeCount:         uint32_t = 0
        pPushConstantRanges:            const VkPushConstantRange* = 0x7ffe6343aaec
    pAllocator:                     const VkAllocationCallbacks* = NULL
    pPipelineLayout:                VkPipelineLayout* = 0xcf

Thread 0, Frame 0:
vkCreateComputePipelines(device, pipelineCache, createInfoCount, pCreateInfos, pAllocator, pPipelines) returns VkResult VK_SUCCESS (0):
    device:                         VkDevice = 0x7f811c6cfb40
    pipelineCache:                  VkPipelineCache = 0
    createInfoCount:                uint32_t = 1
    pCreateInfos:                   const VkComputePipelineCreateInfo* = 0x7ffe6343ab50
        pCreateInfos[0]:                const VkComputePipelineCreateInfo = 0x7ffe6343ab50:
            sType:                          VkStructureType = VK_STRUCTURE_TYPE_COMPUTE_PIPELINE_CREATE_INFO (29)
            pNext:                          const void* = NULL
            flags:                          VkPipelineCreateFlags = 0
            stage:                          VkPipelineShaderStageCreateInfo = 0x7ffe6343ab68:
                sType:                          VkStructureType = VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO (18)
                pNext:                          const void* = NULL
                flags:                          VkPipelineShaderStageCreateFlags = 0
                stage:                          VkShaderStageFlagBits = 32 (VK_SHADER_STAGE_COMPUTE_BIT | VK_SHADER_STAGE_ALL)
                module:                         VkShaderModule = 0xcb
                pName:                          const char* = "main"
                pSpecializationInfo:            const VkSpecializationInfo* = NULL
            layout:                         VkPipelineLayout = 0xcf
            basePipelineHandle:             VkPipeline = 0
            basePipelineIndex:              int32_t = 0
    pAllocator:                     const VkAllocationCallbacks* = NULL
    pPipelines:                     VkPipeline* = 0x7ffe6343aa50
        pPipelines[0]:                  VkPipeline = 0xd0

Thread 0, Frame 0:
vkCreateDescriptorSetLayout(device, pCreateInfo, pAllocator, pSetLayout) returns VkResult VK_SUCCESS (0):
    device:                         VkDevice = 0x7f811c6cfb40
    pCreateInfo:                    const VkDescriptorSetLayoutCreateInfo* = 0x7ffe6343a6b0:
        sType:                          VkStructureType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_CREATE_INFO (32)
        pNext:                          const void* = NULL
        flags:                          VkDescriptorSetLayoutCreateFlags = 0
        bindingCount:                   uint32_t = 6
        pBindings:                      const VkDescriptorSetLayoutBinding* = 0x7ffe6343a728
            pBindings[0]:                   const VkDescriptorSetLayoutBinding = 0x7ffe6343a728:
                binding:                        uint32_t = 0
                descriptorType:                 VkDescriptorType = VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER (1)
                descriptorCount:                uint32_t = 1
                stageFlags:                     VkShaderStageFlags = 32 (VK_SHADER_STAGE_COMPUTE_BIT | VK_SHADER_STAGE_ALL)
                pImmutableSamplers:             const VkSampler* = NULL
            pBindings[1]:                   const VkDescriptorSetLayoutBinding = 0x7ffe6343a740:
                binding:                        uint32_t = 1
                descriptorType:                 VkDescriptorType = VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER (1)
                descriptorCount:                uint32_t = 1
                stageFlags:                     VkShaderStageFlags = 32 (VK_SHADER_STAGE_COMPUTE_BIT | VK_SHADER_STAGE_ALL)
                pImmutableSamplers:             const VkSampler* = NULL
            pBindings[2]:                   const VkDescriptorSetLayoutBinding = 0x7ffe6343a758:
                binding:                        uint32_t = 2
                descriptorType:                 VkDescriptorType = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE (3)
                descriptorCount:                uint32_t = 1
                stageFlags:                     VkShaderStageFlags = 32 (VK_SHADER_STAGE_COMPUTE_BIT | VK_SHADER_STAGE_ALL)
                pImmutableSamplers:             const VkSampler* = UNUSED
            pBindings[3]:                   const VkDescriptorSetLayoutBinding = 0x7ffe6343a770:
                binding:                        uint32_t = 3
                descriptorType:                 VkDescriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER (7)
                descriptorCount:                uint32_t = 1
                stageFlags:                     VkShaderStageFlags = 32 (VK_SHADER_STAGE_COMPUTE_BIT | VK_SHADER_STAGE_ALL)
                pImmutableSamplers:             const VkSampler* = UNUSED
            pBindings[4]:                   const VkDescriptorSetLayoutBinding = 0x7ffe6343a788:
                binding:                        uint32_t = 4
                descriptorType:                 VkDescriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER (7)
                descriptorCount:                uint32_t = 1
                stageFlags:                     VkShaderStageFlags = 32 (VK_SHADER_STAGE_COMPUTE_BIT | VK_SHADER_STAGE_ALL)
                pImmutableSamplers:             const VkSampler* = UNUSED
            pBindings[5]:                   const VkDescriptorSetLayoutBinding = 0x7ffe6343a7a0:
                binding:                        uint32_t = 5
                descriptorType:                 VkDescriptorType = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE (3)
                descriptorCount:                uint32_t = 1
                stageFlags:                     VkShaderStageFlags = 32 (VK_SHADER_STAGE_COMPUTE_BIT | VK_SHADER_STAGE_ALL)
                pImmutableSamplers:             const VkSampler* = UNUSED
    pAllocator:                     const VkAllocationCallbacks* = NULL
    pSetLayout:                     VkDescriptorSetLayout* = 0xd1

Thread 0, Frame 0:
vkCreateDescriptorSetLayout(device, pCreateInfo, pAllocator, pSetLayout) returns VkResult VK_SUCCESS (0):
    device:                         VkDevice = 0x7f811c6cfb40
    pCreateInfo:                    const VkDescriptorSetLayoutCreateInfo* = 0x7ffe6343a6b0:
        sType:                          VkStructureType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_CREATE_INFO (32)
        pNext:                          const void* = NULL
        flags:                          VkDescriptorSetLayoutCreateFlags = 0
        bindingCount:                   uint32_t = 4
        pBindings:                      const VkDescriptorSetLayoutBinding* = 0x7ffe6343a728
            pBindings[0]:                   const VkDescriptorSetLayoutBinding = 0x7ffe6343a728:
                binding:                        uint32_t = 0
                descriptorType:                 VkDescriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER (7)
                descriptorCount:                uint32_t = 1
                stageFlags:                     VkShaderStageFlags = 32 (VK_SHADER_STAGE_COMPUTE_BIT | VK_SHADER_STAGE_ALL)
                pImmutableSamplers:             const VkSampler* = UNUSED
            pBindings[1]:                   const VkDescriptorSetLayoutBinding = 0x7ffe6343a740:
                binding:                        uint32_t = 1
                descriptorType:                 VkDescriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER (7)
                descriptorCount:                uint32_t = 1
                stageFlags:                     VkShaderStageFlags = 32 (VK_SHADER_STAGE_COMPUTE_BIT | VK_SHADER_STAGE_ALL)
                pImmutableSamplers:             const VkSampler* = UNUSED
            pBindings[2]:                   const VkDescriptorSetLayoutBinding = 0x7ffe6343a758:
                binding:                        uint32_t = 2
                descriptorType:                 VkDescriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER (7)
                descriptorCount:                uint32_t = 1
                stageFlags:                     VkShaderStageFlags = 32 (VK_SHADER_STAGE_COMPUTE_BIT | VK_SHADER_STAGE_ALL)
                pImmutableSamplers:             const VkSampler* = UNUSED
            pBindings[3]:                   const VkDescriptorSetLayoutBinding = 0x7ffe6343a770:
                binding:                        uint32_t = 3
                descriptorType:                 VkDescriptorType = VK_DESCRIPTOR_TYPE_UNIFORM_BUFFER (6)
                descriptorCount:                uint32_t = 1
                stageFlags:                     VkShaderStageFlags = 32 (VK_SHADER_STAGE_COMPUTE_BIT | VK_SHADER_STAGE_ALL)
                pImmutableSamplers:             const VkSampler* = UNUSED
    pAllocator:                     const VkAllocationCallbacks* = NULL
    pSetLayout:                     VkDescriptorSetLayout* = 0xd2

Thread 0, Frame 0:
vkCreatePipelineLayout(device, pCreateInfo, pAllocator, pPipelineLayout) returns VkResult VK_SUCCESS (0):
    device:                         VkDevice = 0x7f811c6cfb40
    pCreateInfo:                    const VkPipelineLayoutCreateInfo* = 0x7ffe6343aa90:
        sType:                          VkStructureType = VK_STRUCTURE_TYPE_PIPELINE_LAYOUT_CREATE_INFO (30)
        pNext:                          const void* = NULL
        flags:                          VkPipelineLayoutCreateFlags = 0
        setLayoutCount:                 uint32_t = 2
        pSetLayouts:                    const VkDescriptorSetLayout* = 0x7ffe6343ac00
            pSetLayouts[0]:                 const VkDescriptorSetLayout = 0xd1
            pSetLayouts[1]:                 const VkDescriptorSetLayout = 0xd2
        pushConstantRangeCount:         uint32_t = 0
        pPushConstantRanges:            const VkPushConstantRange* = 0x7ffe6343aaec
    pAllocator:                     const VkAllocationCallbacks* = NULL
    pPipelineLayout:                VkPipelineLayout* = 0xd3

[1]    32135 segmentation fault (core dumped)  cargo run --example simple --release

Looks good to me. But the problem is still there.

[nicokoch]

Can you provide a backtrace using gdb?
Otherwise I'll try to clone your repo and reproduce later.

This might very well be a bug in

1. vulkano or
2. glsl or
3. vulkan driver

can you also post the output of vulkaninfo?

[nicokoch]

Something fishy I noticed: You do not create the window on the main thread.
On MacOS this panics: RUST_BACKTRACE=1 cargo run --example simple Finished dev [unoptimized + debuginfo] target(s) in 1.33s Running target/debug/examples/simple Starting render builder Created App Info Created Instance thread '<unnamed>' panicked at 'Windows can only be created on the main thread on macOS'

[SiebenCorgie]

Hi,
Yes, the window and its input run in another thread. I did not know that this is a problem on Mac (works on Linux and Windows so far). The problem is, that it is not so easy to move it to the main thread atm. Do you have a access to a Vulkan capable computer with Windows or Linux?

Anyways, I attached the output of vulkan info.
vulkaninfo.txt

The gdb backtrace looks like this:

(gdb) backtrace full
#0  0x00007ffff2fcddc3 in ?? () from /usr/lib/amdvlk64.so
No symbol table info available.
#1  0x00007ffff2fcbd59 in ?? () from /usr/lib/amdvlk64.so
No symbol table info available.
#2  0x00007ffff42cc61e in ?? () from /usr/lib/amdvlk64.so
No symbol table info available.
#3  0x00007ffff2fb1d58 in ?? () from /usr/lib/amdvlk64.so
No symbol table info available.
#4  0x00007ffff2f9e494 in ?? () from /usr/lib/amdvlk64.so
No symbol table info available.
#5  0x00007ffff2f9f43f in ?? () from /usr/lib/amdvlk64.so
No symbol table info available.
#6  0x00007ffff2f954fd in ?? () from /usr/lib/amdvlk64.so
No symbol table info available.
#7  0x00007ffff2f6f4e6 in ?? () from /usr/lib/amdvlk64.so
No symbol table info available.
[1]    6096 segmentation fault (core dumped)  gdb target/debug/examples/simple

I never used gdb before (or debuged correctly in general) so I uploaded the core file as well for you. I only runed it once and then generate-core-file. I hope it helps.

[SiebenCorgie]

I got my shader now working. What I did was that I now handle the three variables of the struct TriangleHit explicit instead. So I got a float tvec vec2 uv and int has_hit which are changed if the return value of the line

        TriangleHit leafhit = ray_intersect_mesh(ray, node.left, node.right);

has actually hit something.

So I suspect that something gets screwed up when trying to overwrite a custom struct variable with a new one.
I also tried the old code on windows (with AMD and intel cards) again and it fails there as well, so I think its not a driver problem but more in the layer of the shader compiler / vulkano.

[nicokoch]

Okay, then it's probably an issue with shaderc.
Any reason you use amdvlk instead of radv?

[SiebenCorgie]

I hoped to resolve the issue by changing the driver and then was to lazy to go back. Also, there is a realy nice GPU-Profiler you can use with AMDVLK (RadeonProfiler). But I'll probably go back to RADV at some point.