[tools] Add proper Dev Container

[benz0li]

Test online with GitHub Codespaces: https://codespaces.new/benz0li/mojo?hide_repo_select=true&ref=add-proper-dev-container

Parent image: glcr.b-data.ch/mojo/base:nightly
👉 Tag nightly is rebuilt at the start of every 4th hour.

[benz0li]

@gabrieldemarmiesse I am keeping this as a draft for now.

Currently, there is an externally managed Mojo Dev Container available. Cross reference:

• [docs] Add section 'Dev Container' to the Mojo stdlib docs #2664

[benz0li]

Not to myself: Dev Containers do not preserve the user's home directory by default. markdownlint is installed in the user's home directory, though.

This results in markdownlint being reinstalled every time the 'Dev Container' (VS Code, local) or Codespace (GitHub, web) is rebuilt.

(No problem with the externally managed Mojo Dev Container as it preserved the user's home directory)

[benz0li]

@JoeLoser Could you please review, i.e. answer (or give input to) the above questions some time?

Thank you.

[benz0li]

FYI @MussaCharles If this is merged, my Mojo dev container is obsolete.

[MussaCharles]

FYI @MussaCharles If this is merged, my Mojo dev container is obsolete.

Hi @benz0li , thanks for the update and for adding devcontainer support. I'll definitely be keeping an eye on this.

[JoeLoser]

@benz0li this is great, thanks for working on this! I left a few comments, but I'm open to having this in the mojo repo.

@gabrieldemarmiesse — do you have any feedback for this? I think you mentioned having experience using Devcontainers in the past when we briefly talked about this a while back.

[JoeLoser]

Suggestion Is it possible to use the same script CI is using (/~https://github.com/modularml/mojo/blob/405e369b84ce32fe87f617b210eef81867371bdf/stdlib/scripts/install-build-tools-linux.sh) for downloading LLVM and setting up the various alternatives? It may lead to some reduced maintenance/drift over time. I think we'll just want to teach /~https://github.com/modularml/mojo/blob/405e369b84ce32fe87f617b210eef81867371bdf/stdlib/scripts/install-build-tools-linux.sh to take an argument for the LLVM version and specify it here in the Dockerfile.

[MussaCharles]

Hi @benz0li , I tried running the test cases in the examples folder, but some of them failed. I'm not sure if these are breaking changes from the nightly build or if it's something related to the Dev Container configurations. For reference, I'm using GitHub Codespaces to open the container.
cc: @JoeLoser

Please see the attached output of examples/run-examples.sh script.

@MussaCharles ➜ /workspaces/mojo/examples (add-proper-dev-container) $ ./run-examples.sh 
FAIL: Mojo Public Examples :: mandelbrot.mojo (1 of 5)
******************** TEST 'Mojo Public Examples :: mandelbrot.mojo' FAILED ********************
Exit Code: 2

Command Output (stderr):
--
RUN: at line 14: mojo /workspaces/mojo/examples/mandelbrot.mojo | FileCheck /workspaces/mojo/examples/mandelbrot.mojo
+ mojo /workspaces/mojo/examples/mandelbrot.mojo
+ FileCheck /workspaces/mojo/examples/mandelbrot.mojo
/workspaces/mojo/examples/mandelbrot.mojo:24:24: error: use of unknown declaration 'simdwidthof'
alias simd_width = 2 * simdwidthof[float_type]()
                       ^~~~~~~~~~~
/workspaces/mojo/examples/mandelbrot.mojo:38:15: error: 'Matrix' has no 'DTypePointer' member
    var data: DTypePointer[type]
              ^~~~~~~~~~~~
/workspaces/mojo/examples/mandelbrot.mojo:81:69: error: invalid call to '__mul__': could not deduce parameter 'type' of parent struct 'SIMD'
            var cx = min_x + (col + iota[float_type, simd_width]()) * scale_x
                             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~
/workspaces/mojo/examples/mandelbrot.mojo:1:1: note:  struct declared here
# ===----------------------------------------------------------------------=== #
^
/workspaces/mojo/examples/mandelbrot.mojo:81:71: note: failed to infer parameter #0, parameter inferred to two different values: 'float32' and 'float64'
            var cx = min_x + (col + iota[float_type, simd_width]()) * scale_x
                                                                      ^~~~~~~
/workspaces/mojo/examples/mandelbrot.mojo:1:1: note: function declared here
# ===----------------------------------------------------------------------=== #
^
/workspaces/mojo/examples/mandelbrot.mojo:108:11: error: 'Matrix[int32, 960, 960]' value has no attribute 'data'
    matrix.data.free()
    ~~~~~~^
/workspaces/mojo/examples/mandelbrot.mojo:41:21: error: use of unknown declaration 'DTypePointer'
        self.data = DTypePointer[type].alloc(rows * cols)
                    ^~~~~~~~~~~~
/workspaces/mojo/examples/mandelbrot.mojo:44:13: error: 'Matrix[type, rows, cols]' value has no attribute 'data'
        self.data.store[width=nelts](row * cols + col, val)
        ~~~~^
mojo: error: failed to parse the provided Mojo source module
FileCheck error: '<stdin>' is empty.
FileCheck command line:  FileCheck /workspaces/mojo/examples/mandelbrot.mojo

--

********************
FAIL: Mojo Public Examples :: deviceinfo.mojo (2 of 5)
******************** TEST 'Mojo Public Examples :: deviceinfo.mojo' FAILED ********************
Exit Code: 2

Command Output (stderr):
--
RUN: at line 13: mojo /workspaces/mojo/examples/deviceinfo.mojo | FileCheck /workspaces/mojo/examples/deviceinfo.mojo
+ mojo /workspaces/mojo/examples/deviceinfo.mojo
+ FileCheck /workspaces/mojo/examples/deviceinfo.mojo
/workspaces/mojo/examples/deviceinfo.mojo:18:5: error: module 'info' does not contain '_current_cpu'
    _current_cpu,
    ^
mojo: error: failed to parse the provided Mojo source module
FileCheck error: '<stdin>' is empty.
FileCheck command line:  FileCheck /workspaces/mojo/examples/deviceinfo.mojo

--

********************
FAIL: Mojo Public Examples :: reduce.mojo (3 of 5)
******************** TEST 'Mojo Public Examples :: reduce.mojo' FAILED ********************
Exit Code: 2

Command Output (stderr):
--
RUN: at line 13: mojo /workspaces/mojo/examples/reduce.mojo | FileCheck /workspaces/mojo/examples/reduce.mojo
+ FileCheck /workspaces/mojo/examples/reduce.mojo
+ mojo /workspaces/mojo/examples/reduce.mojo
/workspaces/mojo/examples/reduce.mojo:82:21: error: use of unknown declaration 'DTypePointer'
    var ptr_small = DTypePointer[type].alloc(size_small)
                    ^~~~~~~~~~~~
/workspaces/mojo/examples/reduce.mojo:83:21: error: use of unknown declaration 'DTypePointer'
    var ptr_large = DTypePointer[type].alloc(size_large)
                    ^~~~~~~~~~~~
mojo: error: failed to parse the provided Mojo source module
FileCheck error: '<stdin>' is empty.
FileCheck command line:  FileCheck /workspaces/mojo/examples/reduce.mojo

--

********************
                                                                   -- Testing: 5 tests, 2 workers --                                                                    
********************
Failed Tests (3):
  Mojo Public Examples :: deviceinfo.mojo
  Mojo Public Examples :: mandelbrot.mojo
  Mojo Public Examples :: reduce.mojo


Testing Time: 4.22s

Total Discovered Tests: 5
  Passed: 2 (40.00%)
  Failed: 3 (60.00%)

[benz0li]

Hi @benz0li , I tried running the test cases in the examples folder, but some of them failed. I'm not sure if these are breaking changes from the nightly build or if it's something related to the Dev Container configurations.

IMHO the code examples are not up-to-date for Mojo nightly.

After checking out branch nightly¹, running the standard library tests (i.e. ./stdlib/scripts/run-tests.sh) must succeed.

Footnotes

1. Do not rebuild the container! ↩

[MussaCharles]

IMHO the code examples are not up-to-date for Mojo nightly.

After checking out branch nightly1, running the standard library tests (i.e. ./stdlib/scripts/run-tests.sh) must succeed.

Footnotes

1. Do not rebuild the container! ↩

Thanks for detailed explanations. @benz0li

[JoeLoser]

Still very excited for this as it would be a big lift for contributors, but I think now that we have magic to replace the modular CLI tool, we'll need to change a few things in the Dockerfile.

Have you checked recently to see if the prelude changes I made in the 24.5 release have fixed the examples for you? They should all be up-to-date as they run in our open source CI.

[JoeLoser]

Suggestion I recommend changing this a bit to use magic now. See https://docs.modular.com/magic/ for more info.

[benz0li]

Closed in favour of #3618.