[do not merge] Proof of concept implementation of forward compatible v0 symbols (MCP 737)

[michaelwoerister]

This PR changes v0 symbol names to enable graceful degradation for demanglers that don't support newer language features yet (as described in MCP 737).

The corresponding changes in rustc-demangle are at /~https://github.com/michaelwoerister/rustc-demangle/tree/skip_unknown.

This proof-of-concept implementation is meant to help evaluating if MCP 737 is viable.

r? @ghost

[michaelwoerister]

@bors try @rust-timer queue

[bors]

⌛ Trying commit 17c7fc4 with merge df5a899...

[rust-log-analyzer]

The job mingw-check-tidy failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

Getting action download info
Download action repository 'msys2/setup-msys2@v2.22.0' (SHA:cc11e9188b693c2b100158c3322424c4cc1dadea)
Download action repository 'actions/checkout@v4' (SHA:a5ac7e51b41094c92402da3b24376905380afc29)
Download action repository 'actions/upload-artifact@v4' (SHA:65462800fd760344b1a7b4382951275a0abb4808)
Complete job name: PR - mingw-check-tidy
git config --global core.autocrlf false
shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
---
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh

COPY host-x86_64/mingw-check/reuse-requirements.txt /tmp/
RUN pip3 install --no-deps --no-cache-dir --require-hashes -r /tmp/reuse-requirements.txt \
    && pip3 install virtualenv
COPY host-x86_64/mingw-check/validate-toolstate.sh /scripts/
COPY host-x86_64/mingw-check/validate-error-codes.sh /scripts/

# NOTE: intentionally uses python2 for x.py so we can test it still works.
# NOTE: intentionally uses python2 for x.py so we can test it still works.
# validate-toolstate only runs in our CI, so it's ok for it to only support python3.
ENV SCRIPT TIDY_PRINT_DIFF=1 python2.7 ../x.py test \
           --stage 0 src/tools/tidy tidyselftest --extra-checks=py:lint
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
#    pip-compile --allow-unsafe --generate-hashes reuse-requirements.in
---

#12 [5/8] COPY host-x86_64/mingw-check/reuse-requirements.txt /tmp/
#12 DONE 0.0s

#13 [6/8] RUN pip3 install --no-deps --no-cache-dir --require-hashes -r /tmp/reuse-requirements.txt     && pip3 install virtualenv
#13 0.422   Downloading binaryornot-0.4.4-py2.py3-none-any.whl (9.0 kB)
#13 0.437 Collecting boolean-py==4.0
#13 0.441   Downloading boolean.py-4.0-py3-none-any.whl (25 kB)
#13 0.454 Collecting chardet==5.1.0
---
#13 3.533 Building wheels for collected packages: reuse
#13 3.534   Building wheel for reuse (pyproject.toml): started
#13 3.855   Building wheel for reuse (pyproject.toml): finished with status 'done'
#13 3.856   Created wheel for reuse: filename=reuse-1.1.0-cp310-cp310-manylinux_2_35_x86_64.whl size=181117 sha256=f5f58750481f69515c2c0d1d503daf565e2565c370d07fc6aeb95fe3498b4269
#13 3.856   Stored in directory: /tmp/pip-ephem-wheel-cache-afyt1o5y/wheels/c2/3c/b9/1120c2ab4bd82694f7e6f0537dc5b9a085c13e2c69a8d0c76d
#13 3.859 Installing collected packages: boolean-py, binaryornot, setuptools, reuse, python-debian, markupsafe, license-expression, jinja2, chardet
#13 3.880   Attempting uninstall: setuptools
#13 3.881     Found existing installation: setuptools 59.6.0
#13 3.882     Not uninstalling setuptools at /usr/lib/python3/dist-packages, outside environment /usr
#13 3.882     Not uninstalling setuptools at /usr/lib/python3/dist-packages, outside environment /usr
#13 3.883     Can't uninstall 'setuptools'. No files were found to uninstall.
#13 4.544 Successfully installed binaryornot-0.4.4 boolean-py-4.0 chardet-5.1.0 jinja2-3.1.2 license-expression-30.0.0 markupsafe-2.1.1 python-debian-0.1.49 reuse-1.1.0 setuptools-66.0.0
#13 4.544 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
#13 5.055 Collecting virtualenv
#13 5.095   Downloading virtualenv-20.26.2-py3-none-any.whl (3.9 MB)
#13 5.154      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.9/3.9 MB 69.0 MB/s eta 0:00:00
#13 5.189 Collecting distlib<1,>=0.3.7
#13 5.195   Downloading distlib-0.3.8-py2.py3-none-any.whl (468 kB)
#13 5.201      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 468.9/468.9 KB 121.6 MB/s eta 0:00:00
#13 5.229 Collecting platformdirs<5,>=3.9.1
#13 5.233   Downloading platformdirs-4.2.2-py3-none-any.whl (18 kB)
#13 5.268 Collecting filelock<4,>=3.12.2
#13 5.271   Downloading filelock-3.14.0-py3-none-any.whl (12 kB)
#13 5.356 Installing collected packages: distlib, platformdirs, filelock, virtualenv
#13 5.517 Successfully installed distlib-0.3.8 filelock-3.14.0 platformdirs-4.2.2 virtualenv-20.26.2
#13 DONE 5.6s

#14 [7/8] COPY host-x86_64/mingw-check/validate-toolstate.sh /scripts/
#14 DONE 0.0s
---
DirectMap4k:      200640 kB
DirectMap2M:     7139328 kB
DirectMap1G:    11534336 kB
##[endgroup]
Executing TIDY_PRINT_DIFF=1 python2.7 ../x.py test            --stage 0 src/tools/tidy tidyselftest --extra-checks=py:lint
+ TIDY_PRINT_DIFF=1 python2.7 ../x.py test --stage 0 src/tools/tidy tidyselftest --extra-checks=py:lint
    Finished `dev` profile [unoptimized] target(s) in 0.03s
##[endgroup]
downloading https://ci-artifacts.rust-lang.org/rustc-builds-alt/464987730ab36e3456ba57c53965372989382deb/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
extracting /checkout/obj/build/cache/llvm-464987730ab36e3456ba57c53965372989382deb-true/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz to /checkout/obj/build/x86_64-unknown-linux-gnu/ci-llvm
extracting /checkout/obj/build/cache/llvm-464987730ab36e3456ba57c53965372989382deb-true/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz to /checkout/obj/build/x86_64-unknown-linux-gnu/ci-llvm
##[group]Building stage0 tool tidy (x86_64-unknown-linux-gnu)
warning: /checkout/library/core/Cargo.toml: unused manifest key: lints.rust.unexpected_cfgs.check-cfg
warning: /checkout/library/std/Cargo.toml: unused manifest key: lints.rust.unexpected_cfgs.check-cfg
warning: /checkout/library/alloc/Cargo.toml: unused manifest key: lints.rust.unexpected_cfgs.check-cfg
    Updating crates.io index
    Updating git repository `/~https://github.com/michaelwoerister/rustc-demangle.git`
---
    Finished `release` profile [optimized] target(s) in 25.40s
##[endgroup]
fmt check
tidy check
tidy error: invalid source: "git+/~https://github.com/michaelwoerister/rustc-demangle.git?branch=skip_unknown#7e674e483a7c61803edc8d59f2641a4ad52a1a30"
removing old virtual environment
removing old virtual environment
creating virtual environment at '/checkout/obj/build/venv' using 'python3.10'
Requirement already satisfied: pip in ./build/venv/lib/python3.10/site-packages (24.0)
Collecting black==23.3.0 (from -r /checkout/src/tools/tidy/config/requirements.txt (line 7))
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 52.5 MB/s eta 0:00:00
Collecting click==8.1.3 (from -r /checkout/src/tools/tidy/config/requirements.txt (line 34))
  Downloading click-8.1.3-py3-none-any.whl (96 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.6/96.6 kB 28.5 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.6/96.6 kB 28.5 MB/s eta 0:00:00
Collecting importlib-metadata==6.7.0 (from -r /checkout/src/tools/tidy/config/requirements.txt (line 38))
  Downloading importlib_metadata-6.7.0-py3-none-any.whl (22 kB)
Collecting mypy-extensions==1.0.0 (from -r /checkout/src/tools/tidy/config/requirements.txt (line 42))
  Downloading mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)
Collecting packaging==23.1 (from -r /checkout/src/tools/tidy/config/requirements.txt (line 46))
  Downloading packaging-23.1-py3-none-any.whl (48 kB)
Collecting pathspec==0.11.1 (from -r /checkout/src/tools/tidy/config/requirements.txt (line 50))
  Downloading pathspec-0.11.1-py3-none-any.whl (29 kB)
Collecting platformdirs==3.6.0 (from -r /checkout/src/tools/tidy/config/requirements.txt (line 54))
  Downloading platformdirs-3.6.0-py3-none-any.whl (16 kB)
  Downloading platformdirs-3.6.0-py3-none-any.whl (16 kB)
Collecting ruff==0.0.272 (from -r /checkout/src/tools/tidy/config/requirements.txt (line 58))
  Downloading ruff-0.0.272-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.9 MB)
Collecting tomli==2.0.1 (from -r /checkout/src/tools/tidy/config/requirements.txt (line 77))
  Downloading tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting typed-ast==1.5.4 (from -r /checkout/src/tools/tidy/config/requirements.txt (line 81))
  Downloading typed_ast-1.5.4-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (877 kB)
  Downloading typed_ast-1.5.4-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (877 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 877.7/877.7 kB 129.6 MB/s eta 0:00:00
Collecting typing-extensions==4.6.3 (from -r /checkout/src/tools/tidy/config/requirements.txt (line 107))
  Downloading typing_extensions-4.6.3-py3-none-any.whl (31 kB)
Collecting zipp==3.15.0 (from -r /checkout/src/tools/tidy/config/requirements.txt (line 114))
  Downloading zipp-3.15.0-py3-none-any.whl (6.8 kB)
Installing collected packages: zipp, typing-extensions, typed-ast, tomli, ruff, platformdirs, pathspec, packaging, mypy-extensions, click, importlib-metadata, black
Successfully installed black-23.3.0 click-8.1.3 importlib-metadata-6.7.0 mypy-extensions-1.0.0 packaging-23.1 pathspec-0.11.1 platformdirs-3.6.0 ruff-0.0.272 tomli-2.0.1 typed-ast-1.5.4 typing-extensions-4.6.3 zipp-3.15.0
some tidy checks failed
Build completed unsuccessfully in 0:00:58
  local time: Fri May 24 12:26:21 UTC 2024
  network time: Fri, 24 May 2024 12:26:21 GMT

[bors]

☀️ Try build successful - checks-actions
Build commit: df5a899 (df5a8997fafb6125bad45edd15f03ebfce52e359)

[michaelwoerister]

The above perf run includes the changes due to making v0 the default.

The performance comparison that is actually relevant uses v0=default as baseline:
https://perf.rust-lang.org/compare.html?start=6d746cd09206996abb0d30d184212d972b0328a8&end=df5a8997fafb6125bad45edd15f03ebfce52e359&stat=instructions%3Au

[michaelwoerister]

I'm somewhat skeptical about the results though, since check builds show up as being affected the most, but as far as I know check builds don't generate symbol names much, if at all.

I'll investigate with cachegrind diff when I get a chance.

[michaelwoerister]

We discussed this in the compiler team design meeting on Jun 14, and decided to not pursue this any further for now.