WIP: rebase llvm to new version which supports symbols as patchpoint call destinations #911

undingen · 2015-09-10T08:37:11Z

Sadly this slows down the tests :-( (just updating llvm - even without using the llvm symbolic target feature). I haven't yet found out why it's slower - everything looks correct.

           django_template3.py             2.7s (4)             2.8s (4)  +4.9%
                 pyxl_bench.py             2.2s (4)             2.2s (4)  +1.3%
     sqlalchemy_imperative2.py             2.6s (4)             2.6s (4)  +0.6%
                       geomean                 2.5s                 2.5s  +2.3%

I didn't port the "Filter-out-extraneous-registers-from-live-outs-like" patch because I didn't know how to verify if it works (maybe that's slowing down things?).
I also didn't add the "getSymbolLoadAddress" patch because I found a way to retrieve the address without the patch.
The patch makes pyston not compatible with an older LLVM version because I didn't add all ifdefs (especially for the create GEPs). Thought I wanted to wait until the new cache is working to know if we need to upgrade.
This only upgrades to Apr 22 llvm revision and not to the current one.
clang-format for multivar decls changed :-(.

but this is enough for getting further with the new cache

rewrite additional nonzero() cases

microoptimizations

rewrite float comparison base on CPython implementation

NumPy is huge, bigger than our previous (arbitrary) number by an order of magnitude.

C extensions (NumPy) might inherit classes in C code and expect to find tp_number. This is just copied from CPython's PyType_Ready. This requires assigning some of the runtime functions to thesq_ and mp_ slots otherwise there are infinite loops from Pyston attributes.

Those should never exist because all Python objects should be created through the CPython API except for type objects. Unfortunately, some places like NumPy do that so we need a mean of patching it for now.

…bjects. Instead of GCKind::HIDDEN_CLASS, use GCKind::RUNTIME and require that the runtime objects have a gc_visit functions by inheriting from GCAllocatedRuntime.

Refactors on types of GC objects

Previously, we would just call these "conservative python" objects, and scan their memory conservatively. The problem with this is that the builtin type might have defined a custom GC handler that needs to be called in addition to the conservative scanning (ie it stores GC pointers out-of-band that are not discoverable via other gc pointers). We had dealt with these kinds of issues before which is why I added the "conservative python kind", but I think the real solution here is to say that to the GC, these objects are just python objects, and then let the type machinery decide how to scan the objects, including how to handle the inheritance rules. We were already putting "conservativeGCHandler" as the gc_handler on these conservative types, so let's use it.

Previously it would have to call out to checkAndThrowCAPIException(), which is quite a bit slower than what it now can do, which is directly checking the return value.

Get Pyston building on Fedora

Improve dictionary performance

Some work on the NumPy test

More small optimizations

This will reallocate all objects in the small heap and update all references that were pointing to this object. This is not functional yet, there are still references that we are not tracking at other points in the program, so it's still gated behind the MOVING_GC flag.

Last few fixes to make sqlalchemy_declarative work

Optionally move objects around in memory to prepare Pyston for a moving collector.

rewrite oldstyle class getattro

ie concerning things like: a, b = 1, 2 The irgen phase already knows how to do unpacking in a type-analyzed way (a and b will be of type int), but type speculation needed to have that added.

Used a hardcoded CXX exception style in the non-rewriteable case.

Type system fix: need to add unpacking to the type system

…CPython with some Pyston changes

PrintVisitor: use raw_ostream

Implment some PyNumber_XXX function, to enable "test_operator"

We already supported changing the values, but not the number of them. The main trickiness here is - We had been assuming that the number of defaults was immutable, so I had to find the places that we used it and add invalidation. - We assumed that all functions based on the same source function would have the same number of defaults. For the first one, I found all the places that looked at the defaults array, which should hopefully be all the places that need invalidation. One tricky part is that we will embed the num_defaults data into code produced by the LLVM tier, and we currently don't have any mechanism for invalidating those functions. This commit side-steps around that since the only functions that we can inline are the builtins, and those you aren't allowed to change the defaults anyway. So I added a "can_change_defaults" flag. For the second part, I moved "num_defaults" from the CLFunction (our "code" object) to the BoxedFunction (our "function" object), and then changed the users to pull it from there.

Allow changing the number of default arguments

Fix set comparisons

Add support for symbolic patchpoint targets to SelectionDAG and the X86 backend.

they changed the * location for multivar decls...

kmod and others added 30 commits August 10, 2015 15:09

Merge pull request pyston#818 from undingen/perf4

a71a661

rewrite additional nonzero() cases

Merge pull request pyston#819 from kmod/perf3

51b19ce

microoptimizations

rewrite float comparison base on CPython implementation

76ec496

Merge pull request pyston#821 from Daetalus/float_comparision

ae12787

rewrite float comparison base on CPython implementation

Add missing #define for NumPy's npy_common.h

6c4a0a1

Increase maximum size of BSS section.

b42cb29

NumPy is huge, bigger than our previous (arbitrary) number by an order of magnitude.

Simple implementation of abs(complex) PyComplex_AsCComplex.

b1b8d67

Stub implementation for Ellipsis.

da72a44

Add function to register nonheap root objects.

477c209

Those should never exist because all Python objects should be created through the CPython API except for type objects. Unfortunately, some places like NumPy do that so we need a mean of patching it for now.

Extension modules might want to have their own tp_free.

30c004f

Update testsuite submodule to include NumPy test.

d837de3

Make GC handling of HiddenClass more general, support other runtime o…

5de3104

…bjects. Instead of GCKind::HIDDEN_CLASS, use GCKind::RUNTIME and require that the runtime objects have a gc_visit functions by inheriting from GCAllocatedRuntime.

Move GC-related declarations to gc folder and add comments.

4c99ad4

Get Pyston building on Fedora

a0ce81f

Merge pull request pyston#820 from rudi-c/gc_types

45b15b3

Refactors on types of GC objects

Misc fixes / helpers

c30e503

Dict change: scan manually instead of conservatively

7c6b521

Switch BoxedDict to llvm::DenseMap

779cb5b

Have the rewriter check for CAPI excs directly

893dbbb

Previously it would have to call out to checkAndThrowCAPIException(), which is quite a bit slower than what it now can do, which is directly checking the return value.

Support checking for return codes other than NULL

8813c42

Merge pull request pyston#823 from kmod/fedora_fixes

cd5a4d0

Get Pyston building on Fedora

Merge pull request pyston#822 from kmod/perf4

76c4219

Improve dictionary performance

Microoptimizations

be7aae7

Have rearrangeArguments return in place

61a68d3

Optimize some type-checking

ee5b6d4

Merge pull request pyston#783 from rudi-c/numpy_fix

109df64

Some work on the NumPy test

Merge pull request pyston#824 from kmod/perf4

47d0270

More small optimizations

Remove the broken LLVM rules in the Makefile

f0efd51

rudi-c and others added 21 commits September 4, 2015 15:10

ifdef out some moving GC code for now.

8a510e3

Merge pull request pyston#895 from kmod/sqlalchemy

db991b3

Last few fixes to make sqlalchemy_declarative work

Merge pull request pyston#889 from rudi-c/movingmerge

7b84d99

Optionally move objects around in memory to prepare Pyston for a moving collector.

rewrite oldstyle class getattro

4975866

Merge pull request pyston#899 from undingen/perf_oldstyle2

9a3a43c

rewrite oldstyle class getattro

Type system fix: need to add unpacking to the type system

414d207

ie concerning things like: a, b = 1, 2 The irgen phase already knows how to do unpacking in a type-analyzed way (a and b will be of type int), but type speculation needed to have that added.

Callattr fix

80f4bc3

Used a hardcoded CXX exception style in the non-rewriteable case.

Merge pull request pyston#901 from kmod/speculation_fix

1782dd5

Type system fix: need to add unpacking to the type system

enable test_operator

b8e204b

get slice copy before calculate the posistion

1eb9a9a

Implement some PyNumber_XXX function, most of the code are copy from …

c66b6f1

…CPython with some Pyston changes

add PySequence_SetItem and GetItem

5bacdd4

PrintVisitor: use raw_ostream

fe6885a

Merge pull request pyston#904 from undingen/printvisitor

f96c49a

PrintVisitor: use raw_ostream

Merge pull request pyston#900 from Daetalus/test_operator

2294c2a

Implment some PyNumber_XXX function, to enable "test_operator"

I think this test was in the wrong file

cd43a0f

Merge pull request pyston#905 from kmod/change_numdefaults

54a9559

Allow changing the number of default arguments

Fix set comparisons

b91071c

Merge pull request pyston#908 from undingen/fix_set_cmp

9df41bb

Fix set comparisons

undingen force-pushed the llvm_upgrade branch 2 times, most recently from 9f0eddd to 3df9934 Compare September 10, 2015 14:18

undingen added 3 commits September 10, 2015 19:57

rebase llvm to r235483 Apr 22 2015

5ce93e8

Add support for symbolic patchpoint targets to SelectionDAG and the X86 backend.

Remove workaround for symbolic patchpoint targets

8fde2cc

run clang-format

6ae7e32

they changed the * location for multivar decls...

undingen force-pushed the llvm_upgrade branch from 3df9934 to 6ae7e32 Compare September 10, 2015 19:00

kmod added the wip label Sep 14, 2015

kmod force-pushed the master branch 2 times, most recently from 352fd89 to 6488a3e Compare October 28, 2020 21:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: rebase llvm to new version which supports symbols as patchpoint call destinations #911

WIP: rebase llvm to new version which supports symbols as patchpoint call destinations #911

undingen commented Sep 10, 2015

WIP: rebase llvm to new version which supports symbols as patchpoint call destinations #911

Are you sure you want to change the base?

WIP: rebase llvm to new version which supports symbols as patchpoint call destinations #911

Conversation

undingen commented Sep 10, 2015