Total rewrite of the package (aka make PackageCompilerX the new PackageCompiler) #304

KristofferC · 2020-02-09T14:22:55Z

This PR completely replaces PackageCompiler.jl with that of PackageCompilerX.jl. The maintainers of this package as well as those of PackageCompilerX.jl feels like this is the best way forward.
A few options were considered:

Just deprecate this repo, rename PackageCompilerX to something else (AppCompiler.jl). There are some drawbacks to this. Firstly, this package name is already pretty good and it would be a shame to waste it. Secondly, it is a bit confusing to have a lot of references spread out over the internet and then have it be deprecated. Also, it would remove the history of everyone contributing to this package which is a shame.
Just repoint the URL in the registry to PackageCompilerX. This would cause existing manifests to break so it is a non-starter.

Instead, just putting a complete rewrite on top of the existing history felt like the best choice.

Some improvements / changes in PackageCompilerX.jl vs the current PackageCompiler:

Support for artifacts in Apps. This allows one to distribute apps that depend on e.g. libraries via the artifact system to other machines.
More CI, including e.g. ARM support.
Better defaults for cpu targets, matching those that Julia itself uses
Proper documentation including some examples, tutorials and developer docs.
No command line interface, this felt like adding a lot of code for little benefit.
Uses the Artifact system to get a compiler for Windows instead of the largely deprecated WinRPM.
Drops pre 1.3.1 Julia support.
There is no need to annotate the entry points of apps with Base.@ccallable since that is done automatically.
The julia_main function no longer take any arguments. Just like in normal Julia code, just access the global ARGS.
An app needs to be structured as a package. (project file and src/Package.jl file)

I will go through earlier issues and add here which are either fixed by this PR, no longer relevant due to fixes in Julia or changes in PackageCompiler functionality:

Closes #52
Closes #83
Closes #108
Closes #129
Closes #131
Closes #289
Closes #297
Closes #290
Closes #281
Closes #192
Closes #277
Closes #276
Closes #272
Closes #271
Closes #270
Closes #258
Closes #255
Closes #252
Closes #234
Closes #233
Closes #230
Closes #236
Closes #228
Closes #225
Closes #223
Closes #222
Closes #221
Closes #220
Closes #215
Closes #212
Closes #208
Closes #202
Closes #201
Closes #194
Closes #193
Closes #190
Closes #184
Closes #223
Closes #182
Closes #180
Closes #163
Closes #151
Closes #155
Closes #154
Closes #152
Closes #145
Closes #141
Closes #226
Closes #140
Closes #138
Closes #117
Closes #76
Closes #59

Got kinda lost with isues at this point. Will take another pass after this PR is merged.

After this PR is mergd, we should transfer over issues PR from PackgeCompilerX to this repo.

make a hardcoded app

* implement support for precompile file

… it. (#13)

* wip only using needed stdlibs * only include needed stdlibs in apps * make default off

Don't add an rpath to local absolute path

* improvements to cpu targets * more * more * remove precompile from testing due to JuliaLang/julia#34076

* wip make app work on windows * wip win * test artifact load path * realpath

SimonDanisch · 2020-02-09T17:20:13Z

Yay :)

DilumAluthge · 2020-02-09T18:28:19Z

@KristofferC Are you planning on making a release as soon as you merge this PR?

If not, it might be better to set the version to 1.0.0-DEV.

SimonDanisch · 2020-02-10T18:14:22Z

Awesome :) Do we want to try to fix the CI, or is that not really doable, since tests take so long? Could we split up stuff?

SimonDanisch · 2020-02-10T18:15:09Z

No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.

I guess this time out can be upped?

KristofferC · 2020-02-10T18:20:01Z

Strange, it passed on ARM in the previous commit and it took like 25 min. Maybe it was barely though.

* JuliaComputing -> JuliaLang * Build documentation on correct conditions push to master and pull request based on master

DilumAluthge · 2020-02-10T20:05:49Z

Travis is passing.

It looks like the AppVeyor error isn’t actually a test failure, but a problem with the configuration of AppVeyor on this repository.

How about we uninstall AppVeyor from this repo and use Travis for the Windows tests? After all, this PR deletes the .appveyor.yml file, which implies that we aren’t planning on using AppVeyor.

I think you need to be an organization admin of the JuliaLang organization in order to uninstall the AppVeyor app from this repo. @ViralBShah @StefanKarpinski

SimonDanisch · 2020-02-10T20:17:43Z

LICENSE

@@ -0,0 +1,19 @@
+Copyright (c) 2019 Kristoffer Carlsson <kristoffer.carlsson@juliacomputing.com>


Hm, I mean I see that this code is mainly by you, but would it make sense to leave me in here, or do we just say, its in the history?

Maybe something like: Kristoffer, Simon, Luca, and other contributors? Just to be comprehensive.

That sounds fair!

I would appreciate that, as I put a lot of effort on PackageCompiler.jl development.

Definitely ;) That I was alone in the license was wrong to begin with!

Can someone make a suggestion here so it can just be accepted? Otherwise, we absolutely must add the names. Also let's add Julia Computing to the mix please.

Yeah I'll make a suggestion

Sorry, didn't think about this, it was just the default that got generated by PkgTemplates. Let's merge all names in here :)

ViralBShah · 2020-02-10T22:03:04Z

I deleted the appveyor webhook.

LICENSE

Co-Authored-By: Dilum Aluthge <dilum@aluthge.com>

NHDaly · 2020-02-13T18:58:10Z

Hooray! Thanks for the hard work on this, everyone! 😁 The future is bright!

KristofferC and others added 30 commits February 9, 2020 14:42

clear everything out

23748d0

Files generated by PkgTemplates

7d84910

test build script from PackageCompiler (#1)

a5a55a3

make a hardcoded app

implement support for precompile file (#5)

8e94994

* implement support for precompile file

use gcc for now

f1e4ee4

allow for multiple packages

c4e4338

some generalizations

3666883

add an example (#7)

80ccbe1

use Travis for docs (#8)

bc76828

fixups

7b15882

fix windows

48fb117

start with an create_app function

f421ab9

Implement a create app function that bundles everything needed to run…

3498873

… it. (#13)

add an audit function (#17)

613f7c2

add option to compile with incremental=false and set to default (#19)

b59a7cb

fix on macos (#21)

c6e2bac

only include needed stdlibs in apps (#25)

d6c0109

* wip only using needed stdlibs * only include needed stdlibs in apps * make default off

try not use rpath to local julia install on linux (#23)

5f0a90e

Don't add an rpath to local absolute path

Initial work on docs (#34)

76ed66c

add docs for apps (#37)

a3372bc

fix support

0ec0309

fix typo in docs

8692e96

docs: fix typo (#40)

8a8c612

fix a typo (#41)

2dac507

add a mention that this is still WIP

c43f07c

Create CompatHelper.yml (#48)

4cb1fd2

CompatHelper: only update root manifest (#50)

15eddce

improvements to cpu targets (#43)

8f7d453

* improvements to cpu targets * more * more * remove precompile from testing due to JuliaLang/julia#34076

fix some links

a9d336a

WIP on making app work on windows. (#55)

4031766

* wip make app work on windows * wip win * test artifact load path * realpath

use better defaults for march and cpu target (#105)

817a366

asinghvi17 and others added 2 commits February 9, 2020 19:47

Allow PRs to push preview documentation (#305)

54f89dc

Add default value (no packages) to create_sysimage. (#306)

c02c4e7

Some fixes for documentation build and deployment (#307)

00462ee

* JuliaComputing -> JuliaLang * Build documentation on correct conditions push to master and pull request based on master

SimonDanisch approved these changes Feb 10, 2020

View reviewed changes

SimonDanisch reviewed Feb 10, 2020

View reviewed changes

DilumAluthge requested changes Feb 10, 2020

View reviewed changes

LICENSE Outdated Show resolved Hide resolved

add back authors to LICENSE file

2742c71

Co-Authored-By: Dilum Aluthge <dilum@aluthge.com>

SimonDanisch merged commit 42b4353 into master Feb 10, 2020

SimonDanisch deleted the kc/packagecompilerx branch February 10, 2020 22:58

DilumAluthge mentioned this pull request Feb 11, 2020

Re-enable TagBot on this repository #312

Merged

tkf mentioned this pull request Feb 11, 2020

Use PackageCompiler.jl 1.0 (and drop Julia < 1.3.1 for custom sysimage) JuliaPy/pyjulia#360

Closed

KristofferC mentioned this pull request Feb 11, 2020

Function default_sysimg_path #71

Closed

terasakisatoshi mentioned this pull request Feb 11, 2020

Use package compiler not x terasakisatoshi/MyWorkflow.jl#18

Merged

Alexander-Barth mentioned this pull request Feb 13, 2020

Moving to an org / making a golden standard for NetCDF files in Julia JuliaGeo/NCDatasets.jl#57

Closed

OliverEvans96 mentioned this pull request Apr 15, 2020

Question: How do we use build_shared_lib function Properly ? #257

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Total rewrite of the package (aka make PackageCompilerX the new PackageCompiler) #304

Total rewrite of the package (aka make PackageCompilerX the new PackageCompiler) #304

KristofferC commented Feb 9, 2020 •

edited

Loading

SimonDanisch commented Feb 9, 2020

DilumAluthge commented Feb 9, 2020

SimonDanisch commented Feb 10, 2020

SimonDanisch commented Feb 10, 2020

KristofferC commented Feb 10, 2020 •

edited

Loading

DilumAluthge commented Feb 10, 2020

SimonDanisch Feb 10, 2020

DilumAluthge Feb 10, 2020

SimonDanisch Feb 10, 2020

lucatrv Feb 10, 2020

SimonDanisch Feb 10, 2020

ViralBShah Feb 10, 2020

DilumAluthge Feb 10, 2020

KristofferC Feb 10, 2020 •

edited

Loading

ViralBShah commented Feb 10, 2020

NHDaly commented Feb 13, 2020

		@@ -0,0 +1,19 @@
		Copyright (c) 2019 Kristoffer Carlsson <kristoffer.carlsson@juliacomputing.com>

Total rewrite of the package (aka make PackageCompilerX the new PackageCompiler) #304

Total rewrite of the package (aka make PackageCompilerX the new PackageCompiler) #304

Conversation

KristofferC commented Feb 9, 2020 • edited Loading

SimonDanisch commented Feb 9, 2020

DilumAluthge commented Feb 9, 2020

SimonDanisch commented Feb 10, 2020

SimonDanisch commented Feb 10, 2020

KristofferC commented Feb 10, 2020 • edited Loading

DilumAluthge commented Feb 10, 2020

SimonDanisch Feb 10, 2020

Choose a reason for hiding this comment

DilumAluthge Feb 10, 2020

Choose a reason for hiding this comment

SimonDanisch Feb 10, 2020

Choose a reason for hiding this comment

lucatrv Feb 10, 2020

Choose a reason for hiding this comment

SimonDanisch Feb 10, 2020

Choose a reason for hiding this comment

ViralBShah Feb 10, 2020

Choose a reason for hiding this comment

DilumAluthge Feb 10, 2020

Choose a reason for hiding this comment

KristofferC Feb 10, 2020 • edited Loading

Choose a reason for hiding this comment

ViralBShah commented Feb 10, 2020

NHDaly commented Feb 13, 2020

KristofferC commented Feb 9, 2020 •

edited

Loading

KristofferC commented Feb 10, 2020 •

edited

Loading

KristofferC Feb 10, 2020 •

edited

Loading