Automated monthly GMT 6.5.0 dev builds #100

weiji14 · 2020-07-28T10:04:35Z

Starting a weekly GMT development build as mentioned in #95, so that users can install a more up to date version of GMT using:

conda install -c conda-forge/label/dev gmt

Note that there's a few GMT 6.2.0 dev conda packages already at https://anaconda.org/conda-forge/gmt/files.

Workflow for the bot:

On schedule, get latest GMT sha hash from Github (use git ls-remote)
Update recipe/meta.yaml file with new GMT package version and sha hash, should look like 6.2.0.dev0+abcdefg (use yq)
~~3. Push the changed recipe/meta.yaml file to a branch called 6.2.0.devX+something, where X is some version number (use stefanz/git-auto-commit-action)~~
~~4. Create a pull request for that branch (use vsoch/pull-request-action or repo-sync/pull-request)~~
Just use peter-evans/create-pull-request which commits changed files to a branch and automatically creates the Pull Request!

Notes:

This should work on Linux, macOS and Windows!
See also CFEP05 - Providing pre-release builds on the conda-forge anaconda channel and PEP0440 - Version Identification and Dependency Specification
Conda-forge packages which have the dev label https://anaconda.org/conda-forge/repo?label=dev

TODO:

Bump to latest GMT github sha revision automatically (see https://stackoverflow.com/questions/1161869/how-to-get-sha-of-the-latest-commit-from-remote-git-repository)
Automate pull request creation ~~and auto-merge~~
Review for any potential issues

👉 Manual instructions copied from #100 (comment)

Notes on how to be a bot and bump dev version locally. Assumes origin='weiji14', upstream='conda-forge' and GenericMappingTools='GenericMappingTools'. Need to ensure that GenericMappingTools/gmt-feedstock is in sync with conda-forge/gmt-feedstock first!!
git switch gmt-dev-builds  # Change to local gmt-dev-builds branch
git pull GenericMappingTools gmt-dev-builds  # Pull in changes from GenericMappingTools/gmt-dev-builds remote branch
git merge upstream/devel  # Merge in conda-forge/gmt-feedstock devel remote branch
git cherry-pick origin/gmt-dev-builds  # I.e. the 'Temp trigger auto PR commit' commit
git push -f origin
⚠️ Warning: Double check the git graph visually before doing the force push!

Checklist

Used a personal fork of the feedstock to propose changes
Bumped the build number (if the version is unchanged)
Reset the build number to 0 (if the version changed)
Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
Ensured the license file is being packaged.

Fixes #95

conda-forge-linter · 2020-07-28T10:04:41Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipe:

License is not an SPDX identifier (or a custom LicenseRef) nor an SPDX license expression.

Documentation on acceptable licenses can be found here.

conda-forge-linter · 2020-07-28T10:08:36Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

.github/workflows/bumpdevversion.yml

recipe/meta.yaml

seisman · 2020-07-29T06:05:38Z

@weiji14 You also need to update bld.bat and build.sh files in the devel branch. They're a little outdated.

seisman · 2020-07-29T06:13:35Z

.github/workflows/bumpdevversion.yml

+      - name: Check meta.yaml contents
+        run: cat recipe/meta.yaml
+
+      # TODO Commit changed recipe/meta.yaml file to Github


@weiji14 I believe we can't add the workflow here.

To update a package, we are forced to

Used a personal fork of the feedstock to propose changes

Please see the link. The main reason is

Branches are automatically published.

This means if you push a version update to a branch and then create a PR, conda packages will be published to anaconda.org before the PR is merged.

This isn't what we want.

Yeah I did think of that. Do you think we can push to the GenericMappingTools org repo fork, or does it have to be a personal one?

I think yes. The only limitation is that the org fork can't "allow edits from maintainers".

Right, but it should be fine for you and Leo who are members of the GenericMappingTools org and this gmt-feedstock. I seem to recall we had an issue with updating the pygmt-feedstock from the GenericMappingTools org though? Edit: found it conda-forge/pygmt-feedstock#3.

Yes, I thought it's in this feedstock but couldn't find it.

It's here conda-forge/pygmt-feedstock#3. So the problem is that the bot can't update the @conda-forge-admin bot can't update org repo.

It sounds too complicated. Perhaps the easiest way is still having a github action in a personal repo.

Well, it would be exact same workflow but in a different account. I would still be careful with letting the Actions script push the repo here without going through a PR first. Even the official conda-forge bot doesn't do that.

Not sure if it helps, but I've made a commit at 8a63a23 which would prevent the commit/branch-push job from running on 'conda-forge/gmt-feedstock' (see https://github.community/t/stop-github-actions-running-on-a-fork/17965/2). In theory, this means any other fork of 'gmt-feedstock' (with Github Actions enabled) would get a commit with the updated recipe/meta.yaml file.

So at the very least, anyone can open a pull request from their personal fork (with the updated recipe/meta.yaml file) to this upstream 'conda-forge/gmt-feedstock'. But I'm not sure if it's possible to automate this 😅 This problem has been raised at repo-sync/pull-request#18 though, and there was a proof of concept PR at repo-sync/pull-request#19. We could perhaps test this out somehow.

But I'm not sure if it's possible to automate this sweat_smile This problem has been raised at repo-sync/pull-request#18 though, and there was a proof of concept PR at repo-sync/pull-request#19. We could perhaps test this out somehow.

Ok, I've gone full circle with repo-sync/pull-request#19. The auto PR should work if either:

1. Using secrets.GITHUB_TOKEN: works if PR happens on the same repo, i.e. conda-forge/gmt-feedstock/some-branch to conda-forge/gmt-feedstock/devel
2. Using secrets.PERSONAL_ACCESS_TOKEN: works across forks, i.e. username/gmt-feedstock/some-branch to conda-forge/gmt-feedstock/devel.
3. Experiment with adding /~https://github.com/conda-forge-linter as a member with 'write' permissions solely to the GenericMappingTools/gmt-feedstock repo, to sidestep the "Allow edits from maintainer" issue.

Option 1 is definitely out of question, since we can't create forks on conda-forge/gmt-feedstock itself. Option 2 means we rely on someone's personal repo, and it still requires that someone to setup a personal access token (so it's not as easy as it seems).

Option 3 (if it works) would be a bit more suistainable IMO, as potentially any member in the GenericMappingTools organization can edit the GenericMappingTools/gmt-feedstock repo, crucial if one of us no longer actively contributes to the project. So should we try?

I'm not so sure about the security implications of this (giving a bot write permissions to GenericMappingTools/gmt-feedstock), @leouieda probably has a better grasp on this. Yet another alternative (Option 4) is to make a Github App with the proper permissions, but this becomes an even bigger hassle to maintain (see /~https://github.com/peter-evans/create-pull-request/blob/master/docs/concepts-guidelines.md#authenticating-with-github-app-generated-tokens and /~https://github.com/tibdex/github-app-token).

Edit: The pangeo-dask-feedstock seems to have something very similar to what we want, implemented in conda-forge/pangeo-dask-feedstock#8. They use a pangeo-bot account though.

Ok, I've somewhat got it working (see example PR at #103) using a Personal Access Token (Option 2, make myself a bot). Currently the PR is from GenericMappingTools/gmt-feedstock (rather than weiji14/gmt-feedstock) to conda-forge/gmt-feedstock, so anyone on the GenericMappingTools organization (with permissions) should be able to modify the PR. Let me know if this is confusing.

Some notes:

The bot seems to have linted the PR coming from GenericMappingTools/gmt-feedstock ok, I didn't set any special permissions at all. Maybe because I was the one who made the PR?

Maybe use the default Github Actions bot user, rather than have it look like I'm making the automatic PRs.

Still need to tidy up the Github Actions workflow to be more readable

Change 6.1.1... to 6.2.0...

weiji14 · 2020-07-29T07:00:39Z

@conda-forge-admin, please rerender

Bump tibdex/github-app-token from 1.3.0 to 1.8.0, actions/checkout from 2.3.4 to 3.5.2, peter-evans/create-pull-request from 3.11.0 to 5.0.1.

The set-output command has been deprecated, see https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands

Some sed substitution to fix the pin_compatible lines for fftw, gdal, zlib and curl. Also bumped yq from 4.9.6 to 4.34.1.

Repo at /~https://github.com/actions/create-github-app-token

Bump actions/checkout from 3.5.2 to 4.1.6, peter-evans/create-pull-request from 5.0.1 to 6.0.5.

conda-forge-webservices · 2024-05-22T23:08:56Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipe:

This recipe is using a compiler, which now requires adding a build dependence on {{ stdlib("c") }} as well. For further details, please see META: {{ stdlib("c") }} migration conda-forge.github.io#2102.

In conda-forge.yml: $.compiler_stack = comp7.

'Compiler Stack' is deprecated.
Compiler stack environment variable. This is used to specify the compiler
stack to use for builds. Deprecated.

compiler_stack: comp7

Schema

{
  "anyOf": [
    {
      "type": "string"
    },
    {
      "type": "null"
    }
  ],
  "default": "comp7",
  "deprecated": true,
  "title": "Compiler Stack"
}

In conda-forge.yml: $.max_py_ver = 37.

'Max Py Ver' is deprecated.
Maximum Python version. This is used to specify the maximum Python version
to use for builds. Deprecated.

max_py_ver: 37

Schema

{
  "anyOf": [
    {
      "type": "string"
    },
    {
      "type": "null"
    }
  ],
  "default": "37",
  "deprecated": true,
  "title": "Max Py Ver"
}

In conda-forge.yml: $.max_r_ver = 35.

'Max R Ver' is deprecated.
Maximum R version. This is used to specify the maximum R version to use
for builds. Deprecated.

max_r_ver: 34

Schema

{
  "anyOf": [
    {
      "type": "string"
    },
    {
      "type": "null"
    }
  ],
  "default": "34",
  "deprecated": true,
  "title": "Max R Ver"
}

conda-forge-webservices · 2024-07-16T04:23:19Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

conda-forge-webservices · 2024-07-28T22:19:50Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

Need to run yq first, and then sed

GMT 6.2.0 dev build

4e3c6ce

weiji14 requested review from leouieda, ocefpaf and seisman as code owners July 28, 2020 10:04

Set license as LGPL-3.0-or-later

579c9c7

weiji14 marked this pull request as draft July 28, 2020 10:15

Experimental Github Action to bump dev version number automatically

8d0d5e0

weiji14 force-pushed the gmt-dev-builds branch from 60f7c05 to 8d0d5e0 Compare July 28, 2020 10:47

Let yq update meta.yaml inplace

54824d4

weiji14 force-pushed the gmt-dev-builds branch from 9f4bce0 to 54824d4 Compare July 28, 2020 10:56

Separate tasks and github_rev and build_version update tasks

87ad47a

weiji14 commented Jul 28, 2020

View reviewed changes

.github/workflows/bumpdevversion.yml Outdated Show resolved Hide resolved

weiji14 added 2 commits July 28, 2020 23:16

Remove curly bracket first line to make meta.yaml work with yq

daeb0f5

Use git_rev, not github_rev

388d2c3

weiji14 changed the title ~~WIP GMT 6.2.0 dev build~~ WIP Automated weekly GMT 6.2.0 dev builds Jul 28, 2020

Use git ls-remote to obtain GMT git repo HEAD sha

f69e287

weiji14 commented Jul 28, 2020

View reviewed changes

recipe/meta.yaml Outdated Show resolved Hide resolved

Don't bump build number

ee41a73

weiji14 force-pushed the gmt-dev-builds branch 2 times, most recently from f206dd1 to 9606d3e Compare July 28, 2020 23:56

Update package version to format like 6.1.0_abcdefg_20XX.MM.DD

79219c8

weiji14 force-pushed the gmt-dev-builds branch from 9606d3e to 79219c8 Compare July 29, 2020 00:21

Refactor to produce version string like 6.1.0.dev1+abcdefg

3d2ffd7

weiji14 force-pushed the gmt-dev-builds branch from 03e5244 to 3d2ffd7 Compare July 29, 2020 05:54

seisman reviewed Jul 29, 2020

View reviewed changes

Update bld.bat and build.sh

9a50557

weiji14 added 4 commits June 9, 2023 16:14

Bump GitHub Actions versions

77fb331

Bump tibdex/github-app-token from 1.3.0 to 1.8.0, actions/checkout from 2.3.4 to 3.5.2, peter-evans/create-pull-request from 3.11.0 to 5.0.1.

Write to GITHUB_OUTPUT instead of using set-output workflow command

57d0f8f

The set-output command has been deprecated, see https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands

Fix incorrectly formatted pin_compatible lines

ee572a4

Some sed substitution to fix the pin_compatible lines for fftw, gdal, zlib and curl. Also bumped yq from 4.9.6 to 4.34.1.

Merge remote-tracking branch 'upstream/devel' into gmt-dev-builds

3a5265c

weiji14 force-pushed the gmt-dev-builds branch from 032c2fa to 7a4ba5f Compare June 9, 2023 05:22

Merge remote-tracking branch 'upstream/devel' into gmt-dev-builds

9bf998a

weiji14 force-pushed the gmt-dev-builds branch from 7a4ba5f to d9caca3 Compare September 19, 2023 03:47

Merge remote-tracking branch 'upstream/devel' into gmt-dev-builds

86e885a

weiji14 force-pushed the gmt-dev-builds branch from d9caca3 to e783337 Compare October 4, 2023 22:41

Merge remote-tracking branch 'upstream/devel' into gmt-dev-builds

f4cdc1e

weiji14 force-pushed the gmt-dev-builds branch from e783337 to 5f2e495 Compare December 30, 2023 08:09

weiji14 added 4 commits May 23, 2024 10:50

Rename all instances of GMT 6.4.0 to 6.5.0

a2154dd

Merge remote-tracking branch 'upstream/devel' into gmt-dev-builds

52149d8

Switch from tibdex/github-app-token to actions/create-github-app-token

bf77955

Repo at /~https://github.com/actions/create-github-app-token

Bump GitHub Actions versions

7a2a567

Bump actions/checkout from 3.5.2 to 4.1.6, peter-evans/create-pull-request from 5.0.1 to 6.0.5.

weiji14 force-pushed the gmt-dev-builds branch from 5f2e495 to 668bade Compare May 22, 2024 23:08

Merge remote-tracking branch 'upstream/devel' into gmt-dev-builds

9102b43

weiji14 force-pushed the gmt-dev-builds branch from 668bade to e8bca61 Compare July 16, 2024 04:23

weiji14 force-pushed the gmt-dev-builds branch from e8bca61 to bf6ab7b Compare July 28, 2024 22:19

weiji14 added 2 commits July 29, 2024 10:27

Fix incorrectly formatted stdlib-c line

11d6914

Need to run yq first, and then sed

Merge remote-tracking branch 'upstream/devel' into gmt-dev-builds

02a552a

weiji14 force-pushed the gmt-dev-builds branch from bf6ab7b to 8bb21a7 Compare July 28, 2024 22:29

Merge remote-tracking branch 'upstream/devel' into gmt-dev-builds

a79088a

weiji14 force-pushed the gmt-dev-builds branch from 8bb21a7 to 5dfb45d Compare November 6, 2024 09:13

weiji14 added 2 commits November 24, 2024 13:04

Merge remote-tracking branch 'upstream/devel' into gmt-dev-builds

2684eec

[cf admin skip] ***NO_CI*** Temp trigger auto PR

b842612

weiji14 force-pushed the gmt-dev-builds branch from 5dfb45d to b842612 Compare November 24, 2024 00:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Automated monthly GMT 6.5.0 dev builds #100

Automated monthly GMT 6.5.0 dev builds #100

weiji14 commented Jul 28, 2020 •

edited

Loading

conda-forge-linter commented Jul 28, 2020

conda-forge-linter commented Jul 28, 2020

seisman commented Jul 29, 2020

seisman Jul 29, 2020

weiji14 Jul 29, 2020

seisman Jul 29, 2020

weiji14 Jul 29, 2020 •

edited

Loading

seisman Jul 29, 2020

seisman Jul 29, 2020

leouieda Jul 30, 2020

weiji14 Jul 30, 2020 •

edited

Loading

weiji14 Aug 3, 2020 •

edited

Loading

weiji14 Sep 17, 2020 •

edited

Loading

weiji14 commented Jul 29, 2020

conda-forge-webservices bot commented May 22, 2024

conda-forge-webservices bot commented Jul 16, 2024

conda-forge-webservices bot commented Jul 28, 2024 •

edited by conda-forge-admin

Loading

Automated monthly GMT 6.5.0 dev builds #100

Are you sure you want to change the base?

Automated monthly GMT 6.5.0 dev builds #100

Conversation

weiji14 commented Jul 28, 2020 • edited Loading

conda-forge-linter commented Jul 28, 2020

conda-forge-linter commented Jul 28, 2020

seisman commented Jul 29, 2020

seisman Jul 29, 2020

Choose a reason for hiding this comment

weiji14 Jul 29, 2020

Choose a reason for hiding this comment

seisman Jul 29, 2020

Choose a reason for hiding this comment

weiji14 Jul 29, 2020 • edited Loading

Choose a reason for hiding this comment

seisman Jul 29, 2020

Choose a reason for hiding this comment

seisman Jul 29, 2020

Choose a reason for hiding this comment

leouieda Jul 30, 2020

Choose a reason for hiding this comment

weiji14 Jul 30, 2020 • edited Loading

Choose a reason for hiding this comment

weiji14 Aug 3, 2020 • edited Loading

Choose a reason for hiding this comment

weiji14 Sep 17, 2020 • edited Loading

Choose a reason for hiding this comment

weiji14 commented Jul 29, 2020

conda-forge-webservices bot commented May 22, 2024

conda-forge-webservices bot commented Jul 16, 2024

conda-forge-webservices bot commented Jul 28, 2024 • edited by conda-forge-admin Loading

weiji14 commented Jul 28, 2020 •

edited

Loading

weiji14 Jul 29, 2020 •

edited

Loading

weiji14 Jul 30, 2020 •

edited

Loading

weiji14 Aug 3, 2020 •

edited

Loading

weiji14 Sep 17, 2020 •

edited

Loading

conda-forge-webservices bot commented Jul 28, 2024 •

edited by conda-forge-admin

Loading