Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Constraints that include logical "or" cause Snakemake to hang #58

Closed
w8jcik opened this issue Apr 2, 2024 · 4 comments
Closed

Constraints that include logical "or" cause Snakemake to hang #58

w8jcik opened this issue Apr 2, 2024 · 4 comments
Assignees
Labels
bug/fix Something isn't working

Comments

@w8jcik
Copy link

w8jcik commented Apr 2, 2024

Constraints are used to select GPU types, architectures and CPU types.

Using logical or leads to Snakemake neither failing or submitting a job.

Example profile profiles/default/config.yaml

executor: slurm

default-resources:
  ...
  constraint: "turing|ampere"

Observations

  • Both turing and ampere work when used separately.
  • Specifying non existent constraint leads to failure which is fine.
WorkflowError:
SLURM job submission failed. The error message was sbatch: error: Batch job submission failed: Invalid feature specification
  • Using or operator leads to neither failure or submission. It just keeps hanging.

My first guess is that this parameter is passed directly to shell and then | results in opening a pipe.

@cmeesters cmeesters self-assigned this Apr 2, 2024
@cmeesters cmeesters added the bug/fix Something isn't working label Apr 2, 2024
@cmeesters
Copy link
Member

I'm afraid, your assumption is correct. I will attempt a fix today. Meanwhile, you could try: constraint: "'turing|ampere'".

@w8jcik
Copy link
Author

w8jcik commented Apr 2, 2024

Meanwhile, you could try: constraint: "'turing|ampere'".

I tried with extra quotes and without, the outcome is the same.

@cmeesters
Copy link
Member

We do not have a test case, yet. That might take a little while, yet you can try the code in PR #64 . Just clone, switch to branch fix/#58_logical_constraints and (assuming you are working with a conda/mamba environment, use poetry install from your cloned directory.

I hope, we have the fix tested in a few days.

@w8jcik
Copy link
Author

w8jcik commented Apr 16, 2024

I successfully queued some jobs to the cluster using | operator, so I believe that the change works.

Thank you for help.

cmeesters added a commit that referenced this issue Aug 6, 2024
There is no test - yet. I am not sure how to test constraints in the
dummy SLURM setup of the CI.
cmeesters pushed a commit that referenced this issue Aug 6, 2024
🤖 I have created a release *beep* *boop*
---


##
[0.9.0](v0.8.0...v0.9.0)
(2024-08-06)


### Features

* multicluster
([#56](#56))
([c0f8fee](c0f8fee))


### Bug Fixes

* fixed string for constraints - see issue
[#58](#58)
([#64](#64))
([89e10ff](89e10ff))

---
This PR was generated with [Release
Please](/~https://github.com/googleapis/release-please). See
[documentation](/~https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@w8jcik w8jcik closed this as completed Aug 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug/fix Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants