Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build linux CUDA releases suitable for Colab & other platforms on 12.2 #11226

Draft
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

ochafik
Copy link
Collaborator

@ochafik ochafik commented Jan 14, 2025

TODO

  • Fix build on ci
  • Trigger a branch release if possible (to test entire mechanics)
  • Share example Colab (see draft: needs update to use latest release / or branch release)
  • Incubate install.sh (Unix incl. WSL) & install.ps1 (Windows) scripts that detect os, arch, cpu & gpu caps and install the right release (maybe through brew)

@github-actions github-actions bot added devops improvements to build systems and github actions ggml changes relating to the ggml tensor library for machine learning labels Jan 14, 2025
@slaren
Copy link
Collaborator

slaren commented Jan 14, 2025

What's the reason for making a different release for each arch?

@ochafik
Copy link
Collaborator Author

ochafik commented Jan 14, 2025

What's the reason for making a different release for each arch?

@slaren Building for a single arch seems a lot faster, and having separate artefacts instead of (cuda-)fat binaries means smaller downloads / quicker setup on Colab. I couldn't finish a full build w/ all the architectures locally yet tho, maybe I'll try this to see how much overhead per arch we're talking about.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
devops improvements to build systems and github actions ggml changes relating to the ggml tensor library for machine learning
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants