Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fastmath always loads MKL binaries #15

Closed
mikub opened this issue Apr 6, 2022 · 10 comments
Closed

Fastmath always loads MKL binaries #15

mikub opened this issue Apr 6, 2022 · 10 comments
Labels
help wanted Extra attention is needed

Comments

@mikub
Copy link

mikub commented Apr 6, 2022

Hi there!
We love fastmath! 🙏
Just a small nitpick (it actually broke our deployment pipeline 😄 ):
Fastmath includes com.github.haifengl/smile-mkl dependency which may lead to substantial resources (~800MB) being downloaded and included in the build.

image

This should be either configurable or at least mentioned in the Readme so as users may plan for this and optionally exclude this dependency from their deps tree.

Cheers
Miro

@tomekw
Copy link

tomekw commented Apr 6, 2022

Looks like smile-* deps are pulled in but never used?

@genmeblog
Copy link
Contributor

Thanks. I need to check if I can get rid them of. Generally Smile is used in several places (optimization, clustering, stats) and as far as I remember some algorithms rely on mkl.

@tomekw
Copy link

tomekw commented Apr 6, 2022

Thank you @genmeblog! The thing is it increases our artifact size from 151 MB to 1 GB :)

@genmeblog
Copy link
Contributor

Yes... I'm aware of this. And more people complained about the size.

When I exclude smile-mkl, some things are not working - need to investigate what and when...

@genmeblog genmeblog added the help wanted Extra attention is needed label Apr 6, 2022
@genmeblog
Copy link
Contributor

Ok, when I turn off MKL, OpenBlas is loaded instead, however... my test suite (which is lein codox, which runs examples defined in metadoc) fails with Subprocess failed (exit code: 139). So generally some things are not working without MKL...

@tomekw
Copy link

tomekw commented Apr 6, 2022

@genmeblog we use your awesome clojure2d library to do some basic drawing (put some text onto an image background). Could we use some "lite" version instead?

@genmeblog
Copy link
Contributor

The option is to release version without MKL with a annotation about the potential issues. I need to verify if clojure2d is able to stable operate without MKL. reduce-colors and gradient related functions rely on Smile, hence may try to load BLAS/LAPACK in certain conditions.

@tomekw
Copy link

tomekw commented Apr 6, 2022

@genmeblog that's the entirety of the c2d API we use:

image

@genmeblog
Copy link
Contributor

I can't propose anything more than exclusion of MKL. I need to carefuly test and assure that things around fastmath will work.

@genmeblog
Copy link
Contributor

@mikub I decided to update README and describe how to remove MKL and what's the impact. As far as I tested, most things should work on OpenBlas, however I have some issues during build. That means that by default MKL will be included but can be easily removed.

I'll exclude MKL from Clojure2d - it has been tested already (thanks @tomekw) and works on OpenBlas without the problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

3 participants