feat: A different approach to warning users of fork() issues with Polars

[itamarst]

Just print a warning when fork() is used at the same time as Polars, which is what jax does. Users aren't as likely to see warnings, but it's something, and it's less intrusive hopefully than the previous attempt that got reverted.

[codecov]

Codecov Report

Attention: Patch coverage is 66.66667% with 3 lines in your changes missing coverage. Please review.

Project coverage is 79.97%. Comparing base (20ead46) to head (a65272f).
Report is 236 commits behind head on main.

Files with missing lines	Patch %	Lines
py-polars/polars/__init__.py	66.66%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #19197      +/-   ##
==========================================
- Coverage   80.03%   79.97%   -0.06%     
==========================================
  Files        1528     1529       +1     
  Lines      209657   209822     +165     
  Branches     2416     2417       +1     
==========================================
+ Hits       167791   167809      +18     
- Misses      41317    41463     +146     
- Partials      549      550       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[hassonofer]

The multiprocessing module warning is problematic when the module isn't directly used by the user.
In frameworks like PyTorch, it's used internally through DataLoaders, torchrun, and other components.
The different ways programs can run makes it complex to switch the default context.

Some related discussions are happening in the PyTorch community (like pytorch/pytorch#138957).
As a potential quick fix, would it make sense to add an environment variable to disable this warning?

[itamarst]

Some related discussions are happening in the PyTorch community (like pytorch/pytorch#138957).
As a potential quick fix, would it make sense to add an environment variable to disable this warning?

fork() is not safe in the face of threads. It will deadlock mysteriously. People will waste days on this, I have written an article about this and keep hearing from people telling me this saved them from problems they were fundamentally unable to solve.

So using the fork() context is fundamentally broken, and Python 3.14 is switching to a different default for this reason. Having an environment variable saying "stop telling me I'm shooting myself in the foot" isn't stopping you from shooting yourself in the foot.

Some other options:

1. Instead of having Polars help people break their runtime, those libraries and frameworks should be switching to the non-broken default.
2. In addition, you can set the global multiprocessing default context to "spawn" by doing multiprocessing.set_start_method("spawn") early in program startup. This could be documented in Polars page on this.

[hassonofer]

I'm not arguing with fork being problematic, and I value your work toward helping people.

I was trying to convey, that setting multiprocessing.set_start_method("spawn") or forkserver for that matter, is not simple as one might think at some scenarios.
Also, the ability to influence large frameworks is limited.

The way I see it, my suggestion does not affect regular users who will get the warning and hopefully will go read on the subject.
But for people who are already aware of it, it can clean up the logs - which can get quite big when using stuff like submitit or torchrun (the warning repeats for each worker).