Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reduce spark logging in tests #305

Closed
StrongestNumber9 opened this issue Sep 5, 2024 · 1 comment · Fixed by #336
Closed

Reduce spark logging in tests #305

StrongestNumber9 opened this issue Sep 5, 2024 · 1 comment · Fixed by #336
Assignees
@eemhu
Labels
enhancement New feature or request

Comments

@StrongestNumber9
Copy link
Contributor

StrongestNumber9 commented Sep 5, 2024 •
edited
Loading

Description

Reduce spark logging in tests

Use case or motivation behind the feature request

It prints a lot of irrelevant logs which slows down usage and drowns possibly interesting messages.
I think the default logging level can be set to WARN+ for spark components long as it can be changed on demand

Related issues

Additional context

Any build log from actions
@StrongestNumber9 StrongestNumber9 added the enhancement New feature or request label Sep 5, 2024
@StrongestNumber9
Copy link
Contributor Author

Seems like

pth_10/src/test/java/com/teragrep/pth10/StreamingTestUtil.java

Line 168 in fd6b944

spark.sparkContext().setLogLevel("ERROR");
isn't really doing anything.

This was referenced Sep 17, 2024
Open
Open
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@eemhu eemhu

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Reduce logging in tests by using slf4j-api instead of slf4j-simple, i… StrongestNumber9/pth_10
2 participants
@StrongestNumber9 @eemhu