replaced print statements with log #133

shimwell · 2024-05-08T08:59:57Z

Currently GEOUNED prints quite a lot to the terminal

However we could move all of that terminal output to a logger

The logger then writes the print message along with date, time, filename, line number and other options to a log file

This includes more information than a print statement, it is persistent and searchable

The verbose options is now redundant so this has also been removed

Here is an example log file that gets produced automatically

(geouned_py311) j@mini:~/GEOUNED$ cat geouned.log
2024-05-08 09:58:03,181 :: INFO :: start :: 358 :: start
2024-05-08 09:58:03,181 :: INFO :: start :: 360 :: GEOUNED version 1.0.1 19/03/2024
FreeCAD version 0.21.2
2024-05-08 09:58:03,181 :: INFO :: start :: 393 :: read step file : /home/j/GEOUNED/testing/inputSTEP/biseau.stp
2024-05-08 09:58:03,198 :: INFO :: start :: 398 :: End of loading phase
2024-05-08 09:58:03,199 :: INFO :: start :: 400 :: 0:00:00.017430
2024-05-08 09:58:03,199 :: INFO :: decompose_solids :: 602 :: Decomposing solid: 1/1
2024-05-08 09:58:03,344 :: INFO :: start :: 447 :: End of decomposition phase
2024-05-08 09:58:03,344 :: INFO :: start :: 454 :: Building cell: 1
2024-05-08 09:58:03,347 :: INFO :: start :: 476 :: 0:00:00.148343
2024-05-08 09:58:03,347 :: INFO :: start :: 494 :: Build Void
2024-05-08 09:58:03,347 :: INFO :: start :: 495 :: []
2024-05-08 09:58:03,347 :: INFO :: void_generation :: 44 :: Build Void highest enclosure
2024-05-08 09:58:03,348 :: INFO :: get_void_def :: 96 :: Loop, Box to Split :{iloop}, {nvoid}
2024-05-08 09:58:03,348 :: INFO :: get_void_def :: 121 :: build complementary 1 0
2024-05-08 09:58:03,349 :: INFO :: start :: 530 :: build Time: 2024-05-08 09:58:03.349553 - 2024-05-08 09:58:03.347509
2024-05-08 09:58:03,349 :: INFO :: start :: 532 :: 0:00:00.167967
2024-05-08 09:58:03,349 :: INFO :: write_input :: 75 :: write MCNP file /home/j/GEOUNED/tests_outputs/testing/inputSTEP/biseau/biseau.mcnp
2024-05-08 09:58:03,350 :: INFO :: write_xml :: 28 :: write OpenMC xml file /home/j/GEOUNED/tests_outputs/testing/inputSTEP/biseau/biseau.xml
2024-05-08 09:58:03,350 :: INFO :: write_py :: 94 :: write OpenMC python script /home/j/GEOUNED/tests_outputs/testing/inputSTEP/biseau/biseau.py
2024-05-08 09:58:03,350 :: INFO :: write_input :: 66 :: write Serpent file /home/j/GEOUNED/tests_outputs/testing/inputSTEP/biseau/biseau.serp
2024-05-08 09:58:03,350 :: INFO :: write_phits :: 61 :: write PHITS file /home/j/GEOUNED/tests_outputs/testing/inputSTEP/biseau/biseau.inp
2024-05-08 09:58:03,351 :: INFO :: __write_phits_cell_block__ :: 167 :: Unified the inner void cell(s) definition
2024-05-08 09:58:03,351 :: INFO :: start :: 587 :: End of MCNP, OpenMC, Serpent and PHITS translation phase
2024-05-08 09:58:03,351 :: INFO :: start :: 589 :: Process finished
2024-05-08 09:58:03,351 :: INFO :: start :: 590 :: 0:00:00.169686
2024-05-08 09:58:03,351 :: INFO :: start :: 592 :: Translation time of solid cells 2024-05-08 09:58:03.347509 - 2024-05-08 09:58:03.199163
2024-05-08 09:58:03,351 :: INFO :: start :: 593 :: Translation time of void cells 2024-05-08 09:58:03.349553 - 2024-05-08 09:58:03.347509

The output can be customised to include more details with each print statement if we like.

shimwell · 2024-05-08T10:01:10Z

@alberto743 it would be great to get your views on this PR and any further comments you have.

alberto743 · 2024-05-08T10:32:08Z

Sure. You can assign the review to me. I can look at it tomorrow or on Friday.

shimwell · 2024-05-08T10:42:43Z

Thanks for offering to review, looking forward to your feedback. As I have no privileges on the repo I can't assign anything to anyone. I guess you have the power to self assign if that helps

psauvan · 2024-05-08T10:55:02Z

Its is fine to move all the terminal output into a log file, but I think that we should keep some of them so the user can check the current status of the process, Is there any library implementing progress bar in python?

shimwell · 2024-05-08T10:57:10Z

tqdm is perhaps the best python progress bar.
https://tqdm.github.io/

It can be attached to for loops, while loops and generators

@psauvan could you let me know which loop(s) would you like reporting with a progress bar ?

I shall get it done on this open PR

shimwell · 2024-05-08T12:18:53Z

I have added progress bars to a few key loops in the code. I track progression of decomposition, file loading, void generation.

Is this close to what you were thinking @psauvan

Happy to add more progress bars

psauvan · 2024-05-08T12:21:47Z

yes, this would be nice

shimwell · 2024-05-08T12:53:19Z

I've added the progress bar and also sorted the imports using isort

psauvan · 2024-05-08T13:50:22Z

src/geouned/GEOUNED/__init__.py

at line 451 change the loop by

for j, m in enumerate(tqdm(MetaList, desc="translating solid cells")):

psauvan

see comment 60c5c9e#r1594069544

…ED into replace_print_with_logger

alberto743 · 2024-05-09T13:07:11Z

Hi @shimwell . Thank you very much for the development.
The branch is fine for merging for me, but I do have the following comments

the logging should be used also for other log files, e.g. fuzzySurfaces. This may be described inside another another issue.
While I like the tqdm module for the progress bar, I personally do not agree with @psauvan on the fact that the core module should dump anything on the standard output/error. Instead, any information written to stdout/stderr may arrive from some calling scripts but should not be part of the core progressing, which should interact with other modules exclusively via the stack of function calls or via appropriate logging calls. Again, this may be tracked via another issue.

shimwell · 2024-05-09T14:04:43Z

Hi @shimwell . Thank you very much for the development. The branch is fine for merging for me, but I do have the following comments

the logging should be used also for other log files, e.g. fuzzySurfaces. This may be described inside another another issue.

While I like the tqdm module for the progress bar, I personally do not agree with @psauvan on the fact that the core module should dump anything on the standard output/error. Instead, any information written to stdout/stderr may arrive from some calling scripts but should not be part of the core progressing, which should interact with other modules exclusively via the stack of function calls or via appropriate logging calls. Again, this may be tracked via another issue.

Both good ideas in my view, let track these on a new issue if you are happy to raise one @alberto743

alberto743 · 2024-05-10T16:51:46Z

The new issues to track the suggestions have been opened.

@psauvan

* Add a github workflow that prevents PRs to main that is not comming from dev (#92) * Fix to position of additional cylindrical surface used to limit torus extent (#96) * camel case for all classes (#95) * camel case for all classes * corrected file imports for georeverse * Apply suggestions from code review variable back to snake case * variable back to snake case * back to snake case * back to previous var name * Snake case variables (PR 1 of several) (#98) * snake case for CellDefinitions file, tests pass * vars in snake_case 2 more files * working still * using is to compare with bool (#99) * Add comments to Geometry.py and booleanFunction.py files (#101) * Add comments to Geometry_GU.py * Add comments to booleanFunction.py * Add comments to Geometry_GU.py * Specific imports instead of wild card (#100) * improved import statements flake8 hints * removed extra line * solve issue #75 in getTransMatrix function (#103) * Changes in users initialization (#102) * Add a github workflow that prevents PRs to main that is not comming from dev (#92) (#93) * Add scripting interface Change to allow setting users parameters by either config file or geouned set method * minor fix * Changes in geounedClassCall.py --------- Co-authored-by: AlvaroCubi <55387701+AlvaroCubi@users.noreply.github.com> * Adding formatter to CI to keep code in PEP8 (#104) * added format checker * run black . locally * f strings instead of .format (#106) * replaced all .format with f strings * corrected f string conversion * Snake case variables (PR 2 of several) (#105) * snake case im more varibles * formatting --------- Co-authored-by: Patrick Sauvan <psauvan@ind.uned.es> * Quicker tests for ci (#109) * removing slowest two geometries * removing slow files from CI * format * added draft template (#97) * Removing unused code (#107) @psauvan comment has been taken into account. Unreachable code that was there for future consideration has been changed to comments. * if statement includes False so never runs code * if statement includes True so always run and never reaches else * line after return never reached * unused arg * unused arg * unused arg calling function * commented unused code as requested * formatting * format * Testing writing inputs for more MC codes (#108) * also writting openmc python, phits and serpent files * formatting * decreased lcoal run time and added all outFormats * black using same version as in pyproject * py3.11 in formatting ci * matched exactly black versions * simpler black pinning * format * Snake case functions (#115) * converted more functions * more snake case functions * more snake case function names * more snake case functions * more snake case functions * more snake case functions * formatting * formatting * formatting * formatting * converting function names to snake class * duplicate import openmc (#116) * Sphinx based docs with CI action to build and hosted on gh-pages (#119) * started rtd * testing docs action * update pip * corrected python version * added s to docs * publish every time * added static folder * skipping ci for non code files * changed to rerad the docs theme * started install section * added more sections * added api class * docs ci using conda * added -y for freecad * trying py 3.12 * mimic working conda ci * using rst so no markdown convertion needed * added github edit link * removed path prints * added readme link, badge * format * Add issue templates (#112) * Add a github workflow that prevents PRs to main that is not comming from dev (#92) (#93) * Create feature request template * improve documentation template * update feature request template --------- Co-authored-by: AlvaroCubi <55387701+AlvaroCubi@users.noreply.github.com> * Trigger docs build on push only (#126) * pushing triggers doics build * added if push * added missing = sign (#130) * added missing package (#135) * replaced print statements with log (#133) * replaced print statements with log * added logger * formatting * removed encoding arg * removed verbose as all gets logged * removed unused logger * removed , from logging cmd * sorted imports * added another tqdm loop * isort * format * Fixing logger commands (#137) * fixed multi comma log statements * capital letters for progress bar sections * formatted * removal of Options as global variable (#138) * instances of Options class passed by arg * updated usage to show how to use Options * format pep8 * added missing options arg * added missing args options * passing in options as arg * bug fix for compsolid * format * removal of Tolerances as global variable (#139) * passing tolerance as arg not global var * added tolerance example to usage section * one set of argument names * moved atr and types getting out of loop * format * passing missing options and tolerances * format * found bug hidden in try: except statement * using tolerance for more local varibles * formatting * Numerical format to normal class (#141) * replaced global with class passed by arg * format * Improving docs install section (#145) * added dev install instructions * added method section * testing the setting of all class arguments (#147) * Adding getters setters type checking to Options, Tolerances, NumericFormat class (#148) * added type ching setters * added low limit checking for reals * format * added type checkers for Tolerances * format * Add settings class (#149) * settings being passed through the code * all the attributes show in usage example * Adding export csg method (#150) * added export csg method * format * leaving comments in tests * Adding support for json config file (#152) * testing from json * returned file to pre PR status * returned file to init * returned file to init * temp fixing for serpent universe, issue raised * making use of stp file already on repo * Adding command line tool including documentation and testing (#155) * split usage into two sections * added minimal config from cli example to ci * improved cli docs * format * removed duplication between readme and docs (#156) * Moving code from __init__ file to core.py (#157) * moved core logic from init * format * Fixing issue 154 (#158) * added test that currently fails * fixed bug with freecad attribute usage * added missing arg * format * added missing args bug (#159) * added missing args * format * added description of solid being decomposed (#161) * Context managers for writing try 2 (#160) * Add a github workflow that prevents PRs to main that is not comming from dev (#92) (#93) * using context managers to write files * review comments 4/5 --------- Co-authored-by: AlvaroCubi <55387701+AlvaroCubi@users.noreply.github.com> * Creating folder if needed (#162) * added tests to check writing to folder * back to org * back to org * back to org * creating output folder prior to writing * isort on the imports * format * setting line length to 128 (#163) * Lower case folder names and files (#165) * renamed subfolders to lowercase * renamed files to lower case * openmc_py and openmc_xml lower case (#166) * openmc_py and openmc_xml lower case * removed outdated code comment * Added type checking on export csg (#167) * added input checking on export_csg * compacted same types * Update src/geouned/GEOUNED/core.py --------- Co-authored-by: Patrick Sauvan <psauvan@ind.uned.es> * Adding windows install instructions (#169) * added windows install instructions * improved install instructions --------- Co-authored-by: teade <37874718+teade@users.noreply.github.com> Co-authored-by: Jonathan Shimwell <drshimwell@gmail.com> Co-authored-by: Jonathan Shimwell <mail@jshimwell.com> Co-authored-by: Patrick Sauvan <psauvan@ind.uned.es> Co-authored-by: alberto743 <4104972+alberto743@users.noreply.github.com> Co-authored-by: Alex Valentine <40658938+alexvalentine94@users.noreply.github.com>

shimwell added 6 commits May 8, 2024 09:53

replaced print statements with log

3a8e20d

added logger

52e8a78

formatting

45e6d69

removed encoding arg

8d58f6c

removed verbose as all gets logged

93e190d

removed unused logger

8c52f9b

removed , from logging cmd

e12fb5b

shimwell added 2 commits May 8, 2024 13:52

sorted imports

60c5c9e

Merge branch 'dev' into replace_print_with_logger

de1dd6c

psauvan reviewed May 8, 2024

View reviewed changes

psauvan requested changes May 8, 2024

View reviewed changes

shimwell added 4 commits May 8, 2024 15:01

added another tqdm loop

fceac1b

Merge branch 'replace_print_with_logger' of github.com:shimwell/GEOUN…

20067b9

…ED into replace_print_with_logger

isort

da3a355

format

5549733

psauvan merged commit 6ffff58 into GEOUNED-org:dev May 8, 2024
9 checks passed

shimwell mentioned this pull request May 13, 2024

Avoid stdout/stderr implicit writing #143

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

replaced print statements with log #133

replaced print statements with log #133

shimwell commented May 8, 2024 •

edited

Loading

shimwell commented May 8, 2024

alberto743 commented May 8, 2024

shimwell commented May 8, 2024

psauvan commented May 8, 2024

shimwell commented May 8, 2024 •

edited

Loading

shimwell commented May 8, 2024

psauvan commented May 8, 2024

shimwell commented May 8, 2024

psauvan May 8, 2024 •

edited

Loading

psauvan left a comment

alberto743 commented May 9, 2024

shimwell commented May 9, 2024

alberto743 commented May 10, 2024

replaced print statements with log #133

replaced print statements with log #133

Conversation

shimwell commented May 8, 2024 • edited Loading

shimwell commented May 8, 2024

alberto743 commented May 8, 2024

shimwell commented May 8, 2024

psauvan commented May 8, 2024

shimwell commented May 8, 2024 • edited Loading

shimwell commented May 8, 2024

psauvan commented May 8, 2024

shimwell commented May 8, 2024

psauvan May 8, 2024 • edited Loading

Choose a reason for hiding this comment

psauvan left a comment

Choose a reason for hiding this comment

alberto743 commented May 9, 2024

shimwell commented May 9, 2024

alberto743 commented May 10, 2024

shimwell commented May 8, 2024 •

edited

Loading

shimwell commented May 8, 2024 •

edited

Loading

psauvan May 8, 2024 •

edited

Loading