Implement python logging for output/debugging instead of print().

[alexlyttle]

A possible enhancement to this package would be to use python logging for output and logging errors etc. instead of using print. Logging allows you to:

• See when and where in the code a logging call has been made
• Assign categories to messages which the user can configure what to output/not (great for debugging)
• Can configure output from the module to be sent to specific files - great when running on a remote machine
• Can configure messages to be sent straight to stdout and stderr (print doesn't do this immediately unless you run the script with the python -u option.

I am new to logging myself, but it appears to be a much better way of logging messages within a large python package than print() (which doesn't give much info about where it came from etc.)

Maybe someone with more knowledge on this could implement it?

[grd349]

Hi @alexlyttle,

This is an excellent idea. let's have a chat about how we could manage this. In the mean time, if anyone wants this for themselves, just shout.

[nielsenmb]

Ideas:

0. Use the built-in logging module
1. silent logging at every function call? Set logging level to INFO
2. time stamp at each log step
3. PBjam version logged at beginning of run
4. filename star_date_time.log
5. verbose settings to set amount of messages to print statements (silent, brief, letsjam). silent for cluster runs, brief by default, letsjam for debugging a single star.

[nielsenmb]

This is a problem description for the PBjam Hackday:

Problem: Currently PBjam just dumps all messages whether progress or error related to the terminal. For debugging purposes this is not the most optimal way of doing things. Furthermore, the error messages that PBjam produces may not always be very meaningful.

Possible Solution: A good first step is to make a small class or module that just handles writing statements to a file. This can possibly involve the Python built-in modules warnings and logging.

Fully integrating error handling and messaging at all points in PBjam is beyond the scope of a hack day, but maybe an intermediate step is to research and possibly set up a 'best practice' framework for ensuring that PBjam fails gracefully. This can then later be integrated more completely.

[grd349]

Of course we lost the hack day. This is still something that should be done. @nielsenmb @alexlyttle I invite your thoughts on how to proceed :)

[nielsenmb]

Just trying to list a few options that I can think of.

1. One log for the whole run, stored in cwd. This might get cluttered.
2. One log for the whole run, copied to all the star directories in that session. No log file in cwd. This seems neater to me.
3. A log for each star stored in the star directories. With log file for just the session part in cwd.

My idea is to print all logging levels to the file(s), function enter/exits, progress etc. So do we still want verbosity arguments in various functions? Don't think it'll be that difficult, but might be overkill?

[grd349]

We still want warnings sent to terminal no? We should have logs too but I think 1 log for the whole run should be a starting point. We can always iterate later.

[nielsenmb]

Definitely, I don't know much about the logging module, but it must be possible to pipe one level of logs to the terminal and all the levels to the log.

…

On Tue, 21 Jul 2020 at 16:10, Guy Davies ***@***.***> wrote: We still want warnings sent to terminal no? We should have logs too but I think 1 log for the whole run should be a starting point. We can always iterate later. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#175 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEJWO33MHHZBRRLJS6JPWSDR4WVVPANCNFSM4JGYII5Q> .