问题描述:

What would be a good way to create logs (with python logging module) inside a constant running loop, without producing a large amount of useless log-files?

An example would be a loop that constant list a folder, and does some action when it sees a file of a specific type.

I want to log that no files were found, or files were found but of a wrong type, without logging that same line constantly for each folder check, as it might run many times a second.

网友答案:

Create a Handler that subclasses whatever other functionality you need. Store either the last, or all the previously logged messages that you don't want to emit again:

def make_filetype_aware_handler(handler_class):
    class DontRepeatFiletypeHandler(handler_class):

        def __init__(self, *args, **kwds):
            super().__init__(*args, **kwds)
            self.previous_types = set()

        def emit(self, record):
            if not record.file_type in self.previous_types:
                self.previous_types.add(record.file_type)
                super().emit(record)
    return DontRepeatFiletypeHandler

FiletypeStreamHandler = make_filetype_aware_handler(logging.StreamHandler)

logger = logging.getLogger()
logger.addHandler(FiletypeStreamHandler(sys.stderr))
logger.debug('Found file of type %(file_type)', file_type='x-type/zomg')
网友答案:

Log the less important events with a lower precedence, like DEBUG. See setLevel and SysLogHandler.

At development time set the level to DEBUG, and as your application matures, set it to more reasonable values like INFO or ERROR.

Your app should do something about the errors, like remove files of the wrong type and/or create lacking files; or move the wrongfully configured directories from the job polling to a quarantine location, so your log will not be flooded.

网友答案:
import logging
logger = logging.getLogger(test)
# logging to a file
hdlr = logging.FileHandler(test.log)
formatter = logging.Formatter('%(asctime)s %(filename)s %(lineno)s %(levelname)s %    (message)s')
hdlr.setFormatter(formatter)
logger.addHandler(hdlr) 
logger.setLevel(logging.DEBUG) 

Then in the loop, you have to check for file type and if file is present or not. Then add :

logger.debug('File Type : %s ' % file_type)

also

if file_present:
    logger.debug('File Present : %s ' % present)
网友答案:

My understanding is that you are trying to limit logging the same message over and over again.

If this is your issue, I would create a a set of file_types you have already logged. However you need to be careful, if this is going to run forever, you will eventually crash..

from sets import Set
logged = Set()
while yourCondition:
   file_type = get_next_file_type()
   needToLog = #determine if you need to log this thing

   if needToLog and (not file_type in logged):
      logger.info("BAH! " + file_type)
      logged.add(file_type)
相关阅读:
Top