Log rotate by time and size

[mahadi]

Hi, is it possible to rotate a logfile by duration and file size (and finally compress it)? Currently i only see that o can choose duration or file size but not both at the same time.

[Delgan]

Hi. :)

Loguru does not provide such possibility directly. To do this, you need to implement a custom rotation function. For example:

def rotation(message, file):
    file.seek(0, 2)
    if file.tell() + len(message) > 10000:
        return True
    if message.record["time"].time() > datetime.time(12, 0, 0):
        return True
    return False

logger.add("file.log", rotation=rotation)

[mahadi]

Hi, you are right, this is one way to do this :)

But i guess

if message.record["time"].time() > datetime.time(12, 0, 0):
    return True

would rotate for every message after 12am?

How would you write that for a timedelta? (Since it needs to know the start time of the current file?)

[Delgan]

@mahadi Oups, yeah, sorry. >.<

Well, you probably need to use a class then. Something like this:

from datetime import datetime, timedelta, time

class Rotator:

    def __init__(self, size, time):
        self._size = size
        self._time = datetime.now().replace(hour=time.hour, minute=time.minute, second=time.second)

    def should_rotate(self, message, file):
        file.seek(0, 2)
        if file.tell() + len(message) > self._size:
            return True
        if message.record["time"].timestamp() > self._time.timestamp():
            self._time += timedelta(days=1)
            return True
        return False

rotator = Rotator(10000, time(12, 0, 0))
logger.add("file.log", rotation=rotator.should_rotate)

[mahadi]

Thanks for the hint! Will use it :)
Thanks also for the great library!

Btw: rotation=rotate.should_rotate should be rotation=rotat**or**.should_rotate.

[Delgan]

Great, glad it worked for you! 😄

I'm closing this issue then. 👍

[mahadi]

Just for completeness: This is a small fix to prevent immediate log rotation if the current time is already past the given time:

class Rotator:

    def __init__(self, size, at):
        self._size = size
        now = datetime.now()
        today_at_time = now.replace(hour=at.hour, minute=at.minute, second=at.second)
        if now >= today_at_time:
            # the current time is already past the target time so it would rotate already
            # add one day to prevent an immediate rotation
            self._next_rotate = today_at_time + timedelta(days=1)
        else:
            self._next_rotate = today_at_time

    def should_rotate(self, message, file):
        file.seek(0, 2)
        if file.tell() + len(message) > self._size:
            return True
        if message.record["time"].timestamp() > self._next_rotate.timestamp():
            self._next_rotate += timedelta(days=1)
            return True
        return False

[dushitaoyuan]

that't great!, here a small fix:

from loguru import (
    logger,
    _string_parsers as string_parser
)
from datetime import (
    datetime,
    timedelta
)
class Rotator:
    def __init__(self, str_size, str_time):
        self._size = string_parser.parse_size(str_size)
        at = string_parser.parse_time(str_time)
        now = datetime.now()
        today_at_time = now.replace(hour=at.hour, minute=at.minute, second=at.second)
        if now >= today_at_time:
            # the current time is already past the target time so it would rotate already
            # add one day to prevent an immediate rotation
            self._next_rotate = today_at_time + timedelta(days=1)
        else:
            self._next_rotate = today_at_time

    def should_rotate(self, message, file):
        file.seek(0, 2)
        if file.tell() + len(message) > self._size:
            return True
        if message.record["time"].timestamp() > self._next_rotate.timestamp():
            self._next_rotate += timedelta(days=1)
            return True
        return False
size_time_rotator = Rotator("500 MB", "20:41")

and the test code

limit_min = 43
while True:
    now = datetime.now()
    if now.minute > limit_min:
       break
    else:
        LOG.info(" LOG LOGLOGLOGLOGLOGLOGLOGLOGLOGLOGLOGLOGLOGLOGLOGLOGLOGLOGLOGLOG")
        time.sleep(1)

print("finsh")

[mancioshell]

And if i would like to rotate every hour and every time log exceeds a file size ?

[Delgan]

@mancioshell Surely you can adapt the should_rotate() function in the above snippets so that it returns True only when both condition are met, right?

[mancioshell]

@Delgan I'm sorry, I explained myself badly :)
I would like to rotate every hour or every time log exceeds a file size. But i think in the example above it rotate every time log exceeds a file size or at a particular time (eg. 12:00 PM)

[mahadi]

Then your question is actually, how to achieve a rotation every hour, right? Because the size issue is already solved above.

[mancioshell]

@mahadi yep. I would like to rotate the log every hour or every time it exceeds 100Mb of size. Any advice to edit the example in the docs for my use case?

[Delgan]

@mancioshell Did you try amending the snippet with self._next_rotate += timedelta(hours=1) instead of self._next_rotate += timedelta(days=1)?

[mancioshell]

@Delgan could the following code works ?

class Rotator:

    def __init__(self, *, size, at):
        now = datetime.datetime.now()

        self._size_limit = size
        self._time_limit = now + datetime.timedelta(hours=1)

    def should_rotate(self, message, file):
        file.seek(0, 2)
        if file.tell() + len(message) > self._size_limit:
            return True
        if message.record["time"].timestamp() > self._time_limit.timestamp():
            self._time_limit += datetime.timedelta(hours=1)
            return True
        return False

[Delgan]

@mancioshell I guess it should. :)

[kanghaov]

if I want loguru rotate everytime I run the thread,How should I do ?
just return True?
ple

[Delgan]

@kanghaov Please can you open a new ticket describing more precisely the issue you're facing?
"Every time I run the thread" is not very clear to me.