[Bug]: pre_capture/post_capture not reflected in event clip.mp4

[joakimberg]

Checklist

• I have updated to the latest available Frigate version.
• I have cleared the cache of my browser.
• I have tried a different browser to see if it is related to my browser.
• I have tried reproducing the issue in incognito mode to rule out problems with any third party extensions or plugins I have installed.

Describe the problem you are having

Somewhere between frigate 0.14 and 0.15 event clips stopped including pre_capture / post_capture seconds of the video.

When fetching an mp4-clip of an event ( /api/events/:event_id/clip.mp4 ) the old expected behaviour was that the clip would start at (event_start - pre_capture) and end at (event_start + event_lenght + post_capture).

The new behaviour when making a request for the clip of an event seem to be that it will start exactly from when the detection started to exactly when the detection ended. Since the detect and record streams are not 100% aligned (common issue) this makes the videos even worse (the default 5 second pre/post padding was useful for that).

I'm curious if something has changed or if I have misconfigured anything. A friend of mine is also running Frigate 15.0 and he's experiencing the exact same issue.

Steps to reproduce

1. Set pre_capture/post_capture to 10 seconds
2. Trigger a short event, say 5 seconds.
3. Get the event ID
4. Download event clip from the API (/api/events/:event_id/clip.mp4)
5. Check length of the video. It will not be 25 sec (10+5+10), but rather 5sec (or exactly the length of the event, excluding pre/post).

Version

0.15.0-cea210d

In which browser(s) are you experiencing the issue with?

No response

Frigate config file

record:
  enabled: true
  sync_recordings: true
  retain:
    days: 7
    mode: all
  alerts:
    retain:
      days: 90
    pre_capture: 10
    post_capture: 10
  detections:
    retain:
      days: 90
    pre_capture: 10
    post_capture: 10

docker-compose file or Docker CLI command

Not relevant, get back to me in case you need it anyway.

Relevant Frigate log output

Nothing relevant, get back to me in case you need it anyway.

Relevant go2rtc log output

Nothing relevant, get back to me in case you need it anyway.

Operating system

Debian

Install method

Docker Compose

Network connection

Wired

Camera make and model

All hikvision.

Screenshots of the Frigate UI's System metrics pages

Not relevant.

Any other information that may be helpful

No response

[magma1447]

I have similar experience. This worked fine for me with 0.14. I then upgraded to 0.15.0-beta4 and started to experience this issue.

I never tried to downgrade to confirm that the issue appeared with 0.15.0-beta4 though. It could be something else that I changed around the same time.

I am now using 0.15.0-beta7 and still experiencing the same issue.

[NickM-27]

this is working as expected as the pre / post capture timing is not applied to the individual objects but instead the recordings that are kept.

[magma1447]

this is working as expected as the pre / post capture timing is not applied to the individual objects but instead the recordings that are kept.

So you are saying that this is an intentional change? I can not see any change in the documentation mentioning such change, therefore it seemed unintentional to me.

To me the change seems a bit weird. Object detection usually takes a while to detect the object. Imagine someone walking into the detection zone/camera view. The first 0.1 seconds where we only see a foot there won't be any object detected as a human. With some luck the object is detected when most of the body has entered, if we are less lucky the object might have passed half the camera view already. This will lead to the clip not including all of the object's time in the viewing area.

The pre/post capture helped with this. Especially with moving cars which often are a bit faster than walking humans. Intentional or not. Maybe it should have been two different options in the config otherwise?

From what I understand there are code changes in events/maintainer.py method handle_object_detection() that relates to this.

            start_time = event_data["start_time"] - event_config.pre_capture
            end_time = (
                None
                if event_data["end_time"] is None
                else event_data["end_time"] + event_config.post_capture
            )

vs

            start_time = event_data["start_time"]
            end_time = (
                None if event_data["end_time"] is None else event_data["end_time"]
            )

To me the clips have become more or less useless in 25% of the cases, and definitely worse in 75-90% of the cases. Some clips doesn't even include the object for me anymore, though it's fairly rare. But almost none have the full sequence of the object included.

The change first appeared in v0.15.0-beta1. It seems to have happened with this commit.

The Full Reference Config still includes this.

  # Optional: detection recording settings
  detections:
    # Optional: Number of seconds before the detection to include (default: shown below)
    pre_capture: 5
    # Optional: Number of seconds after the detection to include (default: shown below)
    post_capture: 5

If I understand you correctly the pre/post capture here will only be relevant if it's the first/last object detected in the same stream? And even if it's the first object, it will only store video from before the object detection, it won't return it when asking the api for the clip related to the object? That doesn't really make sense to me right now. But there is no doubt that you guys know Frigate better than I do, so I am probably missing something. It's also not super clear from the documentation, as I read it.

[NickM-27]

The change was intentional, this side effect wasn't expected. Frigate displays video in the UI based on review items not individual objects. In that case the extra time is recorded and viewable.

Requesting the event (object) clip no longer knows what the pre/post capture time is. Also worth mentioning that your description of an object being halfway across the frame before being detected is not normal and likely a config issue.

We may be able to work around this behavior and add a pre / post capture time, will have to see