[Config Support]: Frigate to Home Assistant Zone Occupancy Issues

[es5445]

Describe the problem you are having

One of my frigate goals is to trigger a Home Assistant automation when a cat is detected within a "Countertop" zone. My configuration is exposing a Countertop device in Home Assistant and I'm using the "Countertop cat occupancy became occupied" entity to trigger the automation. This has been working well when there actually is a cat in the zone. The issue I'm having is the "Countertop cat occupancy became occupied" entity will occasionally show "Detected" when frigate history shows there are no alerts or detections and the recording shows nothing, not even any motion that seems significant.

My frigate instance is running in docker on an unraid machine, home assistant is in a VM on a separate machine. I feel I must be missing something in the config.

Version

0.14.1-f4f3cfa

Frigate config file

mqtt:
  host: 192.168.50.3
  user: ******
  password: ******

database:
  path: /config/frigate.db

ffmpeg:
  hwaccel_args: preset-vaapi


record:
  enabled: true
  retain:
    days: 31
    mode: all
  events:
    retain:
      default: 60
      mode: motion

detectors:
  ov:
    type: openvino
    device: GPU

#detectors:
#  coral:
#    type: edgetpu
#    device: pci

go2rtc:
  streams:
    murphy:
      - rtsp://admin:******@192.168.50.48:554/live
      - ffmpeg:murphy#audio=opus
    counter:
      - rtsp://admin:******@192.168.50.66:554/live
      - ffmpeg:counter#audio=opus
    driveway:
      - rtsp://admin:******@192.168.50.42:554/live
      - ffmpeg:driveway#audio=opus
    garage_se:
      - rtsp://admin:******@192.168.50.131:554/live
      - ffmpeg:garage_se#audio=opus
    garage_sw:
      - rtsp://admin:******@192.168.50.47:554/live
      - ffmpeg:garage_sw#audio=opus
    doorbell:
      - rtsp://admin:******@192.168.50.150:554/h264Preview_01_main
      - ffmpeg:doorbell#audio=aac
cameras:
  murphy: # <------ Name the camera
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/murphy # <----- The stream you want to use for detection
          roles:
            - record
        - path: rtsp://admin:******@192.168.50.48:554/cam/realmonitor?channel=1&subtype=1
          roles:
            - detect
    detect:
      enabled: true # <---- disable detection until you have a working camera feed
      width: 640
      height: 480
    snapshots:
      enabled: true
    motion:
      mask: 0.747,0.041,0.75,0.075,0.968,0.076,0.965,0.039
      threshold: 15
      contour_area: 10
      improve_contrast: 'true'
  doorbell: # <------ Name the camera
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/doorbell # <----- The stream you want to use for detection
          roles:
            - detect
            - record
    detect:
      enabled: true # <---- disable detection until you have a working camera feed
      width: 640
      height: 480
    snapshots:
      enabled: true
    motion:
      threshold: 15
      contour_area: 90
      improve_contrast: 'true'
      mask: 0.636,0.944,0.636,0.972,0.964,0.973,0.964,0.942
  driveway: # <------ Name the camera
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/driveway # <----- The stream you want to use for detection
          roles:
            - detect
        - path: rtsp://admin:******@192.168.50.42:554/cam/realmonitor?channel=1&subtype=1
          roles:
            - record
    detect:
      enabled: true # <---- disable detection until you have a working camera feed
      width: 1024
      height: 452
    snapshots:
      enabled: true
    motion:
      threshold: 15
      contour_area: 90
      improve_contrast: 'true'
      mask: 0.738,0.036,0.738,0.088,0.962,0.088,0.962,0.034
  garage_se: # <------ Name the camera
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/garage_se # <----- The stream you want to use for detection
          roles:
            - detect
        - path: rtsp://admin:******@192.168.50.131:554/cam/realmonitor?channel=1&subtype=1
          roles:
            - record
    detect:
      enabled: true # <---- disable detection until you have a working camera feed
      width: 704
      height: 480
    snapshots:
      enabled: true
    motion:
      threshold: 15
      contour_area: 90
      improve_contrast: 'true'
      mask: 0.663,0.031,0.663,0.077,0.973,0.076,0.973,0.032
  garage_sw: # <------ Name the camera
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/garage_sw # <----- The stream you want to use for detection
          roles:
            - detect
        - path: rtsp://admin:******@192.168.50.47:554/cam/realmonitor?channel=1&subtype=1
          roles:
            - record
    detect:
      enabled: true # <---- disable detection until you have a working camera feed
      width: 704
      height: 480
    snapshots:
      enabled: true
    motion:
      threshold: 15
      contour_area: 90
      improve_contrast: 'true'
      mask: 0.729,0.04,0.729,0.081,0.973,0.076,0.973,0.032
  counter: # <------ Name the camera
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/counter # <----- The stream you want to use for detection
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/counter
          roles:
            - record
    detect:
      enabled: true # <---- disable detection until you have a working camera feed
      width: 2560
      height: 1440
    snapshots:
      enabled: true
    motion:
      threshold: 15
      contour_area: 10
      improve_contrast: 'true'
      mask: 0.969,0.088,0.967,0.034,0.752,0.033,0.753,0.089
    zones:
      Countertop:
        coordinates: 
          0,0,0,1,0.38,1,0.077,0.566,0.242,0.477,0.432,0.441,0.511,0.439,0.606,0.38,0.662,0.374,0.66,0.251,0.171,0
        inertia: 3
        loitering_time: 0
        objects: cat
    review:
      detections:
        required_zones: Countertop
      alerts: {}
version: 0.15




objects:
  # Optional: list of objects to track from labelmap.txt (default: shown below)
  track:
    - person
    - cat
  filters:
    cat:
      min_score: 0.88
      threshold: 0.9


model:
  path: plus://*******

Relevant Frigate log output

Logs are clear after startup.

Relevant go2rtc log output

Logs are clear after startup.

Frigate stats

No response

Operating system

UNRAID

Install method

Docker Compose

docker-compose file or Docker CLI command

Standard Unraid template with frigate+ api key and intel igpu /dev/dri device for intel iGPU.

Object Detector

OpenVino

Screenshots of the Frigate UI's System metrics pages

No response

Any other information that may be helpful

No response

[hawkeye217]

See the FAQ in the official docs for the Home Assistant integration: https://docs.frigate.video/integrations/home-assistant/#i-have-set-up-automations-based-on-the-occupancy-sensors-sometimes-the-automation-runs-because-the-sensors-are-turned-on-but-then-i-look-at-frigate-i-cant-find-the-object-that-triggered-the-sensor-is-this-a-bug

[es5445]

I think I understand the thought that the floor for "occupancy" should be as low as possible.

I'm suggesting that many people would expect "occupancy" to be based on a detection or event. Instead it's apparently based on some situation that is unseen outside of the mqtt messages and unconfigurable in frigate. The very fact that there is a FAQ clarifying it's not a bug lends credence to that argument. Not having "occupancy" messages configurable in frigate, or filterable in the integration seems like a huge miss.

I'll take a peak at the mqtt documentation and see if I can work out the filtering. I appreciate the help.

[hawkeye217]

To be clear, occupancy is based on a detection. Whether that detection is seen as a false positive, whether it enters a zone with inertia, or whether it remains stationary or not, for example, are conditions that a user sets in their config file for retaining the footage. These conditions are not things that the HA integration provides any kind of direct access to via toggles, sliders, or other UI elements.

The integration is essentially a GUI on top of Frigate's MQTT API. This is why the recommendation is to use MQTT switches in HA to achieve fine tuned filtering for your particular needs.

[es5445]

"To be clear, occupancy is based on a detection."

I don't find this to be clear at all. When my home assistant sensor entity indicated occupancy detected the Review page of the Frigate web GUI doesn't show any detections or events. If it's based on a detection why am I not seeing a detection?

[hawkeye217]

Sorry, I realize there may be room for confusion. I used "detection" in context to what happens internally with Frigate's object detector.

Frigate searches for objects in your camera frame, but does not immediately mark everything it sees as a valid object. The object detecting AI models can produce false positives, so Frigate does a lot of work behind the scenes to make sure that a person is actually a person. Once Frigate is sufficiently confident, a tracked object is recorded and/or saved (depending on your Frigate config).

The Home Assistant sensors have fewer filters in place and do not go through the process of checking if detected objects are false positives or not. Because this process of checking/ensuring an object is a true positive takes a little time, using a filtered value would introduce a noticeable delay compared to if you were using an unfiltered sensor.

We've decided to leave the sensors unfiltered for that purpose (since many use them for turning on lights or switches), and then allow users to configure more restrictive filters with an MQTT sensor if they so choose.

Does that help explain it better?

[es5445]

That is a much clearer explanation. Thank you very much for taking the time.

I realize the home assistant integration is looking at the MQTT messages. That's exactly why I thought the integration would be the place to add some customization for occupancy detection. It would be great to have something like a toggle in the integration that would force occupancy detection to be based off of the same MQTT messages you're proposing I use for a generic MQTT sensor. Maybe it's not that simple or maybe my use case is unusual. Either way, I think I have a better understanding of how the system works, and I have a clear solution in using a generic MQTT sensor.

[hawkeye217]

Right. Unfortunately it's not that simple because users can configure Frigate in many different ways. For example, one user configuration may be true positives are objects that score above 85% confidence, but someone else's may be lower at 70%.

For more reliable false positive filtering, the recommended approach is to use MQTT sensors on either the frigate/events or frigate/reviews topics.

The frigate/events topic provides detailed messages about tracked objects, including:

• Object type and confidence scores
• Location information
• Zone changes
• Timestamps for events

The frigate/reviews topic includes:

• Detection and alert status
• Camera information
• Start and end times
• Object types detected

These MQTT topics give you more granular control over how you want to handle occupancy detection in your automations.

[es5445]

I've been using the following sensor config for the last few days, and I've been happy with the performance for my application. I appreciate the assistance.

mqtt:
sensor:
- name: "Countertop Cat Sensor"
state_topic: "frigate/events"
value_template: >
{% if (value_json['after']["camera"] == "counter")
and (value_json['after']["label"] == "cat")
and ("Countertop" in value_json['after']["entered_zones"]
or "Countertop" in value_json['after']["current_zones"]) %}
detected
{% else %}
not_detected
{% endif %}

[hawkeye217]

Great, glad it's working well for you. Happy to help.