Non-UTF-8 encoding causes error

[sevmonster]

About

Coming from GH-632, I am trying to make mqttwarn subscribe to Frigate's frigate/<camera_name>/<object_name>/snapshot topic, where Frigate publishes a jpeg encoded picture.

Problem

Currently mqttwarn expects a message encoded with UTF-8. Could it be possible to specify the encoding? I am trying to save a JPEG being sent over MQTT, and since it is obciously not UTF-8 encoded, mqttwarn exits with an error.

Details

mqttwarn.ini

[config:file]
append_newline = False
overwrite = True
targets = {
    'cam1-person': ['/media/snapshots/cam1-person.jpg'],
    'cam2-person': ['/media/snapshots/cam2-person.jpg']
    }

[frigate/cam1/person/snapshot]
targets = file:cam1-person
[frigate/cam2/person/snapshot]
targets = file:cam2-person

Exception

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte
Traceback (most recent call last):                                   File "/usr/local/bin/mqttwarn", line 8, in <module>
    sys.exit(run())
             ^^^^^
  File "/usr/local/lib/python3.11/site-packages/mqttwarn/commands.py", line 91, in run
    run_mqttwarn()
  File "/usr/local/lib/python3.11/site-packages/mqttwarn/commands.py", line 147, in run_mqttwarn                                          subscribe_forever()
  File "/usr/local/lib/python3.11/site-packages/mqttwarn/core.py", line 643, in subscribe_forever
    mqttc.loop_forever()
  File "/usr/local/lib/python3.11/site-packages/paho/mqtt/client.py", line 1756, in loop_forever
    rc = self._loop(timeout)
         ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/paho/mqtt/client.py", line 1164, in _loop
    rc = self.loop_read()                                                   ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/paho/mqtt/client.py", line 1556, in loop_read                                             rc = self._packet_read()
         ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/paho/mqtt/client.py", line 2439, in _packet_read                                          rc = self._packet_handle()
         ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/paho/mqtt/client.py", line 3033, in _packet_handle                                        return self._handle_publish()
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/paho/mqtt/client.py", line 3327, in _handle_publish
    self._handle_on_message(message)                                 File "/usr/local/lib/python3.11/site-packages/paho/mqtt/client.py", line 3570, in _handle_on_message
    on_message(self, self._userdata, message)
  File "/usr/local/lib/python3.11/site-packages/mqttwarn/core.py", line 168, in on_message
    payload = msg.payload.decode("utf-8")                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

[amotl]

Hi again,

thanks for reporting this problem when receiving binary payloads.

Of course, using base64 to encode the image payload, what we discussed ¹ and implemented ² the other day, is not of any value if mqttwarn would need to accept and process data from external systems, like, in this case, Frigate.

I will see what I can do about it.

With kind regards,
Andreas.

Footnotes

1. /~https://github.com/jpmens/mqttwarn/issues/284#issuecomment-360601716 ↩

2. /~https://github.com/daq-tools/mqttshot ↩

[amotl]

I am trying to save a JPEG being sent over MQTT, and since it is obciously not UTF-8 encoded, mqttwarn exits with an error.

mqttwarn should definitively not crash or otherwise exit in such a situation, 2fd8c3d fixes it. Thanks!

[amotl]

Hi again,

with 5eb4c17, mqttwarn will be able to receive and process binary message payloads, and with 60c6229, store them into files.

For the Frigate-to-Ntfy example, a58f7eb configures the new decode_utf8 = False option on the store-jpeg service, in order to make this happen.

With kind regards,
Andreas.

[sevmonster]

PR works for me, files are saved successfully.

[amotl]

Excellent. Thanks!