Replies: 10 comments 24 replies
-
|
My take is that we should define semantics and then just change things as a bug fix, and not worry about compat with the previous world. Secondarily, I lean to looking at what Home Assistant norms are, and esphome, and aligning to that. I could also see battery_voltage for voltage in volts, separate from scaling to battery_level. But I could also see the argument that this is duplicative and thus bad. |
Beta Was this translation helpful? Give feedback.
-
|
There are already |
Beta Was this translation helpful? Give feedback.
-
|
There is always room for improvement. We don't like to break things for existing users, getting to Adding such a central field, say Whenever possible a field suffix should standardize the values, e.g. obvious suffixes like Some additional questions come to mind: what would be the defined top level? 100% i.e. 1.0? Should we cap there? What about values derived from battery mV with very fresh batteries above nominal voltage? What would be the nominal lowest values? 0% i.e. 0.0? Or should the nominal lowest value, e.g. derived from batteries or a battery_low flag be some random other value, 10%? Would there be a defined step for battery_warning, 30%? What about calculations that place minimum nominal battery voltage at, say, 10%, how much lower are they allowed to go? Should there be a cap or possibly negative values? In summary, if we do something like this then we need to think of and document all the weird edge cases, thoroughly. |
Beta Was this translation helpful? Give feedback.
-
|
While I agree that there might be pitfalls for other keys, which need proper investigations first, I think the battery_ok float or int is a pretty clear cut case, well suited as an initial case to bring this into the new auto-discovery world. AFAIK all the battery_ok float values are actually received from the device broadcasts with values between 0.0 - 1.0, but if there really are some decoder derived from the voltage level, yes, I agree, they should be capped at 1.0, but this would already be the current case, where all battery_ok keys are presented as percentage values. |
Beta Was this translation helpful? Give feedback.
-
|
We really have many decoders, at a quick glance at least these would need to be considered and possibly adapted to a new scheme:
|
Beta Was this translation helpful? Give feedback.
-
Have a look at my above linked OpenMQTTGateway fix, at least listing all the relevant float requiring decoder at the time of the RTL_433_ESP inclusion:
As I said, this could go on forever as a mainatnace nightmare, so a key naming fix is definitely the preferred solution, and would also make it possible for the rtl_433_mqtt_hass.py script to easily create correct auto-discovery entries. |
Beta Was this translation helpful? Give feedback.
-
|
I also chose to make the float battery_ok decoder the exception, as the majority do actually have the binary battery_ok state, making this majority actually wrongly discovered at the moment with the rtl_433_mqtt_hass.py script. |
Beta Was this translation helpful? Give feedback.
-
|
So I guess the OP should say concretely what is wrong with Home Assistant, as far as I understand it, uses "Battery Level" with 100% as full/new. I think it should just be scaled to nominal, with 0% = 0V. So basically battery_V / battery_nominal_V. That leaves out a lot of complexity, which is good and bad. We could also add battery_remaining to be intended to be 0% to 100% of the runtime from new batteries. But that seems way harder than anybody is really going to do. Plus, to do it right, you need to know about the battery that was inserted (alkaline vs NiMH, as the biggest example). Personally, I want to know the voltage, but I like the 100%=fresh translation in HA. But, agreed this is harder than I made it out to be and that a deliberate approach is in order. |
Beta Was this translation helpful? Give feedback.
-
HA auto discovery has two distinct device classes for battery values, a sensor class, for numerical percentage values between 0%-100%, and a binary_switch device class, with battery Normal or battery low. For which I made the fix in OpenMQTTGateway for the then included RTL_433_ESP applicable devices. I can post the images again here to visualise the differences now with OMG, with I already liked to above Correctly discovered float battery level
Correctly discovered binary battery states
Currently RTL_433 doesn't differentiate between these two types, float and binary, packing them all under a universal battery_ok key, making it impossible to properly discover this key, and resulting in the ugly crutch of converting the binary 0/1 state to either only o% or 100%, whereas a nice battery binary device class is available with HA discover. |
Beta Was this translation helpful? Give feedback.
-
|
Basically: the below definition of
Proposed change: restrict Introducing a new field should come with a good definion. Possibly with a well defined suffix. It is a possibilty to improve which we might not easily get later on (breaking change). |
Beta Was this translation helpful? Give feedback.
-
Totally agree with that :)
If it's battery_level, battery_percentage, battery_percentage_level or whatever … that is up to the RTL_433 team, as long as it is distinguishable from the binary battery_ok, making it easily possible for OpenMQTTGateway, the rtl_433_mqtt_hass.py script or any other RTL_433 auto-discovery implementation to assign the correct device classes. |
Beta Was this translation helpful? Give feedback.
-
|
Sort of related: https://andrew-codechimp.github.io/HA-Battery-Notes/ |
Beta Was this translation helpful? Give feedback.
-
|
Exactly my point, as described above and in the original post - to be able to correctly auto-discover the different battery_ok types. Currently everything is forced not very nicely into a percentage value. Also the links to the two different device classes for battery https://www.home-assistant.io/integrations/sensor/#device-class for the percentage level, and https://www.home-assistant.io/integrations/binary_sensor/#device-class for the binary battery_ok |
Beta Was this translation helpful? Give feedback.
-
|
There is somewhat of a related issue of not really having boolean types in rtl_433 data schema. Using ints (or floats in battery_ok) allows for overloading which can be very tempting to overload in order to shoehorn something in. But in the long term this can be detrimental for having clear semantics for consumers. Other places this comes up is for most motion, security, leak, etc. sensors that using Home Assistant's terminology are naturally binary sensors. (Side note: I think Home Assistant's binary_sensor device class for battery status didn't exist when the MQTT auto discovery script was created.) Separate discussion, the current MQTT auto discovery script is somewhat of a dead end because it only works on the level of the individual data elements, so it can't make per model/device decisions. It also doesn't really keep state so hard to filter out false positive decodes and control when things get added. @DigiH - I'm not familiar with OpenMQTTGateway, but if it has support for the rtl_433 ESP port, could it be used to read from rtl_433 directly and replace the auto discovery script and/or mqtt gateway script for Home Assistant users? |
Beta Was this translation helpful? Give feedback.
-
|
@DigiH - I finally got around to getting one of the LilyGo Lora v2.1 boards and getting the OpenMQTTGateway firmware installed this afternoon. I wound up buying directly from LilyGo. There were some weird things going on with the AliExpress listings. Wow, I'm pretty impressed with how much this is doing for such little cost and effort! All of those involved have done quite a nice job. I'll have to dig a bunch to see how viable a replacement it is for an RTL-SDR + RPI. It heard/discovered about 16 of my devices so far, so maybe 2/3. Possibly 2 or 3 of these could cover the whole house. Most notable thing missing is some of the messages from my Acurite Atlas weather station. For some reason, it seems to mostly miss message type 37 (temperature and humidity). There might be something else going on there. So far I haven't figured out how to tell what it thinks the noise level looks like. |
Beta Was this translation helpful? Give feedback.
-
|
Good to hear you are enjoying it @rct :) The main work has been done by @NorthernMan54 by porting the rtl_433 decoders and functionality to his rtl_433_ESP library, and @1technophile for the OpenMQTTGateway project. I mainly concentrate on the BLE decoding side and its discoveries, and through that also did some work on rtl_433 discovery, where I found the type ambiguity of identically named keys the biggest problem for producing correct discoveries - hence starting this discussion ;) I also only have several LilyGos now around the house for my rtl_433 devices. Not so much for reception range issues, but because OOK and FSK decoders require different separate binaries to be installed and 433MHz and 868MHz decoders required different appropriate LilyGo hardware. This might also be an issue for part of your non-received devices. My RTL-SDR is resting in a drawer since then. |
Beta Was this translation helpful? Give feedback.
-
|
There is a lot I need to dig into: improving reception, logging, and understanding the strengths/weaknesses of the hardware and platform. I see there is no timestamp in the message so correlating events between different receivers is gong to be problematic since it would be messaged from the time an MQTT client managed to get it from the broker with an unknown amount of latency. Clearly doesn't matter much for environment sensors like temperature, but gets more important for security sensors. Consider a door sensor that opens/closes or closes and opens again quickly. How do you know the last MQTT message you got is the correct state? It could be from a marginal receiver overwriting the correct state from a better performing receiver. I also need to figure out why my device gets into a mode where it reboots frequently.
I'll follow up in the appropriate repo once I have some clues. Sure would be nice if it was syslogging. Guess I might have to connect it to a computer so I can see the serial log output |
Beta Was this translation helpful? Give feedback.
-
Might this be helpful? Not available as a runtime settable MQTT command (so far), but as a custom environment/build build_flag /~https://github.com/1technophile/OpenMQTTGateway/blob/development/main/User_config.h#L410-L412 with an example MQTT message of {"model":"LaCrosse-TX141THBv2","id":37,"channel":0,"battery_ok":1,"temperature_C":-22.9,"humidity":78,"test":"No","mic":"CRC","protocol":"LaCrosse TX141-Bv2, TX141TH-Bv2, TX141-Bv3, TX141W, TX145wsdth, (TFA, ORIA) sensor","rssi":-50,"duration":175992,"UTCtime":"2024-12-30T19:14:43Z"} or the UNIX timestamp underneath. |
Beta Was this translation helpful? Give feedback.
-
I usually find this is happening when other devices, which I do not own, pass by my house, but being in a very rural area this doesn't happen very often for me, once every few days or weeks, definitely not as frequently as in your uptime screenshot above. @NorthernMan54 will be better suited to answer this, but he also provisioned for something like this in his library with the |
Beta Was this translation helpful? Give feedback.
-
|
Doctrine questions:
|
Beta Was this translation helpful? Give feedback.
-
|
I don't think rtl_433 should synthesize anything. If all the device gives is a binary battery low or battery ok, I'd like to know in the consumer and/or display that there will only ever be 2 values. I'd rather not have to think about why that battery is still at 100% when I not sure how many months have elapsed since it was last changed. BTW, Interesting find on the HA-Battery-Notes integration. I have to take a serious look at that. I always thought at a simple/useful thing Home Assistant could do is add a markdown card to the device info page that you could edit to store battery notes, link to manuals, etc. I like Zigbee2MQTTs access to doc links. |
Beta Was this translation helpful? Give feedback.
-
|
Re boolean types: this can be handled much nicer, type-safe and extensible, now that #2667 has landed. There is already one example with My opinion on the open questions:
|
Beta Was this translation helpful? Give feedback.
-
|
I guess there is enough hesitation about synthetic that for now we should say no, and we can add it later. It's harder to go the other way. Let's spell out battery_percent if we use that. I am not a fan of Trying to converge to something we can all be ok with, I lean to two keys, depending on what the device does:
we can then figure out how HA should take level and make percent, but at least we'll be reporting clean, useful data. |
Beta Was this translation helpful? Give feedback.
-
|
I see We currently have some instances where a battery level (sometimes derived from a voltage) is output as float in For other scalings, e.g. the 5-bar example we would need to decide if it's 1 to 5 x20, so 20% as lowest, or 0 to 4 x25. I'm not sure what |
Beta Was this translation helpful? Give feedback.
-
|
discovery probably needs to be upleveled, perhaps to some descriptor emitted by the driver, or in rtl_433 more properly. That's a much bigger rototill. Otherwise we're doing it twice. |
Beta Was this translation helpful? Give feedback.
-
|
Another motivation for upleveling/modernization is from the 2024.11 release notes
This makes things more device-centric where device info / keys like availability, origin, state_topic can all be shared. You wind up with basically one JSON config doc per device instead of one for each entity. The amount of redundant/verbose config info can be greatly reduced. Also with this style of configuration it would make it preferable to have rtl_433 publish an MQTT topic per device instead of a separate topic for each data item. Use a value_template to pull the desired entities out of the JSON state. It would cut the MQTT message load down considerably. (Yes you could do a lot of that now, but it is a lot more clumsy and difficult to maintain.) Note: the Home Assistant MQTT docs are more confusing than they need to be because they are using a lot of abbreviations. |
Beta Was this translation helpful? Give feedback.
-
One approach we've explored for BLE devices involves placing the device properties descriptor at the device level. If you're curious to see how this works in practice, you can try it out at https://parser.theengs.io. |
Beta Was this translation helpful? Give feedback.
-
|
There are two ways to represent information that is derived from voltage. Well, 3. One is to just output the voltage, and I would say any decoder that reads voltage should output battery_V (not mV, but V, this is using SI units vs some synthetic -- we already convert to metric units otherwise). Then, there is "battery_level", which is battery voltage divided by nominal battery voltage * 100%, That's a way to have something that behaves exactly like voltage but is on a common scale if graphing different kinds of devices. For example I am monitoring (not with rtl_433) 3 UPS units. 2 have 2x 12V lead acid, and 1 has 1. It would be nice to look at that as /24V and /12V since then it lines up nicely. Note that this has no belief about remaining life and curve shapes - it is not obscuring the underlying data, just fixed scaling. Finally there is battery_pct (or _percent), which is an estimate of remaining life, from 100% at new battery to 0% at failure and in theory linear in time. This is very tricky. Some devices report things that are like this and mapping is straightforward. Making up percent life from voltage is I think aspiratiional rather than reliable, absent device-specific calibration. If you think there is some min, and want to synthesize percent from new-battery-min (say 1.6V to 1.3V for AA, but that fails for NiMH), then I don't have a problem with that, but I would like to see battery_level which is 100% for 1.5V, sort of "percent of battery nominal voltage". As in also see, not instead. |
Beta Was this translation helpful? Give feedback.
-
|
Another ambiguity and inconsistency is with the uv and uvi keys. The Vevor 7-in-1 has the UV key actually being the UV Index, while my own Fineoffset-WH65B has both UV and UVI, which we discover as UV Level/Raw (UV) and UV Index (UVI), already causing a UV Index inconsistency and discovery mayhem with these two decoders without having even looked at any other UV and UVI implementations. |
Beta Was this translation helpful? Give feedback.
-
|
@DigiH yes, all UV Index fields should be named "uvi", please review #3131 |
Beta Was this translation helpful? Give feedback.
-
|
Having a look at #3131 I wonder what the two Bresser, the Cotech and FineOffset WS80 one decimal doubles are
Either some very obscure way of presenting a usually integer UV Index with additional in between decimals, or also some other raw data not indicating the UV Index at all?!? With the latter also requiring to rename their "UV Index" label. The initial implementations of these decoders might have moire insight into this, or if someone has one of these stations. |
Beta Was this translation helpful? Give feedback.
-
With the current state of RTL_433 keys indiscriminately covering different types with the same named keys, it is hard, if not impossible, to correctly auto-discover such key correctly.
The major example is the current battery_ok key, covering binary integer states for some devices, while others broadcast a float which can be nicely multiplied for a 0-100% battery level state.
Coming from the OpenMQTTGateway side, which uses the RTL_433_ESP port, this previously took over the RTL_433's own rtl_433_mqtt_hass.py script binary state conversion. This always was a thorn in my side side and a very ugly crutch tin my view, jumping from 100% to a nonsensical 0% only for any binary battery devices, making even reviewers think it 's a bug or what not.
I have mentioned this before in
#2948
and eventually implemented a manual filtering in OpenMQTTGateway to correct this. With the recent update of RTL_433_ESP, with new decoders included, this would mean going through all the added decoders again manually to see which new devices might need to be added to the manual filter at
/~https://github.com/1technophile/OpenMQTTGateway/blob/development/main/ZgatewayRTL_433.ino#L160
For which I do not have the time, nor the inclination at the moment.
It all boils down to the question if RTL_433 is willing and interested in making discoverable type distinctions for battery_ok, and possibly other similar keys, and owning up to the fact that it's 2024, well 2025 soon , and most users are non-technical relying on a preferable nicely well implemented auto-discovery, while renaming the float key to battery_ok_f and keeping battery_ok (for int/binary) would also not majorly break it for existing non-discovery users, even though battery_level and battery_ok would be more fitting keys.
Both the rtl_433_mqtt_hass.py script, as well as new devices in RTL_433_ESP would greatly benefit from this, presenting correctly battery device classes for auto-discovery, instead of the current state.
Thanks for reading my rant.
Beta Was this translation helpful? Give feedback.
All reactions