ELM32: right bit shifting more that 32 bits does not work

[anegrin]

Describe the bug
When a long response is parsed, we do get uint64_t shifted by 8,16,24 etc; responseByte_0 to responseByte_0 to responseByte_4 are correctly calculated while responseByte_5 to responseByte_7 seems to be affected by 32bit overflow

To Reproduce
Using ELM32 do parse response "00B0:6240B40A4B001:003E0182DC4B00"

15:47:57.302 64-bit response: 
15:47:57.305 ^IresponseByte_0: 0
15:47:57.308 ^IresponseByte_1: 75
15:47:57.308 ^IresponseByte_2: 220
15:47:57.311 ^IresponseByte_3: 130
15:47:57.313 ^IresponseByte_4: 1
15:47:57.313 ^IresponseByte_5: 127 <- wrong
15:47:57.316 ^IresponseByte_6: 10 <- wrong
15:47:57.319 ^IresponseByte_7: 26 <- wrong

Expected behavior

15:47:57.302 64-bit response: 
15:47:57.305 ^IresponseByte_0: 0
15:47:57.308 ^IresponseByte_1: 75
15:47:57.308 ^IresponseByte_2: 220
15:47:57.311 ^IresponseByte_3: 130
15:47:57.313 ^IresponseByte_4: 1
15:47:57.313 ^IresponseByte_5: 62
15:47:57.316 ^IresponseByte_6: 0
15:47:57.319 ^IresponseByte_7: 0

Equipment
CrowPanel 3.5"-HMI ESP32 Display https://www.elecrow.com/esp32-display-3-5-inch-hmi-display-spi-tft-lcd-touch-screen.html

Code
Provide all applicable code in tags.

Additional Context

Full log:

15:47:57.171 Clearing input serial buffer
15:47:57.171 Sending the following command/query: 2240B43
15:47:57.230 ^IReceived char: 0
15:47:57.230 ^IReceived char: 0
15:47:57.233 ^IReceived char: B
15:47:57.233 ^IReceived char: \r
15:47:57.236 ^IReceived char: 0
15:47:57.238 ^IReceived char: :
15:47:57.238 ^IReceived char: 6
15:47:57.241 ^IReceived char: 2
15:47:57.241 ^IReceived char: 4
15:47:57.244 ^IReceived char: 0
15:47:57.244 ^IReceived char: B
15:47:57.247 ^IReceived char: 4
15:47:57.250 ^IReceived char: 0
15:47:57.250 ^IReceived char: A
15:47:57.253 ^IReceived char: 4
15:47:57.253 ^IReceived char: B
15:47:57.255 ^IReceived char: 0
15:47:57.258 ^IReceived char: 0
15:47:57.258 ^IReceived char: \r
15:47:57.261 ^IReceived char: 1
15:47:57.261 ^IReceived char: :
15:47:57.263 ^IReceived char: 0
15:47:57.266 ^IReceived char: 0
15:47:57.266 ^IReceived char: 3
15:47:57.269 ^IReceived char: E
15:47:57.269 ^IReceived char: 0
15:47:57.272 ^IReceived char: 1
15:47:57.275 ^IReceived char: 8
15:47:57.275 ^IReceived char: 2
15:47:57.277 ^IReceived char: D
15:47:57.277 ^IReceived char: C
15:47:57.280 ^IReceived char: 4
15:47:57.283 ^IReceived char: B
15:47:57.283 ^IReceived char: 0
15:47:57.286 ^IReceived char: 0
15:47:57.286 ^IReceived char: \r
15:47:57.288 ^IReceived char: \r
15:47:57.292 ^IReceived char: >
15:47:57.292 Delimiter found.
15:47:57.294 All chars received: 00B0:6240B40A4B001:003E0182DC4B00
15:47:57.297 Expected response header: 6240B4
15:47:57.302 Single response detected
15:47:57.302 64-bit response: 
15:47:57.305 ^IresponseByte_0: 0
15:47:57.308 ^IresponseByte_1: 75
15:47:57.308 ^IresponseByte_2: 220
15:47:57.311 ^IresponseByte_3: 130
15:47:57.313 ^IresponseByte_4: 1
15:47:57.313 ^IresponseByte_5: 127
15:47:57.316 ^IresponseByte_6: 10
15:47:57.319 ^IresponseByte_7: 26
15:47:57.319 Looking for lagging zeros
15:47:57.322 Lagging zeros not found - assuming leading zeros
15:47:57.327 35602(45964): Value for 40b4 is 1876451839220402944.00

i tried some exlicit casting while shifting without luck - maybe i'm missing some compiler option?

[jimwhitelaw]

That's an odd looking response, perhaps the ':' are being used to separate multiple responses. It looks like the query is expecting 3 responses, is that correct? What is the output if you query service 01 with PID 1C? That should return a value indicating the specific protocol supported.

[anegrin]

What is the output if you query service 01 with PID 1C

I'll check ASAP

That's an odd looking response, perhaps the ':' are being used to separate multiple responses. It looks like the query is expecting 3 responses, is that correct?

this is correct. I'm using it here:

/~https://github.com/anegrin/GiuliaTFT/blob/wip/src/main.cpp#L607-L610

int r = (int) _gt_processPID(0x22, 0x40B1, 3, 7, 1, 0); is indeed calling btELM327.processPID(...) and waiting for the reponse (separate thread so i can block) /~https://github.com/anegrin/GiuliaTFT/blob/wip/src/main.cpp#L362-L387

_gt_processPID signature is 1:1 to processPID (speaking about args)

it is properly "resolving" 003E0182DC4B00 and bit shifting goes crazy for values > 32

I'm thinking about "fixing" this by splitting log responses payload to avout shifting by 40, 48 etc...returned "value" would still be wrong by bytes would be fine

as this function is kinda working:

/~https://github.com/anegrin/GiuliaTFT/blob/wip/src/main.cpp#L260

[anegrin]

What is the output if you query service 01 with PID 1C

it's 6 EOBD (Europe), protocol is AUTOISO157654CAN29500

14:55:03.350 USB device detected
14:55:06.598 Connected to CH34x device
14:55:06.599 E (339) psram: PSRAM ID read err7375(46888): SERVICE_01,OBD_STANDARDS 6
14:55:08.096 7411(46888): Current protocol is AUTOISO157654CAN29500

[PowerBroker2]

Thanks for the report - I made findResponse() virtual, so now you can overload the function directly if need be. I also tried to implement a fix for the log responses (compiled, but not tested). Give the main branch a shot and let me know if it works

[anegrin]

unfortunately it goes nuts

17:58:26.770 Service: 34
17:58:26.770 PID: 16561
17:58:26.773 Long query detected
17:58:26.773 Query string: 2240B13
17:58:26.776 Clearing input serial buffer
17:58:26.779 Sending the following command/query: 2240B13
17:58:26.799 ^IReceived char: 0
17:58:26.799 ^IReceived char: 0
17:58:26.802 ^IReceived char: B
17:58:26.802 ^IReceived char: \r
17:58:26.805 ^IReceived char: 0
17:58:26.808 ^IReceived char: :
17:58:26.808 ^IReceived char: 6
17:58:26.810 ^IReceived char: 2
17:58:26.810 ^IReceived char: 4
17:58:26.813 ^IReceived char: 0
17:58:26.813 ^IReceived char: B
17:58:26.816 ^IReceived char: 1
17:58:26.818 ^IReceived char: 0
17:58:26.818 ^IReceived char: A
17:58:26.821 ^IReceived char: 1
17:58:26.821 ^IReceived char: 4
17:58:26.824 ^IReceived char: 0
17:58:26.827 ^IReceived char: 0
17:58:26.827 ^IReceived char: \r
17:58:26.830 ^IReceived char: 1
17:58:26.830 ^IReceived char: :
17:58:26.832 ^IReceived char: 0
17:58:26.835 ^IReceived char: 0
17:58:26.835 ^IReceived char: 3
17:58:26.838 ^IReceived char: F
17:58:26.838 ^IReceived char: 0
17:58:26.841 ^IReceived char: 1
17:58:26.843 ^IReceived char: 8
17:58:26.843 ^IReceived char: 3
17:58:26.846 ^IReceived char: D
17:58:26.846 ^IReceived char: C
17:58:26.849 ^IReceived char: 1
17:58:26.852 ^IReceived char: 4
17:58:26.852 ^IReceived char: 0
17:58:26.855 ^IReceived char: 0
17:58:26.855 ^IReceived char: \r
17:58:26.857 ^IReceived char: \r
17:58:26.860 ^IReceived char: >
17:58:26.860 Delimiter found.
17:58:26.863 All chars received: 00B0:6240B10A14001:003F0183DC1400
17:58:26.866 Expected response header: 6240B1
17:58:26.871 Single response detected
17:58:26.871 64-bit response: 
17:58:26.874 ^IresponseByte_0: 255
17:58:26.877 ^IresponseByte_1: 223
17:58:26.877 ^IresponseByte_2: 255
17:58:26.880 ^IresponseByte_3: 255
17:58:26.882 ^IresponseByte_4: 255
17:58:26.882 ^IresponseByte_5: 255
17:58:26.885 ^IresponseByte_6: 255
17:58:26.888 ^IresponseByte_7: 255
17:58:26.888 WARNING: Number of payload chars returned by ELM327 is an odd value - returning 0

and all other pids i'm fetching are aslo getting bad values. i'll try to find a way for me to test elmduino with some mock

[PowerBroker2]

Oops, yeah, I had a little bug in the code, but I think it's resolved now. Test with the updated main and let me know

[anegrin]

standard pids are working now. there're still issues with the long ones

i had to change char param to char* or it won't compile

logColonIndex = nextIndex(payload + secondHeadIndex, ":", 1);

21:32:06.232 Service: 34
21:32:06.232 PID: 16561
21:32:06.234 Long query detected
21:32:06.234 Query string: 2240B13
21:32:06.237 Clearing input serial buffer
21:32:06.240 Sending the following command/query: 2240B13
21:32:06.269 	Received char: 0
21:32:06.269 	Received char: 0
21:32:06.272 	Received char: B
21:32:06.272 	Received char: \r
21:32:06.275 	Received char: 0
21:32:06.278 	Received char: :
21:32:06.278 	Received char: 6
21:32:06.280 	Received char: 2
21:32:06.280 	Received char: 4
21:32:06.283 	Received char: 0
21:32:06.283 	Received char: B
21:32:06.286 	Received char: 1
21:32:06.289 	Received char: 0
21:32:06.289 	Received char: A
21:32:06.291 	Received char: 1
21:32:06.291 	Received char: 4
21:32:06.294 	Received char: 0
21:32:06.297 	Received char: 0
21:32:06.297 	Received char: \r
21:32:06.300 	Received char: 1
21:32:06.300 	Received char: :
21:32:06.302 	Received char: 0
21:32:06.305 	Received char: 0
21:32:06.305 	Received char: 3
21:32:06.308 	Received char: F
21:32:06.308 	Received char: 0
21:32:06.311 	Received char: 1
21:32:06.313 	Received char: 8
21:32:06.313 	Received char: 3
21:32:06.316 	Received char: D
21:32:06.316 	Received char: C
21:32:06.322 	Received char: 1
21:32:06.325 	Received char: 4
21:32:06.325 	Received char: 0
21:32:06.326 	Received char: 0
21:32:06.326 	Received char: \r
21:32:06.335 	Received char: \r
21:32:06.335 	Received char: >
21:32:06.338 Delimiter found.
21:32:06.338 All chars received: 00B0:6240B10A14001:003F0183DC1400
21:32:06.344 Expected response header: 6240B1
21:32:06.346 Single response detected
21:32:06.349 Log response detected
21:32:06.352 20 log bytes found - ignoring these bytes during processing
21:32:06.357 64-bit response: 
21:32:06.357 	responseByte_0: 10
21:32:06.360 	responseByte_1: 0
21:32:06.360 	responseByte_2: 0
21:32:06.363 	responseByte_3: 0
21:32:06.366 	responseByte_4: 0
21:32:06.366 	responseByte_5: 0
21:32:06.368 	responseByte_6: 0
21:32:06.371 	responseByte_7: 0
21:32:06.371 WARNING: Number of payload chars is less than the number of expected response chars returned by ELM327 - returning 0

how do you test this? is there a testsuite you can push to github? i'd be glad to help but testing live on the car slow down my process a lot. i was hardconding some values on a main func on a separate c file but then i got stuck as unfortunately c and c++ are not "my cup of tea"

I'm attaching a log file with a bunch of commands - it can maybe help

serial_20250218_213155.zip

[PowerBroker2]

is there a testsuite you can push to github?

No, unless there are unit tests specifically written into the lib code plus a CI/CD pipeline is established. Both are (at least to me) difficult to implement...

how do you test this?

Now that you mention it, @jimwhitelaw made an OBD2 simulator as described here and I made a rudimentary simulator here. A third and more difficult (but lower tech) option is to manually put the OBD2 responses in the serial monitor

I'll try and run some simulations tonight and see what I can do - thanks for the logs!

[PowerBroker2]

@anegrin Check the latest main branch - I validated it myself using my uber-simple python OBD2 simulator connected to a Teensy 4.0 via a TTL-USB converter.

Please note that in processPID() you should specify 1 expected response instead of 3

[anegrin]

so it works! but i had to request 2 expected responses;

this is what i get if i do ask for 1:

14:16:14.269 Service: 34
14:16:14.269 PID: 16561
14:16:14.272 Long query detected
14:16:14.272 Query string: 2240B11
14:16:14.275 Clearing input serial buffer
14:16:14.278 Sending the following command/query: 2240B11
14:16:14.310 	Received char: 0
14:16:14.310 	Received char: 0
14:16:14.312 	Received char: B
14:16:14.312 	Received char: \r
14:16:14.315 	Received char: 0
14:16:14.318 	Received char: :
14:16:14.318 	Received char: 6
14:16:14.320 	Received char: 2
14:16:14.320 	Received char: 4
14:16:14.323 	Received char: 0
14:16:14.323 	Received char: B
14:16:14.326 	Received char: 1
14:16:14.329 	Received char: 0
14:16:14.329 	Received char: A
14:16:14.332 	Received char: 1
14:16:14.332 	Received char: 4
14:16:14.335 	Received char: 0
14:16:14.337 	Received char: 0
14:16:14.337 	Received char: \r
14:16:14.340 	Received char: \r
14:16:14.340 	Received char: >
14:16:14.343 Delimiter found.
14:16:14.346 All chars received: 00B0:6240B10A1400
14:16:14.348 Expected response header: 6240B1
14:16:14.351 Payload: 00B0:6240B10A1400
14:16:14.354 Single response detected
14:16:14.357 	Processing hex nibble: 0
14:16:14.357 	Processing hex nibble: A
14:16:14.359 	Processing hex nibble: 1
14:16:14.362 	Processing hex nibble: 4
14:16:14.365 	Processing hex nibble: 0
14:16:14.368 	Processing hex nibble: 0
14:16:14.371 64-bit response: 
14:16:14.371 	responseByte_0: 0
14:16:14.373 	responseByte_1: 20
14:16:14.376 	responseByte_2: 10
14:16:14.376 	responseByte_3: 0
14:16:14.379 	responseByte_4: 0
14:16:14.379 	responseByte_5: 0
14:16:14.382 	responseByte_6: 0
14:16:14.384 	responseByte_7: 0
14:16:14.384 WARNING: Number of payload chars is less than the number of expected response chars returned by ELM327 - returning 0

this is what i get with expected = 2:

14:16:14.396 Service: 34
14:16:14.396 PID: 16562
14:16:14.398 Long query detected
14:16:14.398 Query string: 2240B22
14:16:14.401 Clearing input serial buffer
14:16:14.404 Sending the following command/query: 2240B22
14:16:14.466 	Received char: 0
14:16:14.466 	Received char: 0
14:16:14.469 	Received char: B
14:16:14.469 	Received char: \r
14:16:14.471 	Received char: 0
14:16:14.474 	Received char: :
14:16:14.474 	Received char: 6
14:16:14.477 	Received char: 2
14:16:14.477 	Received char: 4
14:16:14.480 	Received char: 0
14:16:14.480 	Received char: B
14:16:14.483 	Received char: 2
14:16:14.485 	Received char: 0
14:16:14.485 	Received char: A
14:16:14.488 	Received char: 3
14:16:14.488 	Received char: 0
14:16:14.491 	Received char: 0
14:16:14.493 	Received char: 0
14:16:14.493 	Received char: \r
14:16:14.496 	Received char: 1
14:16:14.496 	Received char: :
14:16:14.499 	Received char: 0
14:16:14.502 	Received char: 0
14:16:14.502 	Received char: 3
14:16:14.504 	Received char: D
14:16:14.504 	Received char: 0
14:16:14.507 	Received char: 1
14:16:14.510 	Received char: 8
14:16:14.510 	Received char: 3
14:16:14.513 	Received char: D
14:16:14.513 	Received char: C
14:16:14.516 	Received char: 3
14:16:14.518 	Received char: 0
14:16:14.518 	Received char: 0
14:16:14.521 	Received char: 0
14:16:14.521 	Received char: \r
14:16:14.524 	Received char: \r
14:16:14.528 	Received char: >
14:16:14.528 Delimiter found.
14:16:14.530 All chars received: 00B0:6240B20A30001:003D0183DC3000
14:16:14.532 Expected response header: 6240B2
14:16:14.538 Payload: 00B0:6240B20A30001:003D0183DC3000
14:16:14.541 Log response detected at index: 18
14:16:14.543 Single response detected
14:16:14.546 	Processing hex nibble: 0
14:16:14.549 	Processing hex nibble: 0
14:16:14.552 	Processing hex nibble: 3
14:16:14.552 	Processing hex nibble: D
14:16:14.555 	Processing hex nibble: 0
14:16:14.557 	Processing hex nibble: 1
14:16:14.560 	Processing hex nibble: 8
14:16:14.565 	Processing hex nibble: 3
14:16:14.568 	Processing hex nibble: D
14:16:14.568 	Processing hex nibble: C
14:16:14.570 	Processing hex nibble: 3
14:16:14.571 	Processing hex nibble: 0
14:16:14.574 	Processing hex nibble: 0
14:16:14.577 	Processing hex nibble: 0
14:16:14.580 64-bit response: 
14:16:14.580 	responseByte_0: 0
14:16:14.582 	responseByte_1: 48
14:16:14.585 	responseByte_2: 220
14:16:14.585 	responseByte_3: 131
14:16:14.588 	responseByte_4: 1
14:16:14.591 	responseByte_5: 61
14:16:14.591 	responseByte_6: 0
14:16:14.593 	responseByte_7: 0

thanx for support i'll try to be more proactive next time with a PR :)

[PowerBroker2]

Nice! Fixed in 3.3.1