Pattern to examine frames with byte stuffing.

[GnuT0203]

I have a protocol that uses byte stuffing (like PPP). Single 0x7E are used to mark boundaries between frames. Within frames special characters can be escaped by 0x7D 0x.., while this escape sequence must be replaced by the byte following to 0x7D XOR 0x02 before I can examine the inner structure of the frame. The frame itself has an arbitrary length.
Is there a way to put this into am ImHex pattern?

[WerWolv]

Hey, I'd try to define a struct with a transform function that handles this case. We always unconditionally read the first byte. If that byte is 0x7D, we parse the second byte and create a new local variable inside that pattern that's the value XOR 0x02. If it's any other value, we just do the same with the value of the first byte.

Then the format function displays the decoded value as a hexadecimal number in the pattern data view and transform function lets you use the instances of that new struct as as if they're regular bytes.

import std.io;

struct EscapedByte {
    u8 firstByte;
    if (firstByte == 0x7D) {
        u8 secondByte;
        
        u8 value = secondByte ^ 0x02 [[export]];
    } else {
        u8 value = firstByte [[export]];
    }
} [[sealed, format("format_escaped_byte"), transform("transform_escaped_byte")]];

fn format_escaped_byte(ref auto escaped_byte) {
    return std::format("0x{:02X}", escaped_byte.value);
};

fn transform_escaped_byte(ref auto escaped_byte) {
    return escaped_byte.value;
};

EscapedByte bytes[10] @ 0x00;

[paxcut]

It is hard to tell what sort of format you want the result to have based on the description given. I made some assumptions and wrote some code:

import std.mem;

struct Escape {
    u8 lead[[hidden]];
    if (lead==0x7D) {
        u8 value[[hidden]];
        u8 result = value ^ 0x02[[export]];
    } else
        u8 result = lead[[export]];
};


struct Frame {
    Escape values[while($[$]!=0x7E && !std::mem::eof())];
    u8 end;
};

struct Start {
    u8 discard[while($[$]!=0x7E)];
    u8 lead;
    Frame frames[while(!std::mem::eof())];
};

Start start@0;

[GnuT0203]

Thank you very much. That gives me an glimpse of how to parse the outer frame and then continue examining the inner payload. Think I need to learn more about ImHex pattern programming.
Regards