Skip to content

Commit

Permalink
feat: switch the default mode to safe_mode
Browse files Browse the repository at this point in the history
  • Loading branch information
PragmaTwice committed Nov 27, 2024
1 parent 4c655c0 commit 61ca335
Show file tree
Hide file tree
Showing 9 changed files with 135 additions and 155 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ fabric.properties
.vs/
CMakeSettings.json
.vscode/
.cache/

build/
doxygen/
209 changes: 94 additions & 115 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,115 +1,94 @@
# ![puffer](asset/puffer.png) Protocol Puffers
[![Codacy](https://api.codacy.com/project/badge/Grade/31e7d1d7bcbe43959aaec3b86093b843)](https://app.codacy.com/gh/PragmaTwice/protopuf?utm_source=github.com&utm_medium=referral&utm_content=PragmaTwice/protopuf&utm_campaign=Badge_Grade)
[![Github Actions](/~https://github.com/PragmaTwice/protopuf/workflows/BuildAndTest/badge.svg)](/~https://github.com/PragmaTwice/protopuf/actions)
[![Vcpkg Port](https://img.shields.io/badge/vcpkg-protopuf-blue)](/~https://github.com/microsoft/vcpkg/blob/master/ports/protopuf/vcpkg.json)
[![Codecov](https://codecov.io/gh/PragmaTwice/protopuf/branch/master/graph/badge.svg?token=4EPLZ6Z4J5)](https://codecov.io/gh/PragmaTwice/protopuf)

*A little, highly templated, and protobuf-compatible serialization/deserialization library written in C++20*

:closed_book: [Documentation](https://protopuf.surge.sh)

## Requirements

- a compiler and a standard library implementation with C++20 support
- GCC 11 or above, or
- Clang 12 or above, or
- MSVC 14.29 (Visual Studio 2019 Version 16.9) or above
- CMake 3
- GoogleTest (optional, for unit tests)
- vcpkg (optional, `vcpkg install protopuf` to install)

## Features

- Data structures are described using type in C++ instead of DSLs like the Protocol Buffer Language (`.proto`)
- Fully compatible with encoding of the Protocol Buffers, capable of mutual serialization/deserialization
- Extensive compile-time operations aimed to improving run-time performance

## An Example
For the following data structure described using `.proto`:
```proto
message Student {
uint32 id = 1;
string name = 3;
}
message Class {
string name = 8;
repeated Student students = 3;
}
```
We can use *protopuf* to describe it as C++ types:
```c++
using namespace pp;

using Student = message<
uint32_field<"id", 1>,
string_field<"name", 3>
>;

using Class = message<
string_field<"name", 8>,
message_field<"students", 3, Student, repeated>
>;
```
Subsequently, both serialization and deserialization become so easy to do:
- Mode without buffer overflow control
```c++
// serialization
Student twice {123, "twice"}, tom{456, "tom"}, jerry{123456, "jerry"};
Class myClass {"class 101", {tom, jerry}};
myClass["students"_f].push_back(twice);

array<byte, 64> buffer{};
auto bufferEnd = message_coder<Class>::encode(myClass, buffer);
assert(begin_diff(bufferEnd, buffer) == 45);

// deserialization
auto [yourClass, bufferEnd2] = message_coder<Class>::decode(buffer);
assert(yourClass["name"_f] == "class 101");
assert(yourClass["students"_f][2]["name"_f] == "twice");
assert(yourClass["students"_f][2]["id"_f] == 123);
assert(yourClass["students"_f][1] == (Student{123456, "jerry"}));
assert(yourClass == myClass);
assert(begin_diff(bufferEnd2, bufferEnd) == 0);
```
- Mode with buffer overflow control (safe mode)
```c++
// serialization
Student twice {123, "twice"}, tom{456, "tom"}, jerry{123456, "jerry"};
Class myClass {"class 101", {tom, jerry}};
myClass["students"_f].push_back(twice);
array<byte, 64> buffer{};
auto result = message_coder<Class>::encode<safe_mode>(myClass, buffer);
assert (result.has_value());
const auto& bufferEnd = *result;
assert(begin_diff(bufferEnd, buffer) == 45);
// deserialization
auto result2 = message_coder<Class>::decode<safe_mode>(buffer);
assert (result2.has_value());
const auto& [yourClass, bufferEnd2] = *result2;
assert(yourClass["name"_f] == "class 101");
assert(yourClass["students"_f][2]["name"_f] == "twice");
assert(yourClass["students"_f][2]["id"_f] == 123);
assert(yourClass["students"_f][1] == (Student{123456, "jerry"}));
assert(yourClass == myClass);
assert(begin_diff(bufferEnd2, bufferEnd) == 0);
```
More examples can be found in our test cases ([test/message.cpp](/~https://github.com/PragmaTwice/protopuf/blob/master/test/message.cpp)).

## Supported Field Types
Category| Supported Types
--------|------------------
Varint | int32, int64, uint32, uint64, sint32, sint64, bool, enum
64-bit | fixed64, sfixed64, double
Length-delimited| string, bytes, embedded messages, packed repeated fields
32-bit | fixed32, sfixed32, float

## Known issues
- There is [a known bug](https://developercommunity2.visualstudio.com/t/Wrong-compile-error-in-MSVC:-identifier-/1270794) related to template parameter lists of lambda expressions in Visual Studio 2019 Version 16.8, which can produce a wrong compilation error while compiling protopuf
- Although class type in NTTP ([P0732R2](http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0732r2.pdf)) is implemented in GCC 10, there is a CTAD bug ([PR96331](https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96331), exists until GCC 10.2) to reject valid NTTP usage, which prevent protopuf to compile successfully

## Acknowledgement
- to @dngrudin for the contribution of out-of-bounds checking and unsafe/safe mode
- to [JetBrains](https://www.jetbrains.com/?from=protopuf) for its [Open Source License](https://www.jetbrains.com/community/opensource/?from=protopuf) of All Products Pack
# ![puffer](asset/puffer.png) Protocol Puffers
[![Codacy](https://api.codacy.com/project/badge/Grade/31e7d1d7bcbe43959aaec3b86093b843)](https://app.codacy.com/gh/PragmaTwice/protopuf?utm_source=github.com&utm_medium=referral&utm_content=PragmaTwice/protopuf&utm_campaign=Badge_Grade)
[![Github Actions](/~https://github.com/PragmaTwice/protopuf/workflows/BuildAndTest/badge.svg)](/~https://github.com/PragmaTwice/protopuf/actions)
[![Vcpkg Port](https://img.shields.io/badge/vcpkg-protopuf-blue)](/~https://github.com/microsoft/vcpkg/blob/master/ports/protopuf/vcpkg.json)
[![Codecov](https://codecov.io/gh/PragmaTwice/protopuf/branch/master/graph/badge.svg?token=4EPLZ6Z4J5)](https://codecov.io/gh/PragmaTwice/protopuf)

*A little, highly templated, and protobuf-compatible serialization/deserialization library written in C++20*

:closed_book: [Documentation](https://protopuf.surge.sh)

## Requirements

- a compiler and a standard library implementation with C++20 support
- GCC 11 or above, or
- Clang 12 or above, or
- MSVC 14.29 (Visual Studio 2019 Version 16.9) or above
- CMake 3
- GoogleTest (optional, for unit tests)
- vcpkg (optional, `vcpkg install protopuf` to install)

## Features

- Data structures are described using type in C++ instead of DSLs like the Protocol Buffer Language (`.proto`)
- Fully compatible with encoding of the Protocol Buffers, capable of mutual serialization/deserialization
- Extensive compile-time operations aimed to improving run-time performance

## An Example
For the following data structure described using `.proto`:
```proto
message Student {
uint32 id = 1;
string name = 3;
}
message Class {
string name = 8;
repeated Student students = 3;
}
```
We can use *protopuf* to describe it as C++ types:
```c++
using namespace pp;

using Student = message<
uint32_field<"id", 1>,
string_field<"name", 3>
>;

using Class = message<
string_field<"name", 8>,
message_field<"students", 3, Student, repeated>
>;
```
Subsequently, both serialization and deserialization become so easy to do:
```c++
// serialization
Student twice {123, "twice"}, tom{456, "tom"}, jerry{123456, "jerry"};
Class myClass {"class 101", {tom, jerry}};
myClass["students"_f].push_back(twice);

array<byte, 64> buffer{};
auto result = message_coder<Class>::encode(myClass, buffer);
assert(result.has_value());
const auto& bufferEnd = *result;
assert(begin_diff(bufferEnd, buffer) == 45);

// deserialization
auto result2 = message_coder<Class>::decode(buffer);
assert(result2.has_value());
const auto& [yourClass, bufferEnd2] = *result2;
assert(yourClass["name"_f] == "class 101");
assert(yourClass["students"_f][2]["name"_f] == "twice");
assert(yourClass["students"_f][2]["id"_f] == 123);
assert(yourClass["students"_f][1] == (Student{123456, "jerry"}));
assert(yourClass == myClass);
assert(begin_diff(bufferEnd2, bufferEnd) == 0);
```
More examples can be found in our test cases ([test/message.cpp](/~https://github.com/PragmaTwice/protopuf/blob/master/test/message.cpp)).
## Supported Field Types
Category| Supported Types
--------|------------------
Varint | int32, int64, uint32, uint64, sint32, sint64, bool, enum
64-bit | fixed64, sfixed64, double
Length-delimited| string, bytes, embedded messages, packed repeated fields
32-bit | fixed32, sfixed32, float
## Known issues
- There is [a known bug](https://developercommunity2.visualstudio.com/t/Wrong-compile-error-in-MSVC:-identifier-/1270794) in Visual Studio 2019 Version 16.8 related to template parameter lists of lambda expressions, which can produce a wrong compilation error while compiling protopuf
- Although class type in NTTP ([P0732R2](http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0732r2.pdf)) is implemented in GCC 10, there is a CTAD bug ([PR96331](https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96331), exists until GCC 10.2) to reject valid NTTP usage, which prevent protopuf to compile successfully
## Acknowledgement
- to @dngrudin for the contribution of out-of-bounds checking and unsafe/safe mode
- to [JetBrains](https://www.jetbrains.com/?from=protopuf) for its [Open Source License](https://www.jetbrains.com/community/opensource/?from=protopuf) of All Products Pack
6 changes: 3 additions & 3 deletions include/protopuf/array.h
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ namespace pp {

array_coder() = delete;

template <coder_mode Mode = unsafe_mode>
template <coder_mode Mode = safe_mode>
static constexpr encode_result<Mode> encode(const R& con, bytes b) {
uint<8> n = 0;

Expand All @@ -65,7 +65,7 @@ namespace pp {
return encode_result<Mode>{safe_b};
}

template <coder_mode Mode = unsafe_mode>
template <coder_mode Mode = safe_mode>
static constexpr decode_result<R, Mode> decode(bytes b) {
decode_value<uint<8>> decode_len;
if (!Mode::get_value_from_result(varint_coder<uint<8>>::decode<Mode>(b), decode_len)) {
Expand Down Expand Up @@ -111,7 +111,7 @@ namespace pp {
return n;
}

template <coder_mode Mode = unsafe_mode>
template <coder_mode Mode = safe_mode>
static constexpr decode_skip_result<Mode> decode_skip(bytes b) {
decode_value<uint<8>> decode_len;
if (!Mode::get_value_from_result(varint_coder<uint<8>>::decode<Mode>(b), decode_len)) {
Expand Down
4 changes: 2 additions & 2 deletions include/protopuf/bool.h
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,12 @@ namespace pp {

bool_coder() = delete;

template <coder_mode Mode = unsafe_mode>
template <coder_mode Mode = safe_mode>
static constexpr encode_result<Mode> encode(bool i, bytes b) {
return integer_coder<uint<1>>::encode<Mode>(i, b);
}

template <coder_mode Mode = unsafe_mode>
template <coder_mode Mode = safe_mode>
static constexpr decode_result<bool, Mode> decode(bytes b) {
return integer_coder<uint<1>>::decode<Mode>(b);
}
Expand Down
8 changes: 4 additions & 4 deletions include/protopuf/int.h
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,7 @@ namespace pp {

static constexpr std::size_t N = sizeof(T);

template <coder_mode Mode = unsafe_mode>
template <coder_mode Mode = safe_mode>
static constexpr encode_result<Mode> encode(T i, bytes b) {
if (!Mode::check_bytes_span(b, N)) {
return {};
Expand All @@ -207,7 +207,7 @@ namespace pp {
return encode_result<Mode>{b.subspan<N>()};
}

template <coder_mode Mode = unsafe_mode>
template <coder_mode Mode = safe_mode>
static constexpr decode_result<T, Mode> decode(bytes b) {
if (!Mode::check_bytes_span(b, N)) {
return {};
Expand All @@ -224,12 +224,12 @@ namespace pp {

integer_coder() = delete;

template <coder_mode Mode = unsafe_mode>
template <coder_mode Mode = safe_mode>
static constexpr encode_result<Mode> encode(T i, bytes b) {
return integer_coder<std::make_unsigned_t<T>>::template encode<Mode>(static_cast<std::make_unsigned_t<T>>(i), b);
}

template <coder_mode Mode = unsafe_mode>
template <coder_mode Mode = safe_mode>
static constexpr decode_result<T, Mode> decode(bytes b) {
return integer_coder<std::make_unsigned_t<T>>::template decode<Mode>(b);
}
Expand Down
30 changes: 15 additions & 15 deletions include/protopuf/message.h
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,10 @@ namespace pp {

/// Checks whether all types in the parameter list are equal
template <typename...>
constexpr bool are_same = true;
constexpr inline bool are_same = true;

template <typename T, typename... Ts>
constexpr bool are_same<T, Ts...> = std::conjunction_v<std::is_same<T, Ts>...>;
constexpr inline bool are_same<T, Ts...> = std::conjunction_v<std::is_same<T, Ts>...>;

template <typename U, typename F>
struct fold_impl {
Expand Down Expand Up @@ -234,10 +234,10 @@ namespace pp {

/// Checks whether the type is a @ref message type
template <typename>
constexpr bool is_message = false;
constexpr inline bool is_message = false;

template <typename ...T>
constexpr bool is_message <message<T...>> = true;
constexpr inline bool is_message <message<T...>> = true;

/// A concept satisfied while type `T` is a @ref message type
template <typename T>
Expand Down Expand Up @@ -301,15 +301,15 @@ namespace pp {
} {}
};

template <coder_mode Mode = unsafe_mode>
template <coder_mode Mode = safe_mode>
using message_skip_map = message_skip_map_impl<Mode, 0, 1, 2, 5>;
template <coder_mode Mode = unsafe_mode>
template <coder_mode Mode = safe_mode>
inline const message_skip_map<Mode> skip_map;

template <coder_mode Mode = unsafe_mode>
template <coder_mode Mode = safe_mode>
using message_decode_map_result = typename Mode::template result_type<std::pair<bytes, bool>>;

template <coder_mode Mode = unsafe_mode>
template <coder_mode Mode = safe_mode>
using message_decode_map_function_result = typename Mode::template result_type<bytes>;

template <coder_mode, message_c>
Expand Down Expand Up @@ -374,7 +374,7 @@ namespace pp {

message_coder() = delete;

template <coder_mode Mode = unsafe_mode>
template <coder_mode Mode = safe_mode>
static constexpr encode_result<Mode> encode(const T& msg, bytes b) {
encode_result<Mode> result{b};
msg.for_each([&result]<field_c F> (const F& f) {
Expand Down Expand Up @@ -410,13 +410,13 @@ namespace pp {
return result;
}

template <coder_mode Mode = unsafe_mode>
template <coder_mode Mode = safe_mode>
static constexpr decode_result<T, Mode> decode(bytes b) {
T v;

while(b.end() > b.begin()) {
std::pair<bytes, bool> bytes_with_next;
if (!Mode::get_value_from_result(decode_map<Mode, T>.decode(v, b), bytes_with_next)) {
if (!Mode::get_value_from_result(decode_map<Mode, T>.template decode<Mode>(v, b), bytes_with_next)) {
return {};
}

Expand Down Expand Up @@ -466,7 +466,7 @@ namespace pp {

embedded_message_coder() = delete;

template <coder_mode Mode = unsafe_mode>
template <coder_mode Mode = safe_mode>
static constexpr encode_result<Mode> encode(const T& msg, bytes b) {
auto n = skipper<message_coder<T>>::encode_skip(msg);

Expand All @@ -477,7 +477,7 @@ namespace pp {
return {};
}

template <coder_mode Mode = unsafe_mode>
template <coder_mode Mode = safe_mode>
static constexpr decode_result<T, Mode> decode(bytes b) {
T v;

Expand All @@ -492,7 +492,7 @@ namespace pp {
const auto origin_b = b;
while(begin_diff(b, origin_b) < len) {
std::pair<bytes, bool> bytes_with_next;
if (!Mode::get_value_from_result(decode_map<Mode, T>.decode(v, b), bytes_with_next)) {
if (!Mode::get_value_from_result(decode_map<Mode, T>.template decode<Mode>(v, b), bytes_with_next)) {
return {};
}

Expand All @@ -518,7 +518,7 @@ namespace pp {
return n;
}

template <coder_mode Mode = unsafe_mode>
template <coder_mode Mode = safe_mode>
static constexpr decode_skip_result<Mode> decode_skip(bytes b) {
decode_value<uint<8>> decode_len;
if (!Mode::get_value_from_result(varint_coder<uint<8>>::decode<Mode>(b), decode_len)) {
Expand Down
Loading

0 comments on commit 61ca335

Please sign in to comment.