First release: make default async, replace specs-generator system, trim down and organize files, etc.

[lcsmuller]

What?

This PR outlines implementation details for the first release of Concord, it differs from Orca in a couple aspects:

• Make async default #19
  • Asynchronous behavior for requests are now default.
• Refactor specs generator, JSON marshalling methods, lists #21
  • Replace specs-gen (JSON-based code generator) with gencodecs (a macro based generator).
  • Replace json_extract() with jsmn-find.
  • Replace json_inject() with json-build.

More

Changes for the user wishing to migrate their codebase to Concord

Why?

• Make async default #19
  • Performing requests synchronously blocks the event loop, consequently making them unresponsive. Because the WebSockets was already implemented in a single-thread, asynchronous fashion, it makes sense for the requests to take advantage of it too.
• Refactor specs generator, JSON marshalling methods, lists #21
  • specs-gen was not flexible and required modification of its source-code for any little fix or change. It relied on a custom-made JSON specs system that would be fed to it for generating code. gencodecs takes advantage of well-known C Preprocessor tools - in case a custom-type or behavior is required the user can create a custom macro wrapping for it, without a need to modify gencodecs source code directly.
  • json_extract() is not debugging friendly because it relies on variadic arguments for fetching JSON values, without any kind of type-checking to guarantee correctness of argument and expected JSON type.
  • json_inject() is not debugging friendly for the same reasons outlined above for json_extract(), and the its size was not respected, leading to crashes if the buffer it not large enough to accommodate the JSON.

How?

• Make async default #19
  See Async requests are now the default behavior
• Refactor specs generator, JSON marshalling methods, lists #21
  • specs-code JSON specs for code conversion, the following would be fed to specs/main.c, which would then generated code and comments via the specs-gen.c implementation.

   {
     "namespace": ["discord"],
     "comment":"https://discord.com/developers/docs/resources/channel",
     "defs":
     [
       {
         "title":"Message Reference Structure",
         "comment":"https://discord.com/developers/docs/resources/channel#message-object-message-reference-structure",
         "namespace":["message"],
         "struct":"reference",
         "fields": [
           {"name":"message_id", "type":{"base":"char", "dec":"*", "converter":"snowflake"}, "option":true, "inject_if_not":0},
           {"name":"channel_id", "type":{"base":"char", "dec":"*", "converter":"snowflake"}, "option":true, "inject_if_not":0},
           {"name":"guild_id", "type":{"base":"char", "dec":"*", "converter":"snowflake"}, "option":true, "inject_if_not":0},
           {"name":"fail_if_not_exists", "type":{"base":"bool"}, "option":true, "inject_if_not":false}
         ]
       }
     ]
   }

  the same conversion as above can be accomplished with gencodecs macros as follows, those macros can be expanded separately to ensure easier debuggability (see gencodecs/Makefile) or applied directly to code

   /** @brief https://discord.com/developers/docs/resources/channel#message-object-message-reference-structure */
   STRUCT(discord_message_reference)
        FIELD_SNOWFLAKE(message_id)
        FIELD_SNOWFLAKE(channel_id)
        FIELD_SNOWFLAKE(guild_id)
        FIELD(fail_if_not_exists, bool, true)
   STRUCT_END

  • json_extract() unsafe conversion

   char json[] = "\"foo\":\"this is a string\"";
   int a;
   
   json_extract(json, sizeof(json), "(foo):s", &a); // foo will match 'a' to a string, despite the fact that it should be a int

  jsmn-find fixes this by simply fetching the JSON token as a string, and leaving the conversion to the user.

   char json[] = "\"foo\":\"this is a string\"";
   int a;
   
   jsmnf *root = jsmnf_init();
   jsmnf_start(root, json, sizeof(json));
   
   jsmnf *f = jsmnf_find(f, "foo", strlen("foo"));
   if (f && f->val->type == JSMN_PRIMITIVE) {
     a = (int)strtol(json + f->val->start, NULL, 10); // conversion is type-checked and done on user-side
   }

  • json_inject() unsafe conversion

   char buf[1024];
   int a = 10;
   
   json_inject(buf, sizeof(buf), "(foo):s", a); // foo will attempt to read 'a' as a string, despite the fact that it should be a int

  json-build fixes this by strict type-checking via its function parameters

   char buf[1024];
   int a = 10;
   jsonb b;
   
   jsonb_init(&b);
   jsonb_object(&b, buf, sizeof(buf));
   {
     jsonb_key(&b, buf, sizeof(buf), "foo", strlen("foo"));
     jsonb_number(&b, buf, sizeof(buf), a); // type-checking done here
     jsonb_object_pop(&b, buf, sizeof(buf));
   }