From bc55ff6dbd9f25c202961739e6a8eb83d9f91703 Mon Sep 17 00:00:00 2001 From: Sorunome Date: Fri, 15 May 2020 11:42:32 +0200 Subject: [PATCH 01/35] add emotes proposal --- proposals/2545-emotes.md | 49 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 proposals/2545-emotes.md diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md new file mode 100644 index 00000000000..81d0b877843 --- /dev/null +++ b/proposals/2545-emotes.md @@ -0,0 +1,49 @@ +# MSC2545: Emotes + +Emotes.....emotes! + +# Proposal +Emotes have at least a shortcode and an mxc uri. They are sent as `` tags currently already in +the spec, as such existing clients *should* be able to render them (support for this is sadly poor, +even within riot flavours). Such an `` tag of a shortcode `:emote:` and an mxc uri `mxc://example.org/emote` +could look as follows: + +```html +:emote: +``` + +Both the `alt` and the `title` attributes are specified as they serve different purposes: `alt` is +displayed if e.g. the emote does not load. `title` is displayed e.g. on mouse hover over the emote. +The height is just a height that looks good on most devices with the normal, default font size. +No width is displayed as to not weirdly squish non-square emotes. + +## Emote sources +In order to be able to send emotes the client needs to have a list of shortcodes and their corresponding +mxc uris. For this there are two different emote sources: + +### User emotes +User emotes are per-user emotes that are defined in the users respective account data. The type for that +is `im.ponies.user_emotes`. The content is as following: + +```json +{ + "short": { + ":emote:": "mxc://example.org/blah", + ":other-emote:": "mxc://example.org/other-blah" + } +} +``` + +The emotes are defined inside of a dict called `short`, meant as the "short and easy" representation +of the emotes. Other, additional, keys may exist to define more metadata for the emotes. No such +guide exists yet. + +### Room emotes +Room emotes are per-room emotes that every user of a specific room can use inside of that room. They +are set with a state event of type `im.ponies-room_emotes` and a blank state key. The contents are +the same as for the user emotes. + +## Ideas + - Tag rooms as "these emotes can be used anywhere" inside of account data somehow + - Some kind of packs - for room emotes the state key could be used as pack identifier, for user + emotes something like `im.ponies.user_emotes.`. From 0f4bd5e6a68e45d49425f9c69f2b6f78e04f0531 Mon Sep 17 00:00:00 2001 From: Sorunome Date: Fri, 15 May 2020 11:47:52 +0200 Subject: [PATCH 02/35] add more stuffs --- proposals/2545-emotes.md | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index 81d0b877843..f4d8f53c681 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -43,7 +43,25 @@ Room emotes are per-room emotes that every user of a specific room can use insid are set with a state event of type `im.ponies-room_emotes` and a blank state key. The contents are the same as for the user emotes. -## Ideas +## Sending +In places where fancy tab-complete with the emote itself is not possible it is suggested that sending +the shortcode will convert to the img tag, e.g. sending `Hey there :wave:` will convert to `Hey there `. + +If there are collisions due to the same emote shortcode being present as both room emote and user emote +a user could specify the emote source by writing e.g. `:room~wave:` or `:user~wave:`. Other sources +could perhaps be added, see ideas section. + +# Ideas - Tag rooms as "these emotes can be used anywhere" inside of account data somehow - Some kind of packs - for room emotes the state key could be used as pack identifier, for user emotes something like `im.ponies.user_emotes.`. + +# Current implementations +## Emote rendering (rendering of the `` tag) + - riot-web + - revolution + - nheko + - fluffychat +## Emote sending, using the mentioned state events here + - revolution + - fluffychat From 2d191f6110467111676f8dfbba54d37f12f92a68 Mon Sep 17 00:00:00 2001 From: Sorunome Date: Fri, 15 May 2020 11:49:21 +0200 Subject: [PATCH 03/35] fix naming thingy --- proposals/2545-emotes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index f4d8f53c681..03d884071fb 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -62,6 +62,6 @@ could perhaps be added, see ideas section. - revolution - nheko - fluffychat -## Emote sending, using the mentioned state events here +## Emote sending, using the mentioned events here - revolution - fluffychat From fa3a8be7c521ee2677f3ef89306162e5056b2fa3 Mon Sep 17 00:00:00 2001 From: Sorunome Date: Fri, 15 May 2020 12:48:06 +0200 Subject: [PATCH 04/35] fix typo --- proposals/2545-emotes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index 03d884071fb..1a266ab85ba 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -40,7 +40,7 @@ guide exists yet. ### Room emotes Room emotes are per-room emotes that every user of a specific room can use inside of that room. They -are set with a state event of type `im.ponies-room_emotes` and a blank state key. The contents are +are set with a state event of type `im.ponies.room_emotes` and a blank state key. The contents are the same as for the user emotes. ## Sending From 96f28647ca72c22d5bd9851448634417882df768 Mon Sep 17 00:00:00 2001 From: Sorunome Date: Wed, 20 May 2020 12:19:07 +0200 Subject: [PATCH 05/35] outline room emote packs and emote rooms --- proposals/2545-emotes.md | 74 +++++++++++++++++++++++++++++++++++----- 1 file changed, 66 insertions(+), 8 deletions(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index 1a266ab85ba..38160174a31 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -23,7 +23,7 @@ mxc uris. For this there are two different emote sources: ### User emotes User emotes are per-user emotes that are defined in the users respective account data. The type for that -is `im.ponies.user_emotes`. The content is as following: +is `im.ponies.user_emotes` (`m.emotes`). The content is as following: ```json { @@ -40,21 +40,79 @@ guide exists yet. ### Room emotes Room emotes are per-room emotes that every user of a specific room can use inside of that room. They -are set with a state event of type `im.ponies.room_emotes` and a blank state key. The contents are -the same as for the user emotes. +are set with a state event of type `im.ponies.room_emotes` (`m.emotes`). The state key denotes a possible +pack, whereas the default one would be a blank state key. E.g. a discord bridge could set as state key +`de.sorunome.mx-puppet-bridge.discord` and have all the bridged emotes in said state event, keeping +bridged emotes from own, custom ones separate. + +The content extends that of the user emotes: It uses the `short` key, which is a map of the shortcode +of the emote to its mxc uri. Additionally, an optional `pack` key can be set, which defines meta-information +on the pack. The following attributes are there: + + - `pack.displayname`: An easy displayname for the pack. Defaults to the room name, if it doesn't exist + - `pack.avatar_url`: The mxc uri of an avatar/icon to display for the pack. Defaults to the room name, + if it doesn't exist. + - `pack.short`: A short identifier of the pack. Defaults to the normalized state key, and if the state + key is blank it defaults to "room". + + Normalized means here, converting spaces to `-`, taking only alphanumerical characters, `-` and `_`, + and casting it all to lowercase. + +As such, a content of a room emote pack could look as following: + +```json +{ + "short": { + ":emote:": "mxc://example.org/blah", + ":other-emote:": "mxc://example.org/other-blah" + }, + "pack": { + "displayname": "Emotes from Discord!", + "avatar_url": "mxc://example.org/discord_guild", + "short": "some_discord_guild" + } +} +``` + +### Emote rooms +While room emotes are specific to a room and are only accessible within that room, emote rooms should +be accessible from everywhere. They do not differentiate themself from room emotes at all, instead you +set an event in your account data of type `im.ponies.emote_rooms` (`m.emotes.rooms`) which outlines +which room emote states should be globally accessible for that user. For that, a `room` key contains +a map of room id that contains a map of state key that contains an optional pack definition override. +As such, the contents of `im.ponies.emote_rooms` could look as follows: + +```json +{ + "rooms": { + "!someroom:example.org": { + "": {}, + "de.sorunome.mx-puppet-bridge.discord": { + "name": "Overriden name", + "short": "new_short_name" + } + }, + "!someotherroom:example.org": { + "": {} + } + } +} +``` + +Here three emote packs are globally accessible to the user: Two defined in `!someroom:example.org` +(one with blank state key and one with state key `de.sorunome.mx-puppet-bridge.discord`) and one in +`!someotherroom:example.org`. ## Sending In places where fancy tab-complete with the emote itself is not possible it is suggested that sending the shortcode will convert to the img tag, e.g. sending `Hey there :wave:` will convert to `Hey there `. If there are collisions due to the same emote shortcode being present as both room emote and user emote -a user could specify the emote source by writing e.g. `:room~wave:` or `:user~wave:`. Other sources -could perhaps be added, see ideas section. +a user could specify the emote source by writing e.g. `:room~wave:` or `:user~wave:`. Here the short +pack name is used for room emotes, and "user" for user emotes. # Ideas - - Tag rooms as "these emotes can be used anywhere" inside of account data somehow - - Some kind of packs - for room emotes the state key could be used as pack identifier, for user - emotes something like `im.ponies.user_emotes.`. + - ??? # Current implementations ## Emote rendering (rendering of the `` tag) From 81ccecbc0421ea73fcc2a818661fc8689c450cb4 Mon Sep 17 00:00:00 2001 From: Sorunome Date: Wed, 20 May 2020 12:20:05 +0200 Subject: [PATCH 06/35] remove comma --- proposals/2545-emotes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index 38160174a31..408434f08c1 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -35,7 +35,7 @@ is `im.ponies.user_emotes` (`m.emotes`). The content is as following: ``` The emotes are defined inside of a dict called `short`, meant as the "short and easy" representation -of the emotes. Other, additional, keys may exist to define more metadata for the emotes. No such +of the emotes. Other, additional keys may exist to define more metadata for the emotes. No such guide exists yet. ### Room emotes From dc43e7996acbf21141698f17a34216f6b21e82df Mon Sep 17 00:00:00 2001 From: Sorunome Date: Wed, 20 May 2020 14:30:28 +0200 Subject: [PATCH 07/35] address things --- proposals/2545-emotes.md | 40 +++++++++++++++++++++------------------- 1 file changed, 21 insertions(+), 19 deletions(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index 408434f08c1..9673cc10964 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -23,7 +23,7 @@ mxc uris. For this there are two different emote sources: ### User emotes User emotes are per-user emotes that are defined in the users respective account data. The type for that -is `im.ponies.user_emotes` (`m.emotes`). The content is as following: +is `im.ponies.user_emotes` (later: `m.emotes`). The content is as following: ```json { @@ -34,31 +34,30 @@ is `im.ponies.user_emotes` (`m.emotes`). The content is as following: } ``` -The emotes are defined inside of a dict called `short`, meant as the "short and easy" representation -of the emotes. Other, additional keys may exist to define more metadata for the emotes. No such -guide exists yet. +The emotes are defined inside of a dict called `short`, which stands for shortcode. Other, additional +keys may exist to define more metadata for the emotes. No such guide exists yet. ### Room emotes Room emotes are per-room emotes that every user of a specific room can use inside of that room. They -are set with a state event of type `im.ponies.room_emotes` (`m.emotes`). The state key denotes a possible +are set with a state event of type `im.ponies.room_emotes` (later: `m.emotes`). The state key denotes a possible pack, whereas the default one would be a blank state key. E.g. a discord bridge could set as state key `de.sorunome.mx-puppet-bridge.discord` and have all the bridged emotes in said state event, keeping -bridged emotes from own, custom ones separate. +bridged emotes from matrix emotes separate. The content extends that of the user emotes: It uses the `short` key, which is a map of the shortcode of the emote to its mxc uri. Additionally, an optional `pack` key can be set, which defines meta-information -on the pack. The following attributes are there: +on the pack. The following keys for `pack` are valid: - - `pack.displayname`: An easy displayname for the pack. Defaults to the room name, if it doesn't exist - - `pack.avatar_url`: The mxc uri of an avatar/icon to display for the pack. Defaults to the room name, + - `displayname`: An easy displayname for the pack. Defaults to the room name, if it doesn't exist + - `avatar_url`: The mxc uri of an avatar/icon to display for the pack. Defaults to the room name, if it doesn't exist. - - `pack.short`: A short identifier of the pack. Defaults to the normalized state key, and if the state + - `name`: A short identifier of the pack. Defaults to the normalized state key, and if the state key is blank it defaults to "room". Normalized means here, converting spaces to `-`, taking only alphanumerical characters, `-` and `_`, - and casting it all to lowercase. + and casting it all to lowercase. In regex, this would be `[a-z0-9-_]+`. -As such, a content of a room emote pack could look as following: +As such, a `im.ponies.room_emotes` (later: `m.emotes`) state event could look like the following: ```json { @@ -69,18 +68,20 @@ As such, a content of a room emote pack could look as following: "pack": { "displayname": "Emotes from Discord!", "avatar_url": "mxc://example.org/discord_guild", - "short": "some_discord_guild" + "name": "some_discord_guild" } } ``` ### Emote rooms While room emotes are specific to a room and are only accessible within that room, emote rooms should -be accessible from everywhere. They do not differentiate themself from room emotes at all, instead you -set an event in your account data of type `im.ponies.emote_rooms` (`m.emotes.rooms`) which outlines +be accessible from everywhere. They do not differentiate themselves from room emotes at all, instead you +set an event in your account data of type `im.ponies.emote_rooms` (later: `m.emotes.rooms`) which outlines which room emote states should be globally accessible for that user. For that, a `room` key contains -a map of room id that contains a map of state key that contains an optional pack definition override. -As such, the contents of `im.ponies.emote_rooms` could look as follows: +a map of room ids that map to state keys that map to an optional pack definition override. +If you are currently viewing a room that is in your `im.ponies.emote_rooms` (later: `m.emotes.rooms`) +it is expected that the client de-duplicates the packs, to only give suggestions for it onces. +The the contents of `im.ponies.emote_rooms` (later: `m.emotes.rooms`) could look like the following: ```json { @@ -88,7 +89,7 @@ As such, the contents of `im.ponies.emote_rooms` could look as follows: "!someroom:example.org": { "": {}, "de.sorunome.mx-puppet-bridge.discord": { - "name": "Overriden name", + "displayname": "Overriden name", "short": "new_short_name" } }, @@ -109,7 +110,8 @@ the shortcode will convert to the img tag, e.g. sending `Hey there :wave:` will If there are collisions due to the same emote shortcode being present as both room emote and user emote a user could specify the emote source by writing e.g. `:room~wave:` or `:user~wave:`. Here the short -pack name is used for room emotes, and "user" for user emotes. +pack name is used for room emotes, and "user" for user emotes. If a room pack does not have a short +pack name and has a blank state key, then "room" is used. # Ideas - ??? From 4435d75564b3b93dee16459cc1c9f983ea6024f1 Mon Sep 17 00:00:00 2001 From: Sorunome Date: Sat, 20 Jun 2020 12:37:49 +0200 Subject: [PATCH 08/35] update emotes msc --- proposals/2545-emotes.md | 45 +++++++++++++++++++++++++++++++++++++--- 1 file changed, 42 insertions(+), 3 deletions(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index 9673cc10964..a6fb9818645 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -5,11 +5,15 @@ Emotes.....emotes! # Proposal Emotes have at least a shortcode and an mxc uri. They are sent as `` tags currently already in the spec, as such existing clients *should* be able to render them (support for this is sadly poor, -even within riot flavours). Such an `` tag of a shortcode `:emote:` and an mxc uri `mxc://example.org/emote` +even within riot flavours). To allow clients to distinguish emotes from other inline images, a new +property `data-mx-emote` is introduced. A client may chose to ignore the size attributes of emotes +when rendering, and instead pick the size based on other circumstances. This could e.g. be used to +display emotes in messages with only emotes and emoji larger than usual, which is commonly found in +messengers. Such an `` tag of a shortcode `:emote:` and an mxc uri `mxc://example.org/emote` could look as follows: ```html -:emote: +:emote: ``` Both the `alt` and the `title` attributes are specified as they serve different purposes: `alt` is @@ -19,7 +23,10 @@ No width is displayed as to not weirdly squish non-square emotes. ## Emote sources In order to be able to send emotes the client needs to have a list of shortcodes and their corresponding -mxc uris. For this there are two different emote sources: +mxc uris. All the emote sources described here are merely suggestions on where to get emotes from. +A client is free to implement / design their own way of fetching emotes, e.g. via dropping image files +into a folder and uploading them on-the-fly. For cross-compatibility between clients it'd still be a +good idea to implement the emote sources described here. For this there are two different emote sources: ### User emotes User emotes are per-user emotes that are defined in the users respective account data. The type for that @@ -125,3 +132,35 @@ pack name and has a blank state key, then "room" is used. ## Emote sending, using the mentioned events here - revolution - fluffychat + +# Security Considerations +When sending an `` tag in an end-to-end encrypted room, the client will make a request to fetch +said image, in this case an emote. As there is no way to encrypt content behind `` tags yet, +this could potentially leak part of the conversation. This is **not** a new security consideration, +it already exists. This, however, isn't much different of posting someone a link in an e2ee chat and +the recipient opens the link. Additionally, images, and thus emotes, are often cached by the client, +not even necessarily leading to an http query. + +# Alternatives +One can easily think of near infinite ways to implement emotes. The aspect of sending an `` tag +and marking it as an emote with `data-mx-emote` seems to be pretty universal, though, so the question +is mainly on different emote sources. For that, a separate MSC, [MSC1951](/~https://github.com/matrix-org/matrix-doc/pull/1951) +already exists, so it is reasonable to compare this one with that one: + +## Comparison with MSC1951 +MSC1951 defines as only emote source a dedicated room. This MSC, however, also allows you to bind emotes +to your own account, offering greater flexibility. In MSC1951 there can also only be a single emote +pack in a room. This could be problematic in e.g. bridged rooms: You set some emotes from the matrix +side and a discord bridge would plop all the discord emotes in another pack in the same room. + +MSC1951 defines a way to recommend using a pack of a different room - this MSC does not have an equivalent +of that. Instead, this MSC allows multiple emote packs for a room, and allows you to enable an emote +pack to be globally available for yourself accross all rooms you are in. + +The core difference is how MSC1951 and this MSC define the emotes themselves: In MSC1951 you have to +set one state event *per emote*. While this might seem like a nice idea on the surface, it doesn't +scale well. There are people who easily use and want hundreds or even thousands of emotes accessible. +A simple dict of shortcode to mxc URI seems more appropriate for this. + +In general, MSC1951 feels like a heavier approach to emote sourcs, while this MSC is more lightweight +and thus should allow significantly larger packs. From 79aae8adf4429fd8f1d88ee514066e85faa2855d Mon Sep 17 00:00:00 2001 From: Sorunome Date: Sat, 20 Jun 2020 12:38:35 +0200 Subject: [PATCH 09/35] update --- proposals/2545-emotes.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index a6fb9818645..f5b30f5ec50 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -162,5 +162,8 @@ set one state event *per emote*. While this might seem like a nice idea on the s scale well. There are people who easily use and want hundreds or even thousands of emotes accessible. A simple dict of shortcode to mxc URI seems more appropriate for this. +Additionally, this MSC is already used in the wild since about two years by some clients, while MSC1951 +only exists in MSC form, so far. + In general, MSC1951 feels like a heavier approach to emote sourcs, while this MSC is more lightweight and thus should allow significantly larger packs. From 4fea69b80103195904061d6a2a66ad4f104565bb Mon Sep 17 00:00:00 2001 From: Sorunome Date: Wed, 24 Jun 2020 09:46:12 +0200 Subject: [PATCH 10/35] Fix typos and grammar mistakes --- proposals/2545-emotes.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index f5b30f5ec50..5af69fbb403 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -137,33 +137,33 @@ pack name and has a blank state key, then "room" is used. When sending an `` tag in an end-to-end encrypted room, the client will make a request to fetch said image, in this case an emote. As there is no way to encrypt content behind `` tags yet, this could potentially leak part of the conversation. This is **not** a new security consideration, -it already exists. This, however, isn't much different of posting someone a link in an e2ee chat and +it already exists. This, however, isn't much different from posting someone a link in an e2ee chat and the recipient opens the link. Additionally, images, and thus emotes, are often cached by the client, not even necessarily leading to an http query. # Alternatives One can easily think of near infinite ways to implement emotes. The aspect of sending an `` tag -and marking it as an emote with `data-mx-emote` seems to be pretty universal, though, so the question +and marking it as an emote with `data-mx-emote` seems to be pretty universal though, so the question is mainly on different emote sources. For that, a separate MSC, [MSC1951](/~https://github.com/matrix-org/matrix-doc/pull/1951) already exists, so it is reasonable to compare this one with that one: ## Comparison with MSC1951 -MSC1951 defines as only emote source a dedicated room. This MSC, however, also allows you to bind emotes +MSC1951 defines a dedicated room as the only emote source. This MSC, however, also allows you to bind emotes to your own account, offering greater flexibility. In MSC1951 there can also only be a single emote pack in a room. This could be problematic in e.g. bridged rooms: You set some emotes from the matrix side and a discord bridge would plop all the discord emotes in another pack in the same room. MSC1951 defines a way to recommend using a pack of a different room - this MSC does not have an equivalent -of that. Instead, this MSC allows multiple emote packs for a room, and allows you to enable an emote -pack to be globally available for yourself accross all rooms you are in. +to that. Instead, this MSC allows multiple emote packs for a room, and allows you to enable an emote +pack to be globally available for yourself across all rooms you are in. The core difference is how MSC1951 and this MSC define the emotes themselves: In MSC1951 you have to set one state event *per emote*. While this might seem like a nice idea on the surface, it doesn't scale well. There are people who easily use and want hundreds or even thousands of emotes accessible. A simple dict of shortcode to mxc URI seems more appropriate for this. -Additionally, this MSC is already used in the wild since about two years by some clients, while MSC1951 -only exists in MSC form, so far. +Additionally, this MSC has already been used in the wild for roughly two years by some clients, while MSC1951 +only exists in MSC form. In general, MSC1951 feels like a heavier approach to emote sourcs, while this MSC is more lightweight -and thus should allow significantly larger packs. +and thus should allow for significantly larger packs. From 0e382e0b4dbd7aa98db0c332a9416908d4ff024d Mon Sep 17 00:00:00 2001 From: Sorunome Date: Tue, 1 Sep 2020 10:07:42 +0200 Subject: [PATCH 11/35] m.emote --> m.emoticon, update dedupe and pack priority --- proposals/2545-emotes.md | 69 ++++++++++++++++++++++------------------ 1 file changed, 38 insertions(+), 31 deletions(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index 5af69fbb403..c103b808283 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -1,19 +1,19 @@ -# MSC2545: Emotes +# MSC2545: Emoticons -Emotes.....emotes! +Emoticons, or short emotes...we need them! -# Proposal -Emotes have at least a shortcode and an mxc uri. They are sent as `` tags currently already in +## Proposal +Emoticons have at least a shortcode and an mxc uri. They are sent as `` tags currently already in the spec, as such existing clients *should* be able to render them (support for this is sadly poor, -even within riot flavours). To allow clients to distinguish emotes from other inline images, a new -property `data-mx-emote` is introduced. A client may chose to ignore the size attributes of emotes +even within riot flavours). To allow clients to distinguish emoticons from other inline images, a new +property `data-mx-emoticon` is introduced. A client may chose to ignore the size attributes of emoticons when rendering, and instead pick the size based on other circumstances. This could e.g. be used to -display emotes in messages with only emotes and emoji larger than usual, which is commonly found in +display emoticons in messages with only emoticons and emoji larger than usual, which is commonly found in messengers. Such an `` tag of a shortcode `:emote:` and an mxc uri `mxc://example.org/emote` could look as follows: ```html -:emote: +:emote: ``` Both the `alt` and the `title` attributes are specified as they serve different purposes: `alt` is @@ -21,16 +21,16 @@ displayed if e.g. the emote does not load. `title` is displayed e.g. on mouse ho The height is just a height that looks good on most devices with the normal, default font size. No width is displayed as to not weirdly squish non-square emotes. -## Emote sources +### Emoticon sources In order to be able to send emotes the client needs to have a list of shortcodes and their corresponding mxc uris. All the emote sources described here are merely suggestions on where to get emotes from. A client is free to implement / design their own way of fetching emotes, e.g. via dropping image files into a folder and uploading them on-the-fly. For cross-compatibility between clients it'd still be a good idea to implement the emote sources described here. For this there are two different emote sources: -### User emotes +#### User emoticons User emotes are per-user emotes that are defined in the users respective account data. The type for that -is `im.ponies.user_emotes` (later: `m.emotes`). The content is as following: +is `im.ponies.user_emotes` (later: `m.emoticons`). The content is as following: ```json { @@ -44,9 +44,9 @@ is `im.ponies.user_emotes` (later: `m.emotes`). The content is as following: The emotes are defined inside of a dict called `short`, which stands for shortcode. Other, additional keys may exist to define more metadata for the emotes. No such guide exists yet. -### Room emotes +#### Room emoticons Room emotes are per-room emotes that every user of a specific room can use inside of that room. They -are set with a state event of type `im.ponies.room_emotes` (later: `m.emotes`). The state key denotes a possible +are set with a state event of type `im.ponies.room_emotes` (later: `m.emoticons`). The state key denotes a possible pack, whereas the default one would be a blank state key. E.g. a discord bridge could set as state key `de.sorunome.mx-puppet-bridge.discord` and have all the bridged emotes in said state event, keeping bridged emotes from matrix emotes separate. @@ -60,11 +60,11 @@ on the pack. The following keys for `pack` are valid: if it doesn't exist. - `name`: A short identifier of the pack. Defaults to the normalized state key, and if the state key is blank it defaults to "room". - + Normalized means here, converting spaces to `-`, taking only alphanumerical characters, `-` and `_`, and casting it all to lowercase. In regex, this would be `[a-z0-9-_]+`. -As such, a `im.ponies.room_emotes` (later: `m.emotes`) state event could look like the following: +As such, a `im.ponies.room_emotes` (later: `m.emoticons`) state event could look like the following: ```json { @@ -80,15 +80,13 @@ As such, a `im.ponies.room_emotes` (later: `m.emotes`) state event could look li } ``` -### Emote rooms +#### Emoticon rooms While room emotes are specific to a room and are only accessible within that room, emote rooms should be accessible from everywhere. They do not differentiate themselves from room emotes at all, instead you -set an event in your account data of type `im.ponies.emote_rooms` (later: `m.emotes.rooms`) which outlines +set an event in your account data of type `im.ponies.emote_rooms` (later: `m.emoticons.rooms`) which outlines which room emote states should be globally accessible for that user. For that, a `room` key contains a map of room ids that map to state keys that map to an optional pack definition override. -If you are currently viewing a room that is in your `im.ponies.emote_rooms` (later: `m.emotes.rooms`) -it is expected that the client de-duplicates the packs, to only give suggestions for it onces. -The the contents of `im.ponies.emote_rooms` (later: `m.emotes.rooms`) could look like the following: +The the contents of `im.ponies.emote_rooms` (later: `m.emoticons.rooms`) could look like the following: ```json { @@ -111,7 +109,19 @@ Here three emote packs are globally accessible to the user: Two defined in `!som (one with blank state key and one with state key `de.sorunome.mx-puppet-bridge.discord`) and one in `!someotherroom:example.org`. -## Sending +### Emoticon source priority and deduplicating +When giving emoticon suggestions, clients are expected to deduplicate emotes by their mxc url. This +not only ensures that, when viewing a room that you also have in `m.emoticons.rooms`, it won't be +displayed twice, but also if you have e.g. a bot which syncs emotes over multiple rooms, those will +also be deduplicated. + +Lastly, in order to have consistent and expected ordering of emotes when suggesting, they should be +suggested in the following order: +1. User emoticons (emotes set to your own account) +2. Emoticon rooms (rooms whos emotes you enabled to be accessible everywhere) +3. Room emoticons (emotes defined in the currently open room) + +### Sending In places where fancy tab-complete with the emote itself is not possible it is suggested that sending the shortcode will convert to the img tag, e.g. sending `Hey there :wave:` will convert to `Hey there `. @@ -120,20 +130,17 @@ a user could specify the emote source by writing e.g. `:room~wave:` or `:user~wa pack name is used for room emotes, and "user" for user emotes. If a room pack does not have a short pack name and has a blank state key, then "room" is used. -# Ideas - - ??? - -# Current implementations -## Emote rendering (rendering of the `` tag) +## Current implementations +### Emote rendering (rendering of the `` tag) - riot-web - revolution - nheko - fluffychat -## Emote sending, using the mentioned events here +### Emote sending, using the mentioned events here - revolution - fluffychat -# Security Considerations +## Security Considerations When sending an `` tag in an end-to-end encrypted room, the client will make a request to fetch said image, in this case an emote. As there is no way to encrypt content behind `` tags yet, this could potentially leak part of the conversation. This is **not** a new security consideration, @@ -141,13 +148,13 @@ it already exists. This, however, isn't much different from posting someone a li the recipient opens the link. Additionally, images, and thus emotes, are often cached by the client, not even necessarily leading to an http query. -# Alternatives +## Alternatives One can easily think of near infinite ways to implement emotes. The aspect of sending an `` tag -and marking it as an emote with `data-mx-emote` seems to be pretty universal though, so the question +and marking it as an emote with `data-mx-emoticon` seems to be pretty universal though, so the question is mainly on different emote sources. For that, a separate MSC, [MSC1951](/~https://github.com/matrix-org/matrix-doc/pull/1951) already exists, so it is reasonable to compare this one with that one: -## Comparison with MSC1951 +### Comparison with MSC1951 MSC1951 defines a dedicated room as the only emote source. This MSC, however, also allows you to bind emotes to your own account, offering greater flexibility. In MSC1951 there can also only be a single emote pack in a room. This could be problematic in e.g. bridged rooms: You set some emotes from the matrix From 72b21746dccf45d02ed7ca85d118293ee04096be Mon Sep 17 00:00:00 2001 From: Sorunome Date: Thu, 24 Sep 2020 15:47:09 +0200 Subject: [PATCH 12/35] update stuff --- proposals/2545-emotes.md | 37 ++++++++++++++++++++++++++----------- 1 file changed, 26 insertions(+), 11 deletions(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index c103b808283..cd02753a493 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -34,15 +34,23 @@ is `im.ponies.user_emotes` (later: `m.emoticons`). The content is as following: ```json { - "short": { - ":emote:": "mxc://example.org/blah", - ":other-emote:": "mxc://example.org/other-blah" + "emoticons": { + ":emote": { + "url": "mxc://example.org/blah" + }, + ":other-emote:": { + "url": "mxc://example.org/other-blah" + } } } ``` -The emotes are defined inside of a dict called `short`, which stands for shortcode. Other, additional -keys may exist to define more metadata for the emotes. No such guide exists yet. +The emotes are defined inside of a dict called `emoticons`. The key is the shortcode of the emoticon. +The value is an object with `url` containing the mxc url of the emote. This is so that it is easier +for clients or future MSCs to define custom metadata to emotes directly. + +Some existing implementations using `im.ponies.user_emotes` currently use a dict called `short` which +is just a map of the shortcode to the mxc url. #### Room emoticons Room emotes are per-room emotes that every user of a specific room can use inside of that room. They @@ -51,9 +59,9 @@ pack, whereas the default one would be a blank state key. E.g. a discord bridge `de.sorunome.mx-puppet-bridge.discord` and have all the bridged emotes in said state event, keeping bridged emotes from matrix emotes separate. -The content extends that of the user emotes: It uses the `short` key, which is a map of the shortcode -of the emote to its mxc uri. Additionally, an optional `pack` key can be set, which defines meta-information -on the pack. The following keys for `pack` are valid: +The content extends that of the user emotes: It uses the `emoticons` key, which is a map of the shortcode +of the emote to an object containing its mxc url. Additionally, an optional `pack` key can be set, +which defines meta-information on the pack. The following keys for `pack` are valid: - `displayname`: An easy displayname for the pack. Defaults to the room name, if it doesn't exist - `avatar_url`: The mxc uri of an avatar/icon to display for the pack. Defaults to the room name, @@ -68,9 +76,13 @@ As such, a `im.ponies.room_emotes` (later: `m.emoticons`) state event could look ```json { - "short": { - ":emote:": "mxc://example.org/blah", - ":other-emote:": "mxc://example.org/other-blah" + "emoticons": { + ":emote:": { + "url": "mxc://example.org/blah" + }, + ":other-emote:": { + "url": "mxc://example.org/other-blah" + } }, "pack": { "displayname": "Emotes from Discord!", @@ -80,6 +92,9 @@ As such, a `im.ponies.room_emotes` (later: `m.emoticons`) state event could look } ``` +Similarly as before, there are already existing implementations using `short` which is a map of shortcode +to mxc url. + #### Emoticon rooms While room emotes are specific to a room and are only accessible within that room, emote rooms should be accessible from everywhere. They do not differentiate themselves from room emotes at all, instead you From 2e2987b646afcffc92f0a53eb13fad84249200c4 Mon Sep 17 00:00:00 2001 From: Sorunome Date: Wed, 30 Sep 2020 15:35:52 +0200 Subject: [PATCH 13/35] add a paragraph over emojis and stuff --- proposals/2545-emotes.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index cd02753a493..fa90557d264 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -2,6 +2,26 @@ Emoticons, or short emotes...we need them! +## Terminology +Since there is a lot of confusion of how this relates to `m.emote`, why this isn't called custom emoji +etc, there it is: + +`m.emtoe` is for emotion - and it has been incorrectly named this way. `m.action` would have been more +appropriate, as you use it to describe *actions*, not *emotions*. E.g. "/me is walking to the gym", or +"/me is happy" and *not* "/me happy". + +That, however, is *not* what this MSC is about. Instead it is about emoticons, also known in short as +emotes. + +Emoticons are just little images or text describing emotions or other things. Emoji are a subset of +emoticons, namely those found within unicode. Custom emoji here would actually refer to a custom emoji +font, that is your own rendering of 🦊, 🐱, etc., *not* new images. New images is what custom emoticons +are for. + +Now, a client may chose to name these however they like, we already have a naming disparity between +spec and clients with groups vs communities. It is, however, imperative to name things in the spec +accurately after what they are. + ## Proposal Emoticons have at least a shortcode and an mxc uri. They are sent as `` tags currently already in the spec, as such existing clients *should* be able to render them (support for this is sadly poor, From e8c2e34410e6bf58067334db359d78c4bb897b36 Mon Sep 17 00:00:00 2001 From: Sorunome Date: Sun, 25 Oct 2020 10:45:51 +0100 Subject: [PATCH 14/35] Re-word proposal to sound more firm, address lots of suggestions --- proposals/2545-emotes.md | 131 +++++++++++++++++++++------------------ 1 file changed, 69 insertions(+), 62 deletions(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index fa90557d264..1a0d0dc5ff1 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -6,7 +6,7 @@ Emoticons, or short emotes...we need them! Since there is a lot of confusion of how this relates to `m.emote`, why this isn't called custom emoji etc, there it is: -`m.emtoe` is for emotion - and it has been incorrectly named this way. `m.action` would have been more +`m.emote` is for emotion - and it has been incorrectly named this way. `m.action` would have been more appropriate, as you use it to describe *actions*, not *emotions*. E.g. "/me is walking to the gym", or "/me is happy" and *not* "/me happy". @@ -18,15 +18,15 @@ emoticons, namely those found within unicode. Custom emoji here would actually r font, that is your own rendering of 🦊, 🐱, etc., *not* new images. New images is what custom emoticons are for. -Now, a client may chose to name these however they like, we already have a naming disparity between +Now, a client may choose to name these however they like, we already have a naming disparity between spec and clients with groups vs communities. It is, however, imperative to name things in the spec accurately after what they are. ## Proposal Emoticons have at least a shortcode and an mxc uri. They are sent as `` tags currently already in -the spec, as such existing clients *should* be able to render them (support for this is sadly poor, -even within riot flavours). To allow clients to distinguish emoticons from other inline images, a new -property `data-mx-emoticon` is introduced. A client may chose to ignore the size attributes of emoticons +the spec, as such existing clients should already be able to render them, though not all clients currently +handle `img` tags. To allow clients to distinguish emoticons from other inline images, a new +property, `data-mx-emoticon`, is introduced. A client can chose to ignore the size attributes of emoticons when rendering, and instead pick the size based on other circumstances. This could e.g. be used to display emoticons in messages with only emoticons and emoji larger than usual, which is commonly found in messengers. Such an `` tag of a shortcode `:emote:` and an mxc uri `mxc://example.org/emote` @@ -38,19 +38,33 @@ could look as follows: Both the `alt` and the `title` attributes are specified as they serve different purposes: `alt` is displayed if e.g. the emote does not load. `title` is displayed e.g. on mouse hover over the emote. -The height is just a height that looks good on most devices with the normal, default font size. -No width is displayed as to not weirdly squish non-square emotes. +Thus, specifying both the `alt` and `title` attributes is required. + +The `height` is just a height that looks good on most devices with the normal, default font size. +No width is displayed as to not weirdly squish non-square emotes. In order to maintain backwards-compatibility +with clients not supporting emotes, specifying the `height` is required. + +If the new `data-mx-emoticon` attribute has a value it is ignored. Due to limitations of some libraries +the attribute may even look like `data-mx-emoticon=""`. + +The `src` attribute *must* be an mxc url. Other URIs, such as `https`, `mailto` etc. are not allowed. + +### Emoticon image types +Emoticons are recommended to have a size of about 128x128 pixels. Even though the fallback specifies +a height of 32, this is to ensure that the emotes still look good on higher DPI screens. + +Furthermore, emotes should either have a mimetype of `image/png`, `image/gif` or `image/webp`. + +Due to the low resolution of emotes, `image/jpg`/`image/jpeg` has been purposefully excluded from this +list. ### Emoticon sources -In order to be able to send emotes the client needs to have a list of shortcodes and their corresponding -mxc uris. All the emote sources described here are merely suggestions on where to get emotes from. -A client is free to implement / design their own way of fetching emotes, e.g. via dropping image files -into a folder and uploading them on-the-fly. For cross-compatibility between clients it'd still be a -good idea to implement the emote sources described here. For this there are two different emote sources: +So that emoticons are compatible between different clients, they should follow the emoticon sources +described here. #### User emoticons -User emotes are per-user emotes that are defined in the users respective account data. The type for that -is `im.ponies.user_emotes` (later: `m.emoticons`). The content is as following: +User emotes are per-user emotes that are defined in the user's respective account data. The type for that +is `m.emoticons`, the content of which is as following: ```json { @@ -66,33 +80,34 @@ is `im.ponies.user_emotes` (later: `m.emoticons`). The content is as following: ``` The emotes are defined inside of a dict called `emoticons`. The key is the shortcode of the emoticon. -The value is an object with `url` containing the mxc url of the emote. This is so that it is easier -for clients or future MSCs to define custom metadata to emotes directly. - -Some existing implementations using `im.ponies.user_emotes` currently use a dict called `short` which -is just a map of the shortcode to the mxc url. +The value is an object with `url` as a required field, containing the mxc uri as a string. This is so +that it is easier for clients or future MSCs to define custom metadata to emotes directly. #### Room emoticons -Room emotes are per-room emotes that every user of a specific room can use inside of that room. They -are set with a state event of type `im.ponies.room_emotes` (later: `m.emoticons`). The state key denotes a possible -pack, whereas the default one would be a blank state key. E.g. a discord bridge could set as state key +Room emotes are per-room emotes that every user of a specific room can only use inside of that room. They +are set with a state event of type `m.emoticons`. Multiple packs can be specified by different state +keys, the identifier being an opague string. An empty state key is the default pack for a room. +E.g. a discord bridge could set as state key `de.sorunome.mx-puppet-bridge.discord` and have all the bridged emotes in said state event, keeping bridged emotes from matrix emotes separate. The content extends that of the user emotes: It uses the `emoticons` key, which is a map of the shortcode -of the emote to an object containing its mxc url. Additionally, an optional `pack` key can be set, +of the emote to an object containing its mxc url, just as with user emoticons. Additionally, an optional `pack` key can be set, which defines meta-information on the pack. The following keys for `pack` are valid: - - `displayname`: An easy displayname for the pack. Defaults to the room name, if it doesn't exist - - `avatar_url`: The mxc uri of an avatar/icon to display for the pack. Defaults to the room name, - if it doesn't exist. - - `name`: A short identifier of the pack. Defaults to the normalized state key, and if the state - key is blank it defaults to "room". + - `display_name`: (String, optional) An easy display name for the pack. Defaults to the room name, + if it doesn't exist. This does not have to be unique within all packs of a room. + - `avatar_url`: (String, optional) The mxc uri of an avatar/icon to display for the pack. Defaults + to the room avatar, if it doesn't exist. If the room also does not have one, then this pack does + not have an avatar. + - `short`: (String, `^[a-z0-9-_]+$`, optional) A short human-readable identifier of the pack. Defaults + to the normalized state key, and if the state key is blank it defaults to "room". This is used to + easily specify which pack to pick an emoticon from, should there be clashes. Normalized means here, converting spaces to `-`, taking only alphanumerical characters, `-` and `_`, - and casting it all to lowercase. In regex, this would be `[a-z0-9-_]+`. + and casting it all to lowercase. In regex, this would be `^[a-z0-9-_]+$`. -As such, a `im.ponies.room_emotes` (later: `m.emoticons`) state event could look like the following: +As such, a `m.emoticons` state event could look like the following: ```json { @@ -105,23 +120,20 @@ As such, a `im.ponies.room_emotes` (later: `m.emoticons`) state event could look } }, "pack": { - "displayname": "Emotes from Discord!", + "display_name": "Emotes from Discord!", "avatar_url": "mxc://example.org/discord_guild", - "name": "some_discord_guild" + "short": "some_discord_guild" } } ``` -Similarly as before, there are already existing implementations using `short` which is a map of shortcode -to mxc url. - #### Emoticon rooms While room emotes are specific to a room and are only accessible within that room, emote rooms should be accessible from everywhere. They do not differentiate themselves from room emotes at all, instead you -set an event in your account data of type `im.ponies.emote_rooms` (later: `m.emoticons.rooms`) which outlines -which room emote states should be globally accessible for that user. For that, a `room` key contains +set an event in your account data of type `m.emoticons.rooms` which outlines +which room emote states are globally accessible for that user. For that, a `room` key contains a map of room ids that map to state keys that map to an optional pack definition override. -The the contents of `im.ponies.emote_rooms` (later: `m.emoticons.rooms`) could look like the following: +The the contents of `m.emoticons.rooms` could look like the following: ```json { @@ -129,7 +141,7 @@ The the contents of `im.ponies.emote_rooms` (later: `m.emoticons.rooms`) could l "!someroom:example.org": { "": {}, "de.sorunome.mx-puppet-bridge.discord": { - "displayname": "Overriden name", + "display_name": "Overriden name", "short": "new_short_name" } }, @@ -142,7 +154,9 @@ The the contents of `im.ponies.emote_rooms` (later: `m.emoticons.rooms`) could l Here three emote packs are globally accessible to the user: Two defined in `!someroom:example.org` (one with blank state key and one with state key `de.sorunome.mx-puppet-bridge.discord`) and one in -`!someotherroom:example.org`. +`!someotherroom:example.org`. The one in `!someroom:example.org` with state key `de.sorunome.mx-puppet-bridge.discord` +has an override for `display_name` and `short` of that pack, so that for this user the pack displays +differently for themselves. ### Emoticon source priority and deduplicating When giving emoticon suggestions, clients are expected to deduplicate emotes by their mxc url. This @@ -157,32 +171,28 @@ suggested in the following order: 3. Room emoticons (emotes defined in the currently open room) ### Sending -In places where fancy tab-complete with the emote itself is not possible it is suggested that sending -the shortcode will convert to the img tag, e.g. sending `Hey there :wave:` will convert to `Hey there `. - -If there are collisions due to the same emote shortcode being present as both room emote and user emote -a user could specify the emote source by writing e.g. `:room~wave:` or `:user~wave:`. Here the short -pack name is used for room emotes, and "user" for user emotes. If a room pack does not have a short -pack name and has a blank state key, then "room" is used. - -## Current implementations -### Emote rendering (rendering of the `` tag) - - riot-web - - revolution - - nheko - - fluffychat -### Emote sending, using the mentioned events here - - revolution - - fluffychat +Clients should consider converting shortcodes like `:wave:` to a relevant `` tag for the emoticon. + +Clients should also consider supporting ways for the user to find the emoticon they are attempting to send, +such as having a syntax like `:room~wave:` to find `wave` emoticon in the current room. Alternatives might also +be `user` or pack `short` attributes to search within. ## Security Considerations -When sending an `` tag in an end-to-end encrypted room, the client will make a request to fetch +When sending an `` tag in an encrypted room, the client will make a request to fetch said image, in this case an emote. As there is no way to encrypt content behind `` tags yet, this could potentially leak part of the conversation. This is **not** a new security consideration, it already exists. This, however, isn't much different from posting someone a link in an e2ee chat and the recipient opens the link. Additionally, images, and thus emotes, are often cached by the client, not even necessarily leading to an http query. +## Unstable prefix +The `m.emoticons` in the account data is replaced with `im.ponies.user_emotes`. The `m.emoticons` in +the room state is replaced with `im.ponies.room_emotes`. The `m.emoticons.rooms` is replaced with +`im.ponies.room_emotes`. + +Some existing implementations using `im.ponies.user_emotes` and `im.ponies.room_emotes` currently use +a dict called `short` which is just a map of the shortcode to the mxc url. + ## Alternatives One can easily think of near infinite ways to implement emotes. The aspect of sending an `` tag and marking it as an emote with `data-mx-emoticon` seems to be pretty universal though, so the question @@ -204,8 +214,5 @@ set one state event *per emote*. While this might seem like a nice idea on the s scale well. There are people who easily use and want hundreds or even thousands of emotes accessible. A simple dict of shortcode to mxc URI seems more appropriate for this. -Additionally, this MSC has already been used in the wild for roughly two years by some clients, while MSC1951 -only exists in MSC form. -In general, MSC1951 feels like a heavier approach to emote sourcs, while this MSC is more lightweight -and thus should allow for significantly larger packs. +In general, MSC1951 feels like a heavier approach to emote sources, while this MSC is more lightweight. From b41c091dce3c29b3ade749f18d3350597a567512 Mon Sep 17 00:00:00 2001 From: Sorunome Date: Sun, 25 Oct 2020 10:54:44 +0100 Subject: [PATCH 15/35] specify that emotes can be animated --- proposals/2545-emotes.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index 1a0d0dc5ff1..2fc103364af 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -53,7 +53,8 @@ The `src` attribute *must* be an mxc url. Other URIs, such as `https`, `mailto` Emoticons are recommended to have a size of about 128x128 pixels. Even though the fallback specifies a height of 32, this is to ensure that the emotes still look good on higher DPI screens. -Furthermore, emotes should either have a mimetype of `image/png`, `image/gif` or `image/webp`. +Furthermore, emotes should either have a mimetype of `image/png`, `image/gif` or `image/webp`. Emotes +can be animated. Due to the low resolution of emotes, `image/jpg`/`image/jpeg` has been purposefully excluded from this list. From 4f6d14724a0440ae060aa84727792e3acc7a3f9f Mon Sep 17 00:00:00 2001 From: Sorunome Date: Thu, 25 Mar 2021 13:41:04 +0100 Subject: [PATCH 16/35] update msc to also include stickers and thelike --- proposals/2545-emotes.md | 196 +++++++++++++++++++-------------------- 1 file changed, 97 insertions(+), 99 deletions(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index 2fc103364af..0478d90e10b 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -1,7 +1,9 @@ -# MSC2545: Emoticons +# MSC2545: Image Packs (Emoticons & Stickers) Emoticons, or short emotes...we need them! +We also need proper stickers, too! + ## Terminology Since there is a lot of confusion of how this relates to `m.emote`, why this isn't called custom emoji etc, there it is: @@ -23,13 +25,14 @@ spec and clients with groups vs communities. It is, however, imperative to name accurately after what they are. ## Proposal +### Emoticons in the formatted body Emoticons have at least a shortcode and an mxc uri. They are sent as `` tags currently already in the spec, as such existing clients should already be able to render them, though not all clients currently handle `img` tags. To allow clients to distinguish emoticons from other inline images, a new property, `data-mx-emoticon`, is introduced. A client can chose to ignore the size attributes of emoticons when rendering, and instead pick the size based on other circumstances. This could e.g. be used to display emoticons in messages with only emoticons and emoji larger than usual, which is commonly found in -messengers. Such an `` tag of a shortcode `:emote:` and an mxc uri `mxc://example.org/emote` +messengers. Such an `` tag of a shortcode `emote` and an mxc uri `mxc://example.org/emote` could look as follows: ```html @@ -49,102 +52,98 @@ the attribute may even look like `data-mx-emoticon=""`. The `src` attribute *must* be an mxc url. Other URIs, such as `https`, `mailto` etc. are not allowed. -### Emoticon image types +### Sending stickers +To send stickers the already speced `m.sticker` is used. + +### Image types Emoticons are recommended to have a size of about 128x128 pixels. Even though the fallback specifies a height of 32, this is to ensure that the emotes still look good on higher DPI screens. -Furthermore, emotes should either have a mimetype of `image/png`, `image/gif` or `image/webp`. Emotes -can be animated. +Stickers are recommended to have a size of about 512x512 pixels. + +Furthermore, these images should either have a mimetype of `image/png`, `image/gif` or `image/webp`. +They can be animated. Due to the low resolution of emotes, `image/jpg`/`image/jpeg` has been purposefully excluded from this list. -### Emoticon sources -So that emoticons are compatible between different clients, they should follow the emoticon sources -described here. - -#### User emoticons -User emotes are per-user emotes that are defined in the user's respective account data. The type for that -is `m.emoticons`, the content of which is as following: - +### Image pack event +The image pack event has a type of `m.image_pack`. It contains a key `images`, which is a map from a +short code to an image object. It may also contain a key `pack`, which is a pack object. + +#### Pack object +The pack object consists of the following keys: + - `display_name`: (String, optional) A display name for the pack. Defaults to the room name, if the + image pack event is in the room. This does not have to be unique within all packs of a room. + - `avatar_url`: (String, optional) The mxc uri of an avatar/icon to dipslay for the pack. Defautls + to the room avatar, if the pack is in the room. If the room also does not have an avatar, or the + image pack event is not in a room, this pack does not have an avatar. + - `usage`: (String[], optional) An array of the usages for this pack. Possible usages are `emoticon` + and `sticker`. If the usage is absent, a usage for all possible usage types is to be assumed. + - `license`: (String, optional) The license of this pack. + +#### Image object +The image object conists of the following keys: + - `url`: (String, requried) The mxc URL for this image + - `body`: (String, optional) An optional body for this image, useful for the sticker body text or + the emote alt text. Defaults to the short code. + - `info`: (`ImageInfo`, optional) The already speced `ImageInfo` object used for the `info` block of + `m.sticker` events. + - `usage`: (String[], optional) An array of the usages for this image. If present, this overrides + the usage defined at pack level for this particular image. + +#### Example image pack event +Taking all this into account, an example pack event may look as following: ```json { - "emoticons": { - ":emote": { + "images": { + "emote": { "url": "mxc://example.org/blah" }, - ":other-emote:": { - "url": "mxc://example.org/other-blah" + "sicker": { + "url": "mxc://example.og/sticker", + "usage": ["sticker"] } + }, + "pack": { + "display_name": "Awesome Pack", + "usage": ["emoticon"] } } ``` -The emotes are defined inside of a dict called `emoticons`. The key is the shortcode of the emoticon. -The value is an object with `url` as a required field, containing the mxc uri as a string. This is so -that it is easier for clients or future MSCs to define custom metadata to emotes directly. +### Image sources +There are several places where a client is expected to look for these `m.image_pack` events, mainly +in their own account data and in room states. -#### Room emoticons -Room emotes are per-room emotes that every user of a specific room can only use inside of that room. They -are set with a state event of type `m.emoticons`. Multiple packs can be specified by different state -keys, the identifier being an opague string. An empty state key is the default pack for a room. +#### User image packs +Each user can have their own personal image pack defined in their own account data, with the normal +`m.image_pack` event. The user is expected to be presented with these images in all rooms. + +#### Room image packs +A room can have an unlimited amount of image packs, by specifying the `m.image_pack` state event with +different state keys. The user is expected to be presented with these images only in the room they +are defined in. To enable them to be presented in all rooms, see the section below. +An empty state key is the default pack for a room. E.g. a discord bridge could set as state key `de.sorunome.mx-puppet-bridge.discord` and have all the bridged emotes in said state event, keeping bridged emotes from matrix emotes separate. -The content extends that of the user emotes: It uses the `emoticons` key, which is a map of the shortcode -of the emote to an object containing its mxc url, just as with user emoticons. Additionally, an optional `pack` key can be set, -which defines meta-information on the pack. The following keys for `pack` are valid: - - - `display_name`: (String, optional) An easy display name for the pack. Defaults to the room name, - if it doesn't exist. This does not have to be unique within all packs of a room. - - `avatar_url`: (String, optional) The mxc uri of an avatar/icon to display for the pack. Defaults - to the room avatar, if it doesn't exist. If the room also does not have one, then this pack does - not have an avatar. - - `short`: (String, `^[a-z0-9-_]+$`, optional) A short human-readable identifier of the pack. Defaults - to the normalized state key, and if the state key is blank it defaults to "room". This is used to - easily specify which pack to pick an emoticon from, should there be clashes. - - Normalized means here, converting spaces to `-`, taking only alphanumerical characters, `-` and `_`, - and casting it all to lowercase. In regex, this would be `^[a-z0-9-_]+$`. - -As such, a `m.emoticons` state event could look like the following: - -```json -{ - "emoticons": { - ":emote:": { - "url": "mxc://example.org/blah" - }, - ":other-emote:": { - "url": "mxc://example.org/other-blah" - } - }, - "pack": { - "display_name": "Emotes from Discord!", - "avatar_url": "mxc://example.org/discord_guild", - "short": "some_discord_guild" - } -} -``` - -#### Emoticon rooms -While room emotes are specific to a room and are only accessible within that room, emote rooms should -be accessible from everywhere. They do not differentiate themselves from room emotes at all, instead you -set an event in your account data of type `m.emoticons.rooms` which outlines -which room emote states are globally accessible for that user. For that, a `room` key contains -a map of room ids that map to state keys that map to an optional pack definition override. -The the contents of `m.emoticons.rooms` could look like the following: +#### Image pack rooms +While room image packs are specific to a room and are only accessible within that room, image pack +rooms should be accessible from everywhere. They do not differentiate themselves from room emotes at +all, instead you set an event in your account data of type `m.image_pack.rooms` which outlines +which room image pack states are globally accessible for that user. For that, a `room` key contains +a map of room ids that map to state keys that map to an object. While this MSC does not define any +contents for this object, having this an object means greater flexibility in case of future changes. +The the contents of `m.image_pack.rooms` could look like the following: ```json { "rooms": { "!someroom:example.org": { "": {}, - "de.sorunome.mx-puppet-bridge.discord": { - "display_name": "Overriden name", - "short": "new_short_name" - } + "de.sorunome.mx-puppet-bridge.discord": {} }, "!someotherroom:example.org": { "": {} @@ -155,28 +154,25 @@ The the contents of `m.emoticons.rooms` could look like the following: Here three emote packs are globally accessible to the user: Two defined in `!someroom:example.org` (one with blank state key and one with state key `de.sorunome.mx-puppet-bridge.discord`) and one in -`!someotherroom:example.org`. The one in `!someroom:example.org` with state key `de.sorunome.mx-puppet-bridge.discord` -has an override for `display_name` and `short` of that pack, so that for this user the pack displays -differently for themselves. - -### Emoticon source priority and deduplicating -When giving emoticon suggestions, clients are expected to deduplicate emotes by their mxc url. This -not only ensures that, when viewing a room that you also have in `m.emoticons.rooms`, it won't be -displayed twice, but also if you have e.g. a bot which syncs emotes over multiple rooms, those will -also be deduplicated. - -Lastly, in order to have consistent and expected ordering of emotes when suggesting, they should be -suggested in the following order: -1. User emoticons (emotes set to your own account) -2. Emoticon rooms (rooms whos emotes you enabled to be accessible everywhere) -3. Room emoticons (emotes defined in the currently open room) +`!someotherroom:example.org`. -### Sending -Clients should consider converting shortcodes like `:wave:` to a relevant `` tag for the emoticon. +### Image pack source priority and deduplicating +If a client gives image suggestions (emotes, stickers) to the user in some ordered fassion (e.g. a +ordered list where you click an entry), the order of the images should be predictable between rooms. +The ordering could look as following: +1. User image pack (images set in your own account) +2. Image pack rooms (rooms whos image packs you enabled to be accessible everywhere) +3. Room image packs (images defined in the currently open room) -Clients should also consider supporting ways for the user to find the emoticon they are attempting to send, -such as having a syntax like `:room~wave:` to find `wave` emoticon in the current room. Alternatives might also -be `user` or pack `short` attributes to search within. +Furthermore, clients are expected to deduplicate images based on their mxc url. This not only ensures +that, when viewing a room that you also have in `m.image_pack.rooms`, it won't be displayed twice, +but also if you have e.g. a bot which syncs emotes over multiple rooms, those will also be deduplicated. + +### Sending +For emoticons a client could add deliminators (e.g. `:`) around around the image shortcode, meaning +that if an image has a shortcode of `emote`, the user can enter `:emote:` to send it. If there are +multiple emoticons with the same shortcode in a room the client could e.g. slugify the packs display +name and then have the user enter `:slug~emote:`. ## Security Considerations When sending an `` tag in an encrypted room, the client will make a request to fetch @@ -187,13 +183,15 @@ the recipient opens the link. Additionally, images, and thus emotes, are often c not even necessarily leading to an http query. ## Unstable prefix -The `m.emoticons` in the account data is replaced with `im.ponies.user_emotes`. The `m.emoticons` in -the room state is replaced with `im.ponies.room_emotes`. The `m.emoticons.rooms` is replaced with +The `m.image_pack` in the account data is replaced with `im.ponies.user_emotes`. The `m.image_pack` in +the room state is replaced with `im.ponies.room_emotes`. The `m.image_pack.rooms` is replaced with `im.ponies.room_emotes`. Some existing implementations using `im.ponies.user_emotes` and `im.ponies.room_emotes` currently use a dict called `short` which is just a map of the shortcode to the mxc url. +Some other implementations currently also call the `images` key `emoticons`. + ## Alternatives One can easily think of near infinite ways to implement emotes. The aspect of sending an `` tag and marking it as an emote with `data-mx-emoticon` seems to be pretty universal though, so the question @@ -201,17 +199,17 @@ is mainly on different emote sources. For that, a separate MSC, [MSC1951](https: already exists, so it is reasonable to compare this one with that one: ### Comparison with MSC1951 -MSC1951 defines a dedicated room as the only emote source. This MSC, however, also allows you to bind emotes -to your own account, offering greater flexibility. In MSC1951 there can also only be a single emote -pack in a room. This could be problematic in e.g. bridged rooms: You set some emotes from the matrix -side and a discord bridge would plop all the discord emotes in another pack in the same room. +MSC1951 defines a dedicated room as the only image pack source. This MSC, however, also allows you to bind image packs +to your own account, offering greater flexibility. In MSC1951 there can also only be a single image pack +pack in a room. This could be problematic in e.g. bridged rooms: You set some emotes or stickers from the matrix +side and a discord bridge would plop all the discord emotes and stickers in another pack in the same room. MSC1951 defines a way to recommend using a pack of a different room - this MSC does not have an equivalent -to that. Instead, this MSC allows multiple emote packs for a room, and allows you to enable an emote +to that. Instead, this MSC allows multiple image packs for a room, and allows you to enable an image pack to be globally available for yourself across all rooms you are in. The core difference is how MSC1951 and this MSC define the emotes themselves: In MSC1951 you have to -set one state event *per emote*. While this might seem like a nice idea on the surface, it doesn't +set one state event *per image*. While this might seem like a nice idea on the surface, it doesn't scale well. There are people who easily use and want hundreds or even thousands of emotes accessible. A simple dict of shortcode to mxc URI seems more appropriate for this. From 5351cb336f24cdb2d7ac613f958e77fde3d467a3 Mon Sep 17 00:00:00 2001 From: Sorunome Date: Thu, 25 Mar 2021 13:55:12 +0100 Subject: [PATCH 17/35] update a few things --- proposals/2545-emotes.md | 31 +++++++++++++++++++++++++------ 1 file changed, 25 insertions(+), 6 deletions(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index 0478d90e10b..bf6155e544d 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -5,6 +5,7 @@ Emoticons, or short emotes...we need them! We also need proper stickers, too! ## Terminology +### Emoticons Since there is a lot of confusion of how this relates to `m.emote`, why this isn't called custom emoji etc, there it is: @@ -24,6 +25,11 @@ Now, a client may choose to name these however they like, we already have a nami spec and clients with groups vs communities. It is, however, imperative to name things in the spec accurately after what they are. +### Stickers +Stickers already exits in Matrix. They are reusable images one can send, usually as a reaction sent +in the timeline to something. This MSC adds a way to distribute and define a source for a client to +send them. + ## Proposal ### Emoticons in the formatted body Emoticons have at least a shortcode and an mxc uri. They are sent as `` tags currently already in @@ -59,7 +65,7 @@ To send stickers the already speced `m.sticker` is used. Emoticons are recommended to have a size of about 128x128 pixels. Even though the fallback specifies a height of 32, this is to ensure that the emotes still look good on higher DPI screens. -Stickers are recommended to have a size of about 512x512 pixels. +Stickers are recommended to have a size of up to 512x512 pixels. Furthermore, these images should either have a mimetype of `image/png`, `image/gif` or `image/webp`. They can be animated. @@ -79,7 +85,7 @@ The pack object consists of the following keys: to the room avatar, if the pack is in the room. If the room also does not have an avatar, or the image pack event is not in a room, this pack does not have an avatar. - `usage`: (String[], optional) An array of the usages for this pack. Possible usages are `emoticon` - and `sticker`. If the usage is absent, a usage for all possible usage types is to be assumed. + and `sticker`. If the usage is absent or empty, a usage for all possible usage types is to be assumed. - `license`: (String, optional) The license of this pack. #### Image object @@ -89,8 +95,8 @@ The image object conists of the following keys: the emote alt text. Defaults to the short code. - `info`: (`ImageInfo`, optional) The already speced `ImageInfo` object used for the `info` block of `m.sticker` events. - - `usage`: (String[], optional) An array of the usages for this image. If present, this overrides - the usage defined at pack level for this particular image. + - `usage`: (String[], optional) An array of the usages for this image. If present and non-emtpy, + this overrides the usage defined at pack level for this particular image. #### Example image pack event Taking all this into account, an example pack event may look as following: @@ -100,8 +106,8 @@ Taking all this into account, an example pack event may look as following: "emote": { "url": "mxc://example.org/blah" }, - "sicker": { - "url": "mxc://example.og/sticker", + "sticker": { + "url": "mxc://example.org/sticker", "usage": ["sticker"] } }, @@ -169,11 +175,22 @@ that, when viewing a room that you also have in `m.image_pack.rooms`, it won't b but also if you have e.g. a bot which syncs emotes over multiple rooms, those will also be deduplicated. ### Sending +#### Emoticons For emoticons a client could add deliminators (e.g. `:`) around around the image shortcode, meaning that if an image has a shortcode of `emote`, the user can enter `:emote:` to send it. If there are multiple emoticons with the same shortcode in a room the client could e.g. slugify the packs display name and then have the user enter `:slug~emote:`. +The alt / title text fo the `` tag is expected to be the `body` of the emote, or, if absent, its +shotcode, optionally with tacked on deliminators. + +#### Stickers +When sending a sticker the `body` of the `m.sticker` event should be set to the `body` defined for that +image, or its shortcode, if absent. + +Furthermore, the `info` of the `m.sticker` event should be set to the `info` defined for that image, +or a blank object, if absent. + ## Security Considerations When sending an `` tag in an encrypted room, the client will make a request to fetch said image, in this case an emote. As there is no way to encrypt content behind `` tags yet, @@ -182,6 +199,8 @@ it already exists. This, however, isn't much different from posting someone a li the recipient opens the link. Additionally, images, and thus emotes, are often cached by the client, not even necessarily leading to an http query. +Related issue: /~https://github.com/matrix-org/matrix-doc/issues/2418 + ## Unstable prefix The `m.image_pack` in the account data is replaced with `im.ponies.user_emotes`. The `m.image_pack` in the room state is replaced with `im.ponies.room_emotes`. The `m.image_pack.rooms` is replaced with From 12c8fa8271d869863c9b67427ae9609920f6b74e Mon Sep 17 00:00:00 2001 From: Sorunome Date: Thu, 25 Mar 2021 14:05:44 +0100 Subject: [PATCH 18/35] add a few more words on slugs --- proposals/2545-emotes.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index bf6155e544d..187b3d90893 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -179,7 +179,8 @@ but also if you have e.g. a bot which syncs emotes over multiple rooms, those wi For emoticons a client could add deliminators (e.g. `:`) around around the image shortcode, meaning that if an image has a shortcode of `emote`, the user can enter `:emote:` to send it. If there are multiple emoticons with the same shortcode in a room the client could e.g. slugify the packs display -name and then have the user enter `:slug~emote:`. +name and then have the user enter `:slug~emote:`. As slugs typically match `^[\w-]+$` that should +ensure complete-ability. The alt / title text fo the `` tag is expected to be the `body` of the emote, or, if absent, its shotcode, optionally with tacked on deliminators. From e8d910f107b70588ffc7c21b5577c893e4dff9e8 Mon Sep 17 00:00:00 2001 From: Sorunome Date: Fri, 26 Mar 2021 11:36:59 +0100 Subject: [PATCH 19/35] Update proposals/2545-emotes.md Co-authored-by: DeepBlueV7.X --- proposals/2545-emotes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index 187b3d90893..36d43a08f5e 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -205,7 +205,7 @@ Related issue: /~https://github.com/matrix-org/matrix-doc/issues/2418 ## Unstable prefix The `m.image_pack` in the account data is replaced with `im.ponies.user_emotes`. The `m.image_pack` in the room state is replaced with `im.ponies.room_emotes`. The `m.image_pack.rooms` is replaced with -`im.ponies.room_emotes`. +`im.ponies.emote_rooms`. Some existing implementations using `im.ponies.user_emotes` and `im.ponies.room_emotes` currently use a dict called `short` which is just a map of the shortcode to the mxc url. From a879bcd06cc06edca648bc1f7d49eea062f6563b Mon Sep 17 00:00:00 2001 From: Sorunome Date: Sun, 4 Jul 2021 12:22:19 +0200 Subject: [PATCH 20/35] license --> attribution --- proposals/2545-emotes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index 187b3d90893..284bcf812ae 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -86,7 +86,7 @@ The pack object consists of the following keys: image pack event is not in a room, this pack does not have an avatar. - `usage`: (String[], optional) An array of the usages for this pack. Possible usages are `emoticon` and `sticker`. If the usage is absent or empty, a usage for all possible usage types is to be assumed. - - `license`: (String, optional) The license of this pack. + - `attribution`: (String, optional) The attribution of this pack. #### Image object The image object conists of the following keys: From 580a836594570ce0daa8b9274a1f9aa4648cb76f Mon Sep 17 00:00:00 2001 From: Sorunome Date: Sun, 18 Jul 2021 19:37:57 +0200 Subject: [PATCH 21/35] Update proposals/2545-emotes.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Šimon Brandner --- proposals/2545-emotes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index 36d43a08f5e..da2ab98fafa 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -26,7 +26,7 @@ spec and clients with groups vs communities. It is, however, imperative to name accurately after what they are. ### Stickers -Stickers already exits in Matrix. They are reusable images one can send, usually as a reaction sent +Stickers already exist in Matrix. They are reusable images one can send, usually as a reaction sent in the timeline to something. This MSC adds a way to distribute and define a source for a client to send them. From 739b4c018e8625d2329ad0556f2b99067215661d Mon Sep 17 00:00:00 2001 From: Sorunome Date: Sun, 18 Jul 2021 19:57:45 +0200 Subject: [PATCH 22/35] Update with "looking further" section and slightly update comparison --- proposals/2545-emotes.md | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index b32a5f12675..36aa1f2e127 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -224,14 +224,25 @@ to your own account, offering greater flexibility. In MSC1951 there can also onl pack in a room. This could be problematic in e.g. bridged rooms: You set some emotes or stickers from the matrix side and a discord bridge would plop all the discord emotes and stickers in another pack in the same room. +The original sharing-focused idea of MSC1951 is still preserved: Once room types are a thing, you could +still easily have an image pack-only room. + MSC1951 defines a way to recommend using a pack of a different room - this MSC does not have an equivalent -to that. Instead, this MSC allows multiple image packs for a room, and allows you to enable an image -pack to be globally available for yourself across all rooms you are in. +to that. Instead, this MSC allows multiple image packs for a room, typically one where you already +chat in anyways. Furthermore it allows you to enable an image pack to be globally available for yourself +across all rooms you are in. -The core difference is how MSC1951 and this MSC define the emotes themselves: In MSC1951 you have to +The core difference is how MSC1951 and this MSC define the image packs themselves: In MSC1951 you have to set one state event *per image*. While this might seem like a nice idea on the surface, it doesn't -scale well. There are people who easily use and want hundreds or even thousands of emotes accessible. +scale well. There are people who easily use and want hundreds or even thousands of image packs accessible. A simple dict of shortcode to mxc URI seems more appropriate for this. +In general, MSC1951 feels like a heavier approach to image pack sources, while this MSC is more lightweight. -In general, MSC1951 feels like a heavier approach to emote sources, while this MSC is more lightweight. +## Looking further +A couple of interesting points have been raised in the discussions of this MSC tangentially touch +custom emoticons but warrant an MSC for themselves, as they touch more on how `` is working. + - Figuring out how `` should work with encrypted media. + - Allow SVGs in the `` tag. Current problem: Clients typically try to thumbnail the mxc URL, + and most media repositories can't thumbnail SVGs. Possible solution: Somehow embed the mimetype. + - For stickers: Recommend rendering sizes / resolutions From 51bcec2d6f0723351a368aed3624b864d5f79f49 Mon Sep 17 00:00:00 2001 From: Sorunome Date: Sat, 24 Jul 2021 16:59:45 +0200 Subject: [PATCH 23/35] add a sentance to suggest image packs from spaces --- proposals/2545-emotes.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index 36aa1f2e127..18e6becb199 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -168,7 +168,9 @@ ordered list where you click an entry), the order of the images should be predic The ordering could look as following: 1. User image pack (images set in your own account) 2. Image pack rooms (rooms whos image packs you enabled to be accessible everywhere) -3. Room image packs (images defined in the currently open room) +3. Space image packs (image packs defined in claimed parents (`m.space.parent`), recursively. Important: + clients must make sure to break recursion loops), if the user is in them +4. Room image packs (images defined in the currently open room) Furthermore, clients are expected to deduplicate images based on their mxc url. This not only ensures that, when viewing a room that you also have in `m.image_pack.rooms`, it won't be displayed twice, From 71a3c5561d2941d295cb38b908bac96742d9d794 Mon Sep 17 00:00:00 2001 From: Sorunome Date: Sat, 24 Jul 2021 17:04:18 +0200 Subject: [PATCH 24/35] soru bamboozled herself! --- proposals/2545-emotes.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index 18e6becb199..90850dba4ed 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -135,6 +135,11 @@ E.g. a discord bridge could set as state key `de.sorunome.mx-puppet-bridge.discord` and have all the bridged emotes in said state event, keeping bridged emotes from matrix emotes separate. +#### Space image packs +Clients should suggest image packs of a space that a room is in, if the user is also in the space. +For that, the client should recursively check the `m.space.parent` state events and suggest emoticons +and stickers from all the image packs found in those rooms/spaces. + #### Image pack rooms While room image packs are specific to a room and are only accessible within that room, image pack rooms should be accessible from everywhere. They do not differentiate themselves from room emotes at @@ -168,8 +173,7 @@ ordered list where you click an entry), the order of the images should be predic The ordering could look as following: 1. User image pack (images set in your own account) 2. Image pack rooms (rooms whos image packs you enabled to be accessible everywhere) -3. Space image packs (image packs defined in claimed parents (`m.space.parent`), recursively. Important: - clients must make sure to break recursion loops), if the user is in them +3. Space image packs (packs sent in the space, if present) 4. Room image packs (images defined in the currently open room) Furthermore, clients are expected to deduplicate images based on their mxc url. This not only ensures From a7a4a3664dfa8a118c6a6400ab02775336e80bfd Mon Sep 17 00:00:00 2001 From: Sorunome Date: Thu, 16 Sep 2021 15:31:55 +0200 Subject: [PATCH 25/35] Fix small wording and ensure that emotes from canonical spaces are to be suggested --- proposals/2545-emotes.md | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index 90850dba4ed..f0ea837ac98 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -95,8 +95,11 @@ The image object conists of the following keys: the emote alt text. Defaults to the short code. - `info`: (`ImageInfo`, optional) The already speced `ImageInfo` object used for the `info` block of `m.sticker` events. - - `usage`: (String[], optional) An array of the usages for this image. If present and non-emtpy, - this overrides the usage defined at pack level for this particular image. + - `usage`: (String[], optional) An array of the usages for this image. If present and non-empty, + this overrides the usage defined at pack level for this particular image. This is useful to e.g. + have one pack contain mixed emotes and stickers. Additionally there is only a single account data + level image pack, meaning this is required to have a mixture of emotes and stickers available in + account data. #### Example image pack event Taking all this into account, an example pack event may look as following: @@ -136,9 +139,9 @@ E.g. a discord bridge could set as state key bridged emotes from matrix emotes separate. #### Space image packs -Clients should suggest image packs of a space that a room is in, if the user is also in the space. -For that, the client should recursively check the `m.space.parent` state events and suggest emoticons -and stickers from all the image packs found in those rooms/spaces. +Clients should suggest image packs of a canonical space that a room is in, if the user is also in the +space. This should be done recursively on canonical spaces. So, if a room has a canonical space and +that space again has a canonical space, the clients should suggest image packs of both of those spaces. #### Image pack rooms While room image packs are specific to a room and are only accessible within that room, image pack @@ -147,7 +150,7 @@ all, instead you set an event in your account data of type `m.image_pack.rooms` which room image pack states are globally accessible for that user. For that, a `room` key contains a map of room ids that map to state keys that map to an object. While this MSC does not define any contents for this object, having this an object means greater flexibility in case of future changes. -The the contents of `m.image_pack.rooms` could look like the following: +The contents of `m.image_pack.rooms` could look like the following: ```json { @@ -227,7 +230,7 @@ already exists, so it is reasonable to compare this one with that one: ### Comparison with MSC1951 MSC1951 defines a dedicated room as the only image pack source. This MSC, however, also allows you to bind image packs to your own account, offering greater flexibility. In MSC1951 there can also only be a single image pack -pack in a room. This could be problematic in e.g. bridged rooms: You set some emotes or stickers from the matrix +in a room. This could be problematic in e.g. bridged rooms: You set some emotes or stickers from the matrix side and a discord bridge would plop all the discord emotes and stickers in another pack in the same room. The original sharing-focused idea of MSC1951 is still preserved: Once room types are a thing, you could From be7d0df38078b83c7f77829879d2a37dd8faa82e Mon Sep 17 00:00:00 2001 From: Sorunome <+@sorunome.de> Date: Fri, 28 Oct 2022 16:36:58 +0200 Subject: [PATCH 26/35] Update spelling mistakes and wording, only suggest deduplication method now --- proposals/2545-emotes.md | 160 +++++++++++++++++++++------------------ 1 file changed, 86 insertions(+), 74 deletions(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index f0ea837ac98..024004a54aa 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -6,14 +6,14 @@ We also need proper stickers, too! ## Terminology ### Emoticons -Since there is a lot of confusion of how this relates to `m.emote`, why this isn't called custom emoji -etc, there it is: +Since there is a lot of confusion of how this relates to `m.emote`, why this isn't called "custom emoji" +etc,: `m.emote` is for emotion - and it has been incorrectly named this way. `m.action` would have been more appropriate, as you use it to describe *actions*, not *emotions*. E.g. "/me is walking to the gym", or "/me is happy" and *not* "/me happy". -That, however, is *not* what this MSC is about. Instead it is about emoticons, also known in short as +That, however, is *not* what this MSC is about. Instead, it is about emoticons, also known in short as emotes. Emoticons are just little images or text describing emotions or other things. Emoji are a subset of @@ -21,24 +21,24 @@ emoticons, namely those found within unicode. Custom emoji here would actually r font, that is your own rendering of 🦊, 🐱, etc., *not* new images. New images is what custom emoticons are for. -Now, a client may choose to name these however they like, we already have a naming disparity between +Now, a client may choose to name these however they like. We already have a naming disparity between spec and clients with groups vs communities. It is, however, imperative to name things in the spec accurately after what they are. ### Stickers -Stickers already exist in Matrix. They are reusable images one can send, usually as a reaction sent -in the timeline to something. This MSC adds a way to distribute and define a source for a client to +Stickers already exist in Matrix. They are reusable images one can send, usually as a reaction to +something sent in the timeline. This MSC adds a way to distribute and define a source for a client to send them. ## Proposal ### Emoticons in the formatted body -Emoticons have at least a shortcode and an mxc uri. They are sent as `` tags currently already in -the spec, as such existing clients should already be able to render them, though not all clients currently -handle `img` tags. To allow clients to distinguish emoticons from other inline images, a new -property, `data-mx-emoticon`, is introduced. A client can chose to ignore the size attributes of emoticons +Emoticons have at least a shortcode and a mxc uri. They are sent as `` tags, which are currently in +the spec. As such, many existing clients are able to render them. +To allow clients to distinguish emoticons from other inline images, a new +property, `data-mx-emoticon`, is introduced. A client can choose to ignore the size attributes of emoticons when rendering, and instead pick the size based on other circumstances. This could e.g. be used to -display emoticons in messages with only emoticons and emoji larger than usual, which is commonly found in -messengers. Such an `` tag of a shortcode `emote` and an mxc uri `mxc://example.org/emote` +display messages with only emoticons and emoji as larger than usual, which is commonly found in +messengers. Such an `` tag of a shortcode `emote` and a mxc uri `mxc://example.org/emote` could look as follows: ```html @@ -53,13 +53,13 @@ The `height` is just a height that looks good on most devices with the normal, d No width is displayed as to not weirdly squish non-square emotes. In order to maintain backwards-compatibility with clients not supporting emotes, specifying the `height` is required. -If the new `data-mx-emoticon` attribute has a value it is ignored. Due to limitations of some libraries +If the new `data-mx-emoticon` attribute has a value, it is ignored. Due to limitations of some libraries the attribute may even look like `data-mx-emoticon=""`. -The `src` attribute *must* be an mxc url. Other URIs, such as `https`, `mailto` etc. are not allowed. +The `src` attribute *must* be a mxc url. Other URIs, such as `https`, `mailto` etc. are not allowed. ### Sending stickers -To send stickers the already speced `m.sticker` is used. +To send stickers, the already spec'd `m.sticker` is used. ### Image types Emoticons are recommended to have a size of about 128x128 pixels. Even though the fallback specifies @@ -70,46 +70,54 @@ Stickers are recommended to have a size of up to 512x512 pixels. Furthermore, these images should either have a mimetype of `image/png`, `image/gif` or `image/webp`. They can be animated. -Due to the low resolution of emotes, `image/jpg`/`image/jpeg` has been purposefully excluded from this +Due to the low resolution of emotes, `image/jpg` and `image/jpeg` have been purposefully excluded from this list. ### Image pack event The image pack event has a type of `m.image_pack`. It contains a key `images`, which is a map from a -short code to an image object. It may also contain a key `pack`, which is a pack object. +short code to an image object. It may also contain a key `pack`, which is a pack object, described in +the following section. #### Pack object The pack object consists of the following keys: - `display_name`: (String, optional) A display name for the pack. Defaults to the room name, if the - image pack event is in the room. This does not have to be unique within all packs of a room. - - `avatar_url`: (String, optional) The mxc uri of an avatar/icon to dipslay for the pack. Defautls - to the room avatar, if the pack is in the room. If the room also does not have an avatar, or the - image pack event is not in a room, this pack does not have an avatar. - - `usage`: (String[], optional) An array of the usages for this pack. Possible usages are `emoticon` - and `sticker`. If the usage is absent or empty, a usage for all possible usage types is to be assumed. + image pack event is in the room. This does not have to be unique from other packs in a room. + - `avatar_url`: (String, optional) The mxc uri of an avatar/icon to display for the pack. Defaults + to the room avatar, if the pack is in the room. Otherwise, the pack does not have an avatar. + - `usage`: (String[], optional) An array of the usages for this pack. Possible usages are `"emoticon"` + and `"sticker"`. If the usage is absent or empty, a usage for all possible usage types is to be assumed. - `attribution`: (String, optional) The attribution of this pack. +Example: +```json +{ + "display_name": "Awesome Pack", + "usage": ["emoticon"] +} +``` + #### Image object -The image object conists of the following keys: - - `url`: (String, requried) The mxc URL for this image - - `body`: (String, optional) An optional body for this image, useful for the sticker body text or - the emote alt text. Defaults to the short code. - - `info`: (`ImageInfo`, optional) The already speced `ImageInfo` object used for the `info` block of +The image object consists of the following keys: + - `url`: (String, required) The mxc URL for this image. + - `body`: (String, optional) An optional text body for this image. Useful for the sticker body text or + the emote alt text. Defaults to the shortcode. + - `info`: (`ImageInfo`, optional) The already spec'd `ImageInfo` object used for the `info` block of `m.sticker` events. - - `usage`: (String[], optional) An array of the usages for this image. If present and non-empty, - this overrides the usage defined at pack level for this particular image. This is useful to e.g. - have one pack contain mixed emotes and stickers. Additionally there is only a single account data - level image pack, meaning this is required to have a mixture of emotes and stickers available in - account data. + - `usage`: (String[], optional) An array of the usages for this image. The possible values match those + of the `usage` key of a pack object. If present and non-empty, this overrides the usage defined at + pack level for this particular image. This is useful to e.g. have one pack contain mixed emotees and + stickers. Additionally, as there is only a single account data level image pack, this is required to + have a mixture of emotes and stickers available in account data. #### Example image pack event -Taking all this into account, an example pack event may look as following: +Taking all of this into account, an example pack event may look like the following: ```json { "images": { - "emote": { + "myemote": { "url": "mxc://example.org/blah" }, - "sticker": { + "mysticker": { "url": "mxc://example.org/sticker", "usage": ["sticker"] } @@ -122,34 +130,35 @@ Taking all this into account, an example pack event may look as following: ``` ### Image sources -There are several places where a client is expected to look for these `m.image_pack` events, mainly -in their own account data and in room states. +There are several places where a client is expected to look for events with type `m.image_pack`, mainly +in their own account data and in room state. #### User image packs -Each user can have their own personal image pack defined in their own account data, with the normal -`m.image_pack` event. The user is expected to be presented with these images in all rooms. +Each user can have their own personal image pack defined in their account data, under the +`m.image_pack` key. The user is expected to be presented with these images in all rooms. #### Room image packs A room can have an unlimited amount of image packs, by specifying the `m.image_pack` state event with -different state keys. The user is expected to be presented with these images only in the room they -are defined in. To enable them to be presented in all rooms, see the section below. -An empty state key is the default pack for a room. +different state keys. By default, the user is expected to be presented with these images only in the room they +they are defined in. To enable them to be presented in all rooms, see the "Image pack rooms" section. E.g. a discord bridge could set as state key `de.sorunome.mx-puppet-bridge.discord` and have all the bridged emotes in said state event, keeping bridged emotes from matrix emotes separate. #### Space image packs -Clients should suggest image packs of a canonical space that a room is in, if the user is also in the -space. This should be done recursively on canonical spaces. So, if a room has a canonical space and +Clients should suggest image packs of a room's canonical space, if the user is also in that space. + This should be done recursively on canonical spaces. So, if a room has a canonical space and that space again has a canonical space, the clients should suggest image packs of both of those spaces. #### Image pack rooms -While room image packs are specific to a room and are only accessible within that room, image pack -rooms should be accessible from everywhere. They do not differentiate themselves from room emotes at -all, instead you set an event in your account data of type `m.image_pack.rooms` which outlines -which room image pack states are globally accessible for that user. For that, a `room` key contains -a map of room ids that map to state keys that map to an object. While this MSC does not define any -contents for this object, having this an object means greater flexibility in case of future changes. +While room image packs are specific to a room, they can be made accessible from anywhere by setting +the `m.image_pack.rooms` key in a user's account data. The value is an object, with a `room` key containing +a map of room ids to the state keys to an object. If a room id / state key combination is provided in this form, +clients should make the corresponding room image pack globally accessible in all rooms. + +Note that while this MSC does not define any keys for the bottom-level object, definint it as an object means greater +flexibility in the case of future changes. + The contents of `m.image_pack.rooms` could look like the following: ```json @@ -168,38 +177,41 @@ The contents of `m.image_pack.rooms` could look like the following: Here three emote packs are globally accessible to the user: Two defined in `!someroom:example.org` (one with blank state key and one with state key `de.sorunome.mx-puppet-bridge.discord`) and one in -`!someotherroom:example.org`. +`!someotherroom:example.org` (with a blank state key). ### Image pack source priority and deduplicating -If a client gives image suggestions (emotes, stickers) to the user in some ordered fassion (e.g. a +If a client gives image suggestions (emotes, stickers) to the user in some ordered fassion (e.g. an ordered list where you click an entry), the order of the images should be predictable between rooms. -The ordering could look as following: -1. User image pack (images set in your own account) -2. Image pack rooms (rooms whos image packs you enabled to be accessible everywhere) -3. Space image packs (packs sent in the space, if present) -4. Room image packs (images defined in the currently open room) +A suggestion for clients of image pack ordering is as follows: +1. User image pack (defined in your own account data) +2. Image pack rooms (defined in the `m.image_pack.rooms` account data object) +3. Space image packs (defined in the hierarchy of canonical spaces for the current room) +4. Room image packs (defined in the currently open room's state) + +Furthermore, clients are expected to de-duplicate images so that same images are not displayed multiple +times. Such scenarios includ e.g.: +1. when viewing a room that has a pack defined in the `m.image_pack.rooms` account data object, and +2. a bot which syncs emotes over multiple rooms. -Furthermore, clients are expected to deduplicate images based on their mxc url. This not only ensures -that, when viewing a room that you also have in `m.image_pack.rooms`, it won't be displayed twice, -but also if you have e.g. a bot which syncs emotes over multiple rooms, those will also be deduplicated. +This could be done by deduplicating by the mxc URI. ### Sending #### Emoticons -For emoticons a client could add deliminators (e.g. `:`) around around the image shortcode, meaning +For emoticons a client could add deliminators (e.g. `:`) around the image shortcode, meaning that if an image has a shortcode of `emote`, the user can enter `:emote:` to send it. If there are -multiple emoticons with the same shortcode in a room the client could e.g. slugify the packs display -name and then have the user enter `:slug~emote:`. As slugs typically match `^[\w-]+$` that should +multiple emoticons with the same shortcode in a room, the client could e.g. slugify the packs display +name and then have the user enter `:slug~emote:`. As slugs typically match `^[\w-]+$`, that should ensure complete-ability. -The alt / title text fo the `` tag is expected to be the `body` of the emote, or, if absent, its -shotcode, optionally with tacked on deliminators. +The alt / title text for the `` tag is expected to be the `body` of the emote. If absent, its +shotcode should be used instead. #### Stickers -When sending a sticker the `body` of the `m.sticker` event should be set to the `body` defined for that -image, or its shortcode, if absent. +When sending a sticker, the `body` of the `m.sticker` event should be set to the `body` defined for that +image, or if absent, its shortcode. -Furthermore, the `info` of the `m.sticker` event should be set to the `info` defined for that image, -or a blank object, if absent. +The `info` object of the `m.sticker` event should be set to the `info` object of the image, or if absent, +an empty object. ## Security Considerations When sending an `` tag in an encrypted room, the client will make a request to fetch @@ -207,7 +219,7 @@ said image, in this case an emote. As there is no way to encrypt content behind this could potentially leak part of the conversation. This is **not** a new security consideration, it already exists. This, however, isn't much different from posting someone a link in an e2ee chat and the recipient opens the link. Additionally, images, and thus emotes, are often cached by the client, -not even necessarily leading to an http query. +not even necessarily leading to a http query. Related issue: /~https://github.com/matrix-org/matrix-doc/issues/2418 @@ -238,7 +250,7 @@ still easily have an image pack-only room. MSC1951 defines a way to recommend using a pack of a different room - this MSC does not have an equivalent to that. Instead, this MSC allows multiple image packs for a room, typically one where you already -chat in anyways. Furthermore it allows you to enable an image pack to be globally available for yourself +chat in anyways. Furthermore, it allows you to enable an image pack to be globally available for yourself across all rooms you are in. The core difference is how MSC1951 and this MSC define the image packs themselves: In MSC1951 you have to @@ -249,8 +261,8 @@ A simple dict of shortcode to mxc URI seems more appropriate for this. In general, MSC1951 feels like a heavier approach to image pack sources, while this MSC is more lightweight. ## Looking further -A couple of interesting points have been raised in the discussions of this MSC tangentially touch -custom emoticons but warrant an MSC for themselves, as they touch more on how `` is working. +A couple of interesting points have been raised in discussions of this MSC that tangentially touch +custom emoticons. Each warrant an MSC for themselves however, as they touch more on how `` is works. - Figuring out how `` should work with encrypted media. - Allow SVGs in the `` tag. Current problem: Clients typically try to thumbnail the mxc URL, and most media repositories can't thumbnail SVGs. Possible solution: Somehow embed the mimetype. From b5ca351dbd8f89aecde4d9aad091341ab5e1bd11 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Tue, 19 Sep 2023 23:20:05 +0200 Subject: [PATCH 27/35] fix some typos --- proposals/2545-emotes.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index 024004a54aa..47ea83283bc 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -105,7 +105,7 @@ The image object consists of the following keys: `m.sticker` events. - `usage`: (String[], optional) An array of the usages for this image. The possible values match those of the `usage` key of a pack object. If present and non-empty, this overrides the usage defined at - pack level for this particular image. This is useful to e.g. have one pack contain mixed emotees and + pack level for this particular image. This is useful to e.g. have one pack contain mixed emotes and stickers. Additionally, as there is only a single account data level image pack, this is required to have a mixture of emotes and stickers available in account data. @@ -156,7 +156,7 @@ the `m.image_pack.rooms` key in a user's account data. The value is an object, w a map of room ids to the state keys to an object. If a room id / state key combination is provided in this form, clients should make the corresponding room image pack globally accessible in all rooms. -Note that while this MSC does not define any keys for the bottom-level object, definint it as an object means greater +Note that while this MSC does not define any keys for the bottom-level object, defining it as an object means greater flexibility in the case of future changes. The contents of `m.image_pack.rooms` could look like the following: @@ -180,7 +180,7 @@ Here three emote packs are globally accessible to the user: Two defined in `!som `!someotherroom:example.org` (with a blank state key). ### Image pack source priority and deduplicating -If a client gives image suggestions (emotes, stickers) to the user in some ordered fassion (e.g. an +If a client gives image suggestions (emotes, stickers) to the user in some ordered fashion (e.g. an ordered list where you click an entry), the order of the images should be predictable between rooms. A suggestion for clients of image pack ordering is as follows: 1. User image pack (defined in your own account data) @@ -189,7 +189,7 @@ A suggestion for clients of image pack ordering is as follows: 4. Room image packs (defined in the currently open room's state) Furthermore, clients are expected to de-duplicate images so that same images are not displayed multiple -times. Such scenarios includ e.g.: +times. Such scenarios include: 1. when viewing a room that has a pack defined in the `m.image_pack.rooms` account data object, and 2. a bot which syncs emotes over multiple rooms. @@ -197,14 +197,14 @@ This could be done by deduplicating by the mxc URI. ### Sending #### Emoticons -For emoticons a client could add deliminators (e.g. `:`) around the image shortcode, meaning +For emoticons a client could add delimiters (e.g. `:`) around the image shortcode, meaning that if an image has a shortcode of `emote`, the user can enter `:emote:` to send it. If there are multiple emoticons with the same shortcode in a room, the client could e.g. slugify the packs display name and then have the user enter `:slug~emote:`. As slugs typically match `^[\w-]+$`, that should ensure complete-ability. The alt / title text for the `` tag is expected to be the `body` of the emote. If absent, its -shotcode should be used instead. +shortcode should be used instead. #### Stickers When sending a sticker, the `body` of the `m.sticker` event should be set to the `body` defined for that From 3517a1650f5cc6c1e40eded7e00fc11f65fd5803 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Thu, 15 Aug 2024 11:48:11 +0100 Subject: [PATCH 28/35] Make reference to spec/client naming disparity timeless --- proposals/2545-emotes.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index 47ea83283bc..59f7ed81618 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -21,9 +21,9 @@ emoticons, namely those found within unicode. Custom emoji here would actually r font, that is your own rendering of 🦊, 🐱, etc., *not* new images. New images is what custom emoticons are for. -Now, a client may choose to name these however they like. We already have a naming disparity between -spec and clients with groups vs communities. It is, however, imperative to name things in the spec -accurately after what they are. +Now, a client may choose to name these however they like. In the spec's history, +it has had naming disparity with clients, i.e. "groups" vs "communities". It is, +however, imperative to name things in the spec accurately after what they are. ### Stickers Stickers already exist in Matrix. They are reusable images one can send, usually as a reaction to From 5d62d52593dda4c86e1bc1cbcaa0c6abb4f95430 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Thu, 15 Aug 2024 12:28:16 +0100 Subject: [PATCH 29/35] Require `alt` and `title` attributes for emotes --- proposals/2545-emotes.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index 59f7ed81618..12af78b0366 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -45,9 +45,10 @@ could look as follows: :emote: ``` -Both the `alt` and the `title` attributes are specified as they serve different purposes: `alt` is -displayed if e.g. the emote does not load. `title` is displayed e.g. on mouse hover over the emote. -Thus, specifying both the `alt` and `title` attributes is required. +`alt` and `title` are required attributes for their roles in accessibility and +improved UX. Their meaning is inherited from the HTML specification +([alt](https://html.spec.whatwg.org/multipage/images.html#alt), +[title](https://html.spec.whatwg.org/multipage/dom.html#attr-title)). The `height` is just a height that looks good on most devices with the normal, default font size. No width is displayed as to not weirdly squish non-square emotes. In order to maintain backwards-compatibility From e5b9c2c503c19d089b28fe1c56265699089d2038 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Thu, 15 Aug 2024 12:50:38 +0100 Subject: [PATCH 30/35] Link `m.sticker` spec; small wording changes --- proposals/2545-emotes.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index 12af78b0366..16b4046ab48 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -51,16 +51,18 @@ improved UX. Their meaning is inherited from the HTML specification [title](https://html.spec.whatwg.org/multipage/dom.html#attr-title)). The `height` is just a height that looks good on most devices with the normal, default font size. -No width is displayed as to not weirdly squish non-square emotes. In order to maintain backwards-compatibility +No width is displayed as to not squish non-square emotes. In order to maintain backwards-compatibility with clients not supporting emotes, specifying the `height` is required. -If the new `data-mx-emoticon` attribute has a value, it is ignored. Due to limitations of some libraries +If the new `data-mx-emoticon` attribute has a value, that value is ignored. Due to limitations of some libraries the attribute may even look like `data-mx-emoticon=""`. -The `src` attribute *must* be a mxc url. Other URIs, such as `https`, `mailto` etc. are not allowed. +The `src` attribute *must* be a mxc URI. Other URIs, such as `https`, `mailto` etc. are not allowed. ### Sending stickers -To send stickers, the already spec'd `m.sticker` is used. +To send stickers, the already spec'd +[`m.sticker`](https://spec.matrix.org/v1.11/client-server-api/#msticker) is +used. ### Image types Emoticons are recommended to have a size of about 128x128 pixels. Even though the fallback specifies From 44c5ed31838c61f1a8fb4ad4700607744c15b563 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Thu, 15 Aug 2024 13:00:42 +0100 Subject: [PATCH 31/35] Prevent clients from rendering emotes behind non-MXC uri schemes --- proposals/2545-emotes.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index 16b4046ab48..3ef7c247438 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -57,7 +57,9 @@ with clients not supporting emotes, specifying the `height` is required. If the new `data-mx-emoticon` attribute has a value, that value is ignored. Due to limitations of some libraries the attribute may even look like `data-mx-emoticon=""`. -The `src` attribute *must* be a mxc URI. Other URIs, such as `https`, `mailto` etc. are not allowed. +The `src` attribute *must* be a mxc URI. Other URI schemes, such as `https`, +`mailto` etc. are not allowed. Clients MUST NOT attempt to render images +accessible through other URI schemes. ### Sending stickers To send stickers, the already spec'd From ea50cc7c6a83724e192de7dc88b938d73d3450d8 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Thu, 15 Aug 2024 13:01:39 +0100 Subject: [PATCH 32/35] Clarify client requirements around emote resolution --- proposals/2545-emotes.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index 3ef7c247438..e59ea90a637 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -67,12 +67,12 @@ To send stickers, the already spec'd used. ### Image types -Emoticons are recommended to have a size of about 128x128 pixels. Even though the fallback specifies -a height of 32, this is to ensure that the emotes still look good on higher DPI screens. +Emoticons SHOULD be at least 128x128 pixels. Even though the fallback specifies +a height of 32, this is to ensure that emotes still look good on higher DPI screens. -Stickers are recommended to have a size of up to 512x512 pixels. +Stickers SHOULD be at least 512x512 pixels, for the same reason. -Furthermore, these images should either have a mimetype of `image/png`, `image/gif` or `image/webp`. +Furthermore, these images SHOULD either have a mimetype of `image/png`, `image/gif` or `image/webp`. They can be animated. Due to the low resolution of emotes, `image/jpg` and `image/jpeg` have been purposefully excluded from this From 3c2e29c37ac690aad1a527e98872bf0080ad00a7 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Thu, 15 Aug 2024 14:20:38 +0100 Subject: [PATCH 33/35] Add a new proposal intro --- proposals/2545-emotes.md | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index e59ea90a637..83eb51efa47 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -1,8 +1,16 @@ # MSC2545: Image Packs (Emoticons & Stickers) -Emoticons, or short emotes...we need them! +Custom emoticons and stickers have become commonplace in modern messaging apps. +They allow users to express themselves in ways that words, nor the +standard Unicode set cannot. And they can be functional, allowing bots to convey +meaning with custom symbols in messages or reactions. -We also need proper stickers, too! +If Matrix is to remain both a fun and useful messaging protocol, as well as a +flexible platform for bridging into other protocols, support for custom +emoticons and stickers are a must. + +This proposal outlines how custom emoticons and stickers can both be sent into +rooms, as well as organised into "packs" and shared between users. ## Terminology ### Emoticons From c429552d70539b59ea89a8a9224d8a403dea0da9 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Wed, 28 Aug 2024 13:31:21 +0100 Subject: [PATCH 34/35] Recommend default of 32px for emote height --- proposals/2545-emotes.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index 83eb51efa47..523042e340a 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -58,9 +58,12 @@ improved UX. Their meaning is inherited from the HTML specification ([alt](https://html.spec.whatwg.org/multipage/images.html#alt), [title](https://html.spec.whatwg.org/multipage/dom.html#attr-title)). -The `height` is just a height that looks good on most devices with the normal, default font size. -No width is displayed as to not squish non-square emotes. In order to maintain backwards-compatibility -with clients not supporting emotes, specifying the `height` is required. +The `height` SHOULD be set to "32px". This is a default that looks good on most +devices. In practice, receiving clients are expected to override the height when +rendering emotes based on their particular environment (the user's font size, +etc.). In order to maintain backwards-compatibility with clients that do not +support emotes, specifying the `height` is required. No width is specified as to +maintain the aspect ratio of non-square emotes. If the new `data-mx-emoticon` attribute has a value, that value is ignored. Due to limitations of some libraries the attribute may even look like `data-mx-emoticon=""`. From d4e9cd22848c0b3943819552b6f63d295b3cfc59 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Thu, 29 Aug 2024 13:35:05 +0100 Subject: [PATCH 35/35] Specify image formats are equivalent to m.image Whenever they are defined. --- proposals/2545-emotes.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/proposals/2545-emotes.md b/proposals/2545-emotes.md index 523042e340a..c32195c4904 100644 --- a/proposals/2545-emotes.md +++ b/proposals/2545-emotes.md @@ -83,11 +83,13 @@ a height of 32, this is to ensure that emotes still look good on higher DPI scre Stickers SHOULD be at least 512x512 pixels, for the same reason. -Furthermore, these images SHOULD either have a mimetype of `image/png`, `image/gif` or `image/webp`. -They can be animated. +Images filetypes are not limited by this proposal. Instead they are +equivalent to the formats allowed in an +[`m.image`](https://spec.matrix.org/v1.11/client-server-api/#mimage) event. As +of writing, no limitations for `m.image` are currently defined (see [this spec +issue](/~https://github.com/matrix-org/matrix-spec/issues/453)). -Due to the low resolution of emotes, `image/jpg` and `image/jpeg` have been purposefully excluded from this -list. +Emoticons and stickers may be animated. ### Image pack event The image pack event has a type of `m.image_pack`. It contains a key `images`, which is a map from a