RBNF Number Formats in Messages

[maennchen]

How can RBNF number formats be used in ICU messages? Example:

{ordering,number,spellout_ordinal_r} Impfungstermin with number: 1 should output Erster Impfungstermin in locale de.

(Same for spellout_ordinal_feminine in fr, spellout_ordinal_r in de etc.)

I would like to replace this monstrosity: /~https://github.com/jshmrtn/hygeia/blob/main/lib/hygeia_web/live/person_live/vaccination.sface#L21

[kipcole9]

The ICU Messages documentation recommends that complex arguments be formatted outside the message itself:

FORMAT THE PARAMETERS SEPARATELY (RECOMMENDED)
You can format the parameter as you need before calling MessageFormat, and then passing the resulting string as a parameter to MessageFormat.

This offers maximum control, and is preferred to using custom format objects (see below).

However I have also implemented the functionality you're after per the examples below. Note the : prior to the format name as a way to signal this is a format supported by Cldr.Number.to_string/3.

iex> Cldr.Message.format("{ordering,number,:spellout_ordinal} jab date", ordering: 1)                  
{:ok, "first jab date"}
iex> Cldr.Message.format("{ordering,number,:ordinal} jab date", ordering: 1)                           
{:ok, "1st jab date"}
# `ordinal` is "well known" and therefore does not require the `:` before it
iex> Cldr.Message.format("{ordering,number,ordinal} jab date", ordering: 1)                           
{:ok, "1st jab date"}

There was a bug in this part of the code though which is fixed in commit 7ae3d02 so you'll need to mix deps.update ex_cldr_messages for this to work correctly.

[kipcole9]

Note that given the grammatical gender requirements (which the message format does not cater for) I suspect you will be better off formatting the number outside the message format in this example.

[kipcole9]

I have been contemplating whether there is a way to normalise RBNF formatting using grammatical case and gender. And thereby derive the right rule name. It can't be a complete solution but it should simplify a lot of formats - like your example.

Most of the rules have a standard structure: rule_type-ordinal_or_spellout_or_cardinal-gender-optional_plural_type. Like spellout_ordinal_feminine and spellout_ordinal_feminine_plural. But I can't decode the meaning of the "de" rules:

• :spellout_ordinal_s,
• :spellout_ordinal_r,
• :spellout_ordinal_n,
• :spellout_ordinal_m

Do you know what the _s, _r, _n and _m refer to? I could assume _n is neuter and _m is masculine but _s and _r I can't decode.

[maennchen]

@kipcole9

However I have also implemented the functionality you're after per the examples below.

Awesome, I will try that 😊

I suspect you will be better off formatting the number outside the message format in this example.

The issue with that is, that I do not know the grammatical gender outside of the translation. (We will have many translations that are done independently from the code.)

Do you know what the _s, _r, _n and _m refer to?

The rule names in german are weird. The letter at the end specifies with what letter the spelled out word ends. It’s a mix of gender and case. Heres a few tables detailing it for the word first: https://m.korrekturen.de/flexion/deklination/Erste/

I get now why people think german is complicated 😅

[maennchen]

Actually, that left out Erstem. Heere is a complete table: https://de.m.wiktionary.org/wiki/Flexion:erste

[kipcole9]

As of this commit in ex_cldr I have added a json configuration file to map grammatical case and grammatical gender to an RBNF rule. Its just a template for now.

To make this work I will need to refactor some function from ex_cldr_units and move them to ex_cldr, then implement a function to map case, gender, plural category into an RBNF rule - but that's all quite straight forward. And I don't see any current issues with backwards compatibility.

Thoughts, comments and suggestions most welcome.

[maennchen]

@kipcole9 This sounds nice, then people will not have to lookup those rule names that are somehow different in every language.

I don’t get though, what this has in common with this issue. How would this help with the problem?

[maennchen]

@kipcole9 Do you have an idea what could be the problem here?

msgctxt "Auto Tracing Vaccination"
msgid "{ordering,number,:spellout_ordinal} jab date"
msgstr "{ordering,number,:spellout_ordinal_r} Impftermin"

== Compilation error in file lib/hygeia_gettext.ex ==
** (exit) an exception was raised:
    ** (CompileError) lib/hygeia_gettext.ex: invalid quoted expression: {:simple_format, {:named_arg, "ordering"}, :number, :spellout_ordinal_r}

Please make sure your quoted expressions are made of valid AST nodes. If you would like to introduce a value into the AST, such as a four-element tuple or a map, make sure to call Macro.escape/1 before
        (elixir 1.13.1) src/elixir_expand.erl:600: :elixir_expand.expand_arg/3
        (elixir 1.13.1) src/elixir_expand.erl:538: :elixir_expand.expand_list/5
        (elixir 1.13.1) src/elixir_expand.erl:448: :elixir_expand.expand/3
        (elixir 1.13.1) src/elixir_expand.erl:600: :elixir_expand.expand_arg/3
        (elixir 1.13.1) src/elixir_expand.erl:616: :elixir_expand.mapfold/5
        (elixir 1.13.1) src/elixir_expand.erl:828: :elixir_expand.expand_remote/8
        (elixir 1.13.1) src/elixir_expand.erl:546: :elixir_expand.expand_block/5
        (elixir 1.13.1) src/elixir_expand.erl:40: :elixir_expand.expand/3
        expanding macro: HygeiaCldr.GettextInterpolation.compile_interpolate/3
        lib/hygeia_gettext.ex:1: HygeiaGettext.T_de_default.de_default_lgettext/3

[kipcole9]

I have published ex_cldr_messages version 0.13.1 to fix this issue (and the bug related to RBNF formats). The changelog entry is:

Bug Fixes

• Fix Cldr.Message.Interpolate.compile_interpolate/3 to ensure that the parsed message is escaped AST. Closes RBNF Number Formats in Messages #11.

• Fix passing RBNF format names as number format styles.

• Fix converting strings to atoms at compile time. Previously these uses String.to_existing_atom/1 however the compilation graph sometimes means that the expected atoms are not available. Therefore String.to_atom/1 is now called. As a result, it is very important to note that allowing arbitrary message formats into the system could open an attack vector to exhaust the atom table.

[maennchen]

@kipcole9 Thanks :)

Do you know what the reason for this is?

[error] GenServer #PID<0.4039.0> terminating
** (BadMapError) expected a map, got: []
    (elixir 1.13.1) lib/map.ex:464: Map.get([], :spellout_ordinal_r, nil)
    (ex_cldr_messages 0.13.1) lib/cldr/messages/format/interpreter.ex:518: Cldr.Message.Interpreter.configured_message_format/2
    (ex_cldr_messages 0.13.1) lib/cldr/messages/format/interpreter.ex:176: Cldr.Message.Interpreter.format_list/5
    (ex_cldr_messages 0.13.1) lib/cldr/messages/format/interpreter.ex:46: Cldr.Message.Interpreter.format_list/5
    (ex_cldr_messages 0.13.1) lib/cldr/messages/format/interpreter.ex:12: Cldr.Message.Interpreter.format_list/3
    (ex_cldr_messages 0.13.1) lib/cldr/messages/backend.ex:90: Cldr.Message.Backend.gettext_interpolate/3
    (gettext 0.19.0) lib/gettext.ex:761: Gettext.dpgettext/5
    (hygeia 1.40.0-beta.2) lib/hygeia_web/live/person_live/vaccination.sface:19: anonymous fn/5 in HygeiaWeb.PersonLive.Vaccination.render/1
    (elixir 1.13.1) lib/enum.ex:2396: Enum."-reduce/3-lists^foldl/2-0-"/3
    (hygeia 1.40.0-beta.2) lib/hygeia_web/live/person_live/vaccination.sface:16: anonymous fn/4 in HygeiaWeb.PersonLive.Vaccination.render/1
    (phoenix_live_view 0.16.4) lib/phoenix_live_view/diff.ex:356: Phoenix.LiveView.Diff.traverse/6
    (phoenix_live_view 0.16.4) lib/phoenix_live_view/diff.ex:444: anonymous fn/4 in Phoenix.LiveView.Diff.traverse_dynamic/6
    (elixir 1.13.1) lib/enum.ex:2396: Enum."-reduce/3-lists^foldl/2-0-"/3
    (phoenix_live_view 0.16.4) lib/phoenix_live_view/diff.ex:356: Phoenix.LiveView.Diff.traverse/6
    (phoenix_live_view 0.16.4) lib/phoenix_live_view/diff.ex:444: anonymous fn/4 in Phoenix.LiveView.Diff.traverse_dynamic/6
    (elixir 1.13.1) lib/enum.ex:2396: Enum."-reduce/3-lists^foldl/2-0-"/3
    (phoenix_live_view 0.16.4) lib/phoenix_live_view/diff.ex:356: Phoenix.LiveView.Diff.traverse/6
    (phoenix_live_view 0.16.4) lib/phoenix_live_view/diff.ex:444: anonymous fn/4 in Phoenix.LiveView.Diff.traverse_dynamic/6
    (elixir 1.13.1) lib/enum.ex:2396: Enum."-reduce/3-lists^foldl/2-0-"/3
    (phoenix_live_view 0.16.4) lib/phoenix_live_view/diff.ex:356: Phoenix.LiveView.Diff.traverse/6
Last message: %Phoenix.Socket.Message{event: "event", join_ref: "4", payload: %{"event" => "add_vaccination_jab_date", "type" => "click", "value" => %{"value" => ""}}, ref: "14", topic: "lv:phx-FsnXl1ykWUq6qq3C"}

(same translation as before)

[kipcole9]

My apologies, I have fixed this properly now and added additional tests. The issue arose because you have no :message_formats configured in your CLDR backend module (which is probably very normal).

I have published ex_cldr_messages 0.13.2 with the fix.