[BUG?] only first character in list of candidates appears for zh-quick

[vwblue]

Describe the bug
I am not sure if this is a bug or not (see "Additional context" below).

When using an input method that offers multiple candidates, such as zh-quick.mim, ibus-typing-booster only offers the first character on the list.

To Reproduce
Steps to reproduce the behavior:

1. Add zh-quick as an input method in ibus-typing-booster
2. Switch to ibus-typing-booster as input (Super+Space)
3. Type the following characters: yy
4. Only the first candidate (訴) is offered. Pressing Tab does not display lookup.

Expected behavior
A list offering all characters for "yy" in the mim file should be offered as possible selections.

Screenshots or videos
My ibus-typing-booster settings:

Screencast showing behavior:

simplescreenrecorder-2024-07-15_05.33.44.mp4

The character list for "yy" is as follows:

  ("yy" ("訴卡卜卞於旃訃誨誹𠁎"))))

Multiple options are available for emojis when "Unicode symbols and emoji predictions" is enabled. Only mim files appear to be affected:

System Information
ibus-typing-booster v2.15.25
IBus 1.5.26
Pop!_OS 22.04 (based on Ubuntu)
Gnome 42.9
X11

Additional context
As I wrote above I am not sure if this is truly a bug or I have misunderstood something. I have tried googling information but nothing has been helpful.

I am currently working on a custom mim file for input and I want to create combinations that offer multiple possible options, but I'm not sure how to do that.

I wasn't sure if my mim file is improperly formatted (it loads without errors, but does not work as expected), which is why I tried zh-quick, because it appears to offer what I am attempting to do.
Here is a link to zh-quick.mim for reference: http://git.savannah.gnu.org/cgit/m17n/m17n-db.git/tree/MIM/zh-quick.mim

So I don't know what the issue is:

1. I notice the README states Chinese and Japanese aren't supported but doesn't clarify why. Does that mean ibus-typing-booster cannot offer multiple possible characters in the way zh-quick does? If so, is there any way for ibus-typing-booster to support multiple possible options so I can get my custom mim to work?
2. Is the zh-quick input method working as intended, and I've simply misunderstood how to use it? (If this is the case, how are other characters selected?)
3. Is there something in my settings that's causing this to happen?
4. Am I missing something, like an extra library or something that needs to be installed for this to work?
5. I notice my version is somewhat old. If this was fixed in a later version, can I upgrade to the latest version without having to wait for my distro to release it? If so, how?
6. Is it a genuine bug?

[mike-fabian]

@vwblue

1. I notice the README states Chinese and Japanese aren't supported but doesn't clarify why. Does that mean ibus-typing-booster cannot offer multiple possible characters in the way zh-quick does? If so, is there any way for ibus-typing-booster to support multiple possible options so I can get my custom mim to work?

You understood correctly that all .mim files which produce multiple options where one of the options needss to be selected do not work well with ibus-typing-booster at the moment.

This effects mostly Chinese and Japanese input methods but also input methods like lsymbol.mim which contains something like:

;;happy faces
  (":)" (delete @0) (("☺️" "😃" "😅" "😆" "😉" "😇")  "😂😏😛😜😝😋" "😉💏💋😍😘😚" "😽😻"))

This is more a “not yet implemented feature” then a bug tough.

And, as you correctly observed, this is the main reason that Chinese and Japanese are currently not supported by typing booster. There are some other problems to fix to get good support for Chinese and Japanese in typing-booster, for example the current prediction algorithm uses the last 3 words to predict the (maybe partially typed) next word. In languages which do not use spaces like Chinese and Japanese, it is sometimes not clear what exactly a word is. Instead of just splitting text at whitespace to get words, it needs to be parsed and split into "words" by other means. This will also need work.

But the first step is to support .mim files which offer multiple options and make it possible to easily and conveniently select a candidate.

I have already thought about how to do that and have some ideas, but it is not easy and I didn't have enough time for that yet. As you are the first user to ask for that, I’ll try to give this higher priority.

2. Is the zh-quick input method working as intended, and I've simply misunderstood how to use it? (If this is the case, how are other characters selected?)

Basically it is working as intended, only you cannot see the candidates offered except the first one.

If you look at zh-quick.mim, you see that it includes

(include (t nil zh-util) state)

which contains:

(map
 ;; Typing 1, 2, ..., 0 selects the 0th, 1st, ..., 9th candidate.
 (choose
  ("1" (select 0))
  ("2" (select 1))
  ("3" (select 2))
  ("4" (select 3))
  ("5" (select 4))
  ("6" (select 5))
  ("7" (select 6))
  ("8" (select 7))
  ("9" (select 8))
  ("0" (select 9)))

 (change-candidate
  ((Left) (select @-))
  ((C-b) (select @-))
  ((Right) (select @+))
  ((C-f) (select @+))
  ((Up) (select @\[))
  ((C-p) (select @\[))
  ((Down) (select @\]))
  ((C-n) (select @\])))

So you need to type 2 to select the second candidate in

("yy" ("訴卡卜卞於旃訃誨誹𠁎"))))

I.e. you need to type 2 to select 卡. But by default, 2 is already bound to select the second candidate in the prediction candidate list shown by typing booster. That means if you type yy and typing-booster shows 訴 in the preedit and it does not show a prediction candidate list (either because the prediction is disabled or because nothing has been learned yet by typing-booster to show any prediction or you use inline-completion without fallback a candidate list or similar options ...) you can type 2 and get 卡. So theoretically that works, but it is not usable without showing the lists of candidates. How should the user know that 2 needs to be typed to get 卡? And if typing-booster happens to show a completion candidate list, not even typing 2 helps anymore because 2 is bound by default to select the second candidate from the completion candidate list if it is shown.

You could remove the digits 1,2,3,4,5,6,7,8,9 from the commands commit_candidate_*_plus_space in the keybindings settings of typing-booster (these digits are bound to these commands by default). Then you can type 2 if you typed yy in zh-quick to get 卡 even if typing-booster happens to show a completion candidate list. But this just shows that zh-quick works as intended, it is still completely unusable in typing-booster.

3. Is there something in my settings that's causing this to happen?

No, not really, if you remove the digits 1,2,3,4,5,6,7,8,9 from the commands commit_candidate_*_plus_space in the keybindings settings of typing-booster, you can verify that zh-quick works as designed, but that doesn’t help to really use it.

4. Am I missing something, like an extra library or something that needs to be installed for this to work?

No.

5. I notice my version is somewhat old. If this was fixed in a later version, can I upgrade to the latest version without having to wait for my distro to release it? If so, how?

No, this still does not work in the latest release ibus-typing-booster-2.25-12.

6. Is it a genuine bug?

It is more a missing feature.

I think I need to do something similar to the compose support:

The chapter “Show possible completions of compose sequences” https://mike-fabian.github.io/ibus-typing-booster/docs/user/#5_4_5
shows how a completely different list of completion candidates is shown while a compose sequence is typed. For example if one types gr a “normal” typing-booster completion candidates list may be shown. If one then continues to type Compose u Tab, a list of options to complete the Compose sequence Compose u is shown instead of the “normal” typing-booster completion candidate list, for example " is a possible completion because Compose u " gives an ü. If that is selected, the preedit becomes grü and “normal” typing-booster completions may be shown again and show completions like grün, grünt, ...

For m17n input methods which produce multiple options, I need to implement something similar:

If an something typed with the m17n input method produces multiple options, these options need to be shown in a “special” candidate list instead of the “normal” completion candidates list.If one of these options is selected, typing-booster can continue to show the “normal” completion candidate list.

This is just the basic idea, maybe there other ways to do it.

Anyway, adding extra code to support this is needed.

[mike-fabian]

Here is a video showing how I use ibus-typing-booster with zh-quick:

• I removed the digits 1,2,3... from the keybinding settings
• I type yy and see 訴 in the preedit \
• I type 2 and now see 卡 in the preedit
• I type space to commit

Peek.2024-07-15.17-14.mp4

Shows that it works, sort of, but is not really usable.

[mike-fabian]

Here is another video where I use zh-quick.mim with ibus-m17n instead of ibus-typing-booster:

• I added zh-quick.mim using ibus-setup on the XFCE desktop.
• I selected zh-quick in the panel
• I opened the setup tool of ibus-m17n for zh-quick (just to show it, I don’t change anything there)
• I type yy in gedit and see 訴 in the preedit
• Now a candidate list with the options offered by zh-quick pops up.
• One cannot select the candidates with the mouse, this is not implemented in ibus-m17n yet
• But one can select the candidates by typing the digits, I type 1 and get 卡 which is committed immediately

Peek.2024-07-15.17-21.mp4

[mike-fabian]

So if you want to use zh-quick right now, you can do that by using ibus-m17n instead of ibus-typing-booster.

But ibus-m17n is very basic and simple and does not offer all the nice extra features ibus-typing-booster has.

Therefore, in the long run, I should add support for such m17n input methods which offer multiple candidates as well.