Feature request: show us intl keys

[mknj]

I am using a us-intl layout on windows and linux and i am missing letters like Ö, Ñ and Ø in the drawings.

I would like to see a blue layer like here: https://en.wikipedia.org/wiki/QWERTY#/media/File:KB_US-International.svg

Bonus: Ideally you could also specify that you are using i.e. a spanish, french or german layout and then the output is adapted to that language.

[caksoylar]

I think this is a duplicate of #32, except for the bonus section. That would require shipping data files that contain display forms of various keyboard layouts. Perhaps it would be possible by automatically parsing cldr files similar to https://github.com/joelspadin/zmk-locale-generator, while extracting shift/ralt keycodes.

[MattSturgeon]

Re: the bonus section, it is relatively simple to override the zmk_keycode_map or qmk_keycode_map in your config. For example I've done this in my config in order to draw the UK layout.

[paoloantinori]

Hi there. I'm experimenting with this feature, since the current behavior generates really ugly looking images on my setup:

https://github.com/paoloantinori/zmk-config-zen-2/raw/uk/keymap-drawer/corneish_zen.svg?raw=true

So far I've found that a funny improvement is to just make the zmk-locale-generator generated headers unavailable.
If that happens, instead of resolving the definitions, the original key is maintained, which is already a good improvement, moving from this:

layers:
  BASE:
  - - {t: '(ZMK HID USAGE(HID USAGE KEY, HID USAGE KEY KEYBOARD Q))', h: ESC}
    - {t: '(ZMK HID USAGE(HID USAGE KEY, HID USAGE KEY KEYBOARD W))', h: TAB}
    - (ZMK HID USAGE(HID USAGE KEY, HID USAGE KEY KEYBOARD E))
    - (ZMK

to this:

layers:
  BASE:
  - - {t: GB Q, h: ESC}
    - {t: GB W, h: TAB}
    - GB E
    - GB R

Just to make a little progress here I've even added an extra config param to keymap-drawer:

parse_config:
  remove_prefixes:
    - "GB"

just to simplify the manipulation that led me to this output:

layers:
  BASE:
  - - {t: ' Q', h: '$$mdi:keyboard-esc$$'}
    - {t: ' W', h: '$$mdi:keyboard-tab$$'}
    - ' E'
    - ' R'

with a very acceptable improvement over the output:

https://github.com/paoloantinori/zmk-config-zen-2/blob/visual_editor/keymap-drawer/corneish_zen.svg

But, I was hoping to have a real permanent solution to this approach and I've got a tip from @nickcoutsos who nicely shared his pre processing approach:

nickcoutsos/keymap-editor#141 (reply in thread)

I've managed to have his script working, so I see a path for a smart detection and delegation of the parsing logic to zmk-locale-generator libraries to avoid rewriting code from scratch.

I stopped here and decided to ask for feedback on this thread because zmk-locale-generator seems not to be a published dependency and I have no idea what could be the best course of action to consume those modules from withing this project.

Open to any feedback!

[caksoylar]

Locale headers like that are where our assumption of "preprocessed keycodes look like the short keycodes in Codes" breaks down, sadly. After reading your comment I thought about the issue further.

I think one main assumption we can make is the following:

• KD should (ideally) work with keycodes in the user's chosen keyboard layout, not the underlying HID keycodes as ZMK resolves the headers to.
  • This means a mapping {DQT: '"'} in zmk_keycode_map should convert GB_DQT (shift+2) to " during parsing, not DQT (shift+' in US layout).

There are two ways I can think of how to approach to achieve this:

1. Preprocess everything like we do now, but find a way to resolve the long form locale header outputs to the short form, e.g. ZMK_HID_USAGE(HID_USAGE_KEY, HID_USAGE_KEY_KEYBOARD_Q)) gets converted to Q. This can be achieved using keys.h. Then, these can be converted to the "right" legends for the user's locale, either through:
   • a user-provided zmk_keycode_map (like MattSturgeon did Feature request: show us intl keys #50)), or
   • a user-provided locale ID, which can map to a set of pre-generated keycode maps (through ZMK locale generator)
2. Instead of the preprocessor applying the header-based layout translation and us undoing it, find a way to prevent it from being applied in the first place, then use the usual zmk_keycode_map
   • how to disable the preprocessing for just that header? (special comment syntax in keymap? // keymap-drawer: ignore)
   • need a way to remove the locale prefix (like your config option), needs user input

Looks like 2. is what you ended up on with above, but I am not sure how using ZMK locale generator helps here -- full disclosure, I don't get what Nick's script is exactly doing.

One fundamental issue I see is that you don't know how the header was generated exactly, i.e. what locale and what prefix is used (from the discussion, looks like Nick makes some assumptions on that). Related to that and the two sub-bullets above, it looks like this cannot be fully automated.

Anyway, 2. seems like a better way to go overall to me. Just need to figure out the preprocessor bit somehow. Maybe Nick's script that you got working is working around some of these issues?

[nickcoutsos]

The script is mostly doing what zmk-locale-generator's batch_generate.py does to generate the header files, but as JSON files and with a subset of the values.

As an example,

{
  "locale": "de",
  "keys": [
    ...,
    {
      "names": [
        "DE_CAPITAL_SHARP_S",
        "DE_CAPITAL_ESZETT",
        "DE_CAPITAL_SZ"
      ],
      "symbol": "\u1e9e",
      "modifiers": [
        "LS",
        "RA"
      ]
    },
    ...
  ]
}

Turning that into a flat map is left as an exercise for the reader, but the idea is that I can

1. Parse #include "keys_de.h" and know that the user probably wants the German language header
2. Load keys_de.json from my locales, and
3. When encountering a key binding with the identifier DE_CAPITAL_SHARP_S (or one of its aliases), I can render the given symbol, and
4. When displaying the keycode picker and the active selection is DE_CAPITAL_SHARP_S I can disable the given modifiers, because they are being sent regardless.

So my assumptions is basically that: if someone uses zmk-locale-generator's headers they'll stick with that naming convention. Conversely, if someone has a "keys_de.h" header that is legitimately something besides a set of localized keycodes the result will just be that my app will show a list of keycodes that they don't have a reason to use.

[caksoylar]

Thanks Nick, seeing the output was helpful!

One thing your approach made me realize is that perhaps it is OK to assume certain things about the locale generator headers. Specifically, if we assume headers are only among the pre-generated ones, we can automate things quite a bit:

• Figure out what the user locale is from the #include filename
• Consequently, know what the prefix is for that locale
• Pre-generate and activate whatever mapping we prefer using that info

...which sounds like your approach, essentially.

I can see how this could be used by keymap-drawer in the following way:

1. Look for the #include pattern, from which determine the locale (falling back to en etc. if not found)
2. Pre-generate default zmk_keycode_maps for each locale in releases, going from HID keycode spec LS(RA) (long form of which the preprocessor would resolve to) to display form \u1e9e
3. Load default map for locale, union it with a user-provided zmk_keycode_map after removing the locale prefixes to look up keycodes
4. Do the mapping as usual

I think that sounds less hacky than approach 2, but it constrain us to the assumption that users are using locale headers from releases.

[caksoylar]

Meanwhile, I think it is a good idea to add #define KEYMAP_DRAWER to my preprocessor input so users can use #ifndef KEYMAP_DRAWER guards in their keymap files if they like to.

[paoloantinori]

Thanks for looking into this. The issue I have found in my experiments with zmk_keycode_map cannot be used as is, due to a split() call that otherwise separates the spaces generated definitions like (ZMK_HID_USAGE(HID_USAGE_KEY, HID_USAGE_KEY_KEYBOARD_A))

Here I'm hacking things badly just to see if it can work and it seems so. But to keep things simple I've replaced the separating space in the keybinding, and also removed it in zmk_keycode_map

'(ZMK_HID_USAGE(HID_USAGE_KEY,HID_USAGE_KEY_KEYBOARD_A))': 'A'
'(ZMK_HID_USAGE(HID_USAGE_KEY,HID_USAGE_KEY_KEYBOARD_B))': 'B'

https://github.com/caksoylar/keymap-drawer/compare/main...paoloantinori:keymap-drawer:multi_language?expand=1

This code works fine in my test

[caksoylar]

For now, before a better/automated solution can be implemented as discussed above, I implemented these two things brought up by @paoloantinori 's approach:

1. Add a way to modify keymap parsing without affecting the ZMK build: 1652cf9
2. Add a config for removing prefixes from keycodes before any further processing: a91689c

When you combine these two, you should be able to implement the following workaround for locale headers:

• Put the locale header #include in a define guard so it isn't processed by keymap-drawer:

  #ifndef KEYMAP_DRAWER
  #include "keys_gb.h"
  #endif

• Specify the locale prefix in the config: parse_config.zmk_remove_keycode_prefix = ["GB_"]

Then any &kp GB_XX will be processed as if it was &kp XX and you can further use parse_config.zmk_keycode_map to map XX to a legend, if you wish.

[englmaxi]

Thank you so much for this!! It works really well.

I noticed a small problem in 1652cf9: It still uses the excluded header in parse_config.raw_binding_map as it doesn't add the define in preprocess_extra_data in dts.py

[caksoylar]

@englmaxi Great catch, thank you! Should be fixed by 01c7d66.

[paoloantinori]

Thank you. I've been fiddling with this over the last couple of days and it's working well!

[rschenk]

I ran into this same problem today, because I use the same trick as zmk-locale-generator to do Colemak at the OS level instead of in my firmware. The #ifndef KEYMAP_DRAWER worked perfectly! Thank you all for the fix