Multilingual character set support

[mathiasm75]

Hi first of all, great project! Thank you. Also for absolute newbie like me, i suceeded!

I made a translation into german, but can I also change the charakter set? I need the characters "ä", "ö" ...

Thank you,
Mathias from Austria

[lmarzen]

Thank you, I am glad to hear you were able to follow the documentation.

To address your question, yes. We can change the character set. The current state of this project has some limitations in this regard. It isn't the most user-friendly as currently the display only has the 7-bit ASCII character set. The solution I have used in the past is described here (#6). It entails using the unused 8th bit of ASCII to extend the character set and add 128 new characters. In the future, I want to research adding Unicode support, not sure how challenging this will be, but it seems like the ideal solution.

I am pretty busy this next week so I may not be able to provide much assistance until March 11.

-Luke

[aggr0lite]

Just looking into the same functionality, what a surprise! Just installed everything and then I was realizing Umlaut characters are not supported.

I was looking at another (similar) project as well which supports Umlaut characters, but I like your dashboard better as it's cleaner, great work!

So the project I was looking at first is this one: https://github.com/G6EJD/ESP32-e-Paper-Weather-Display
It uses this library for Umlaut characters: https://github.com/olikraus/u8g2

Unfortunately I don't have any idea how to integrate it myself but it works without issues.

Keep up your great work, thanks for uploading this!

[lmarzen]

Thanks for the suggestion. When I have some time later next week, I will look into integrating support for u8g2.

[lmarzen]

For now I added the extended ascii character set (5939ad9). See https://www.ascii-code.com/.

Here is an example: "Test: \xE4\xEB"

[aggr0lite]

That was a quick one. Looks great, I'll test and give feedback. Thanks!

[aggr0lite]

Works like a charm! Thanks for fixing this so quickly.
Do you still plan to integrate full support?

[lmarzen]

Glad to hear.

I would like to have better support for more character sets. I am considering adding the u8g2 library you suggested. Alternatively, I think proper utf-8 support would be ideal, but due to flash storage constraints, this may unfortunately not be feasible.

Until I get a chance to evaluate this further I will leave this issue open.

[lmarzen]

Update: I looked into modifying the adafruit-gfx library to add utf-8 support. It seems this will not work due to flash size limitations. Even just the Basic Multilingual Plane (https://en.wikipedia.org/wiki/Plane_(Unicode)) would result in a size of ~13.9MB for only the largest font size needed.

Possible paths forward:

1. It may be possible to compress the font bit map. (Complexity unknown, will compression be effective enough?)
2. FreeSans.otf is less than 1MB. We could store the file on the esp32 and generate the characters as needed. (Complexity unknown and may run slowly on esp32.)
3. Use u8g2 or a similar library. (Advantage of simplicity. Disadvantage of limited character sets and fonts)

[aggr0lite]

I appreciate you're still looking into this. For me your latest addition with the ascii character set works perfectly fine. Are there any lmitiations with it?

[lmarzen]

The main limitation is that you can only use the set of 255 characters in the extended ascii character set. Unicode, more specifically utf-8 would enable the use of many more characters.

[ZYL9]

Thanks for your great work!
I've use the library "olikraus/ u8g2_for_adafruit_gFx.@ ^1.8.0" and just make a few tweaks in renderer.cpp to make it work.
I have successfully displayed the Chinese characters on the screen, and the other unicode characters should be simple.
Hope you can consider adding related features when you are free.

Also, for reference, I used uncompressed Chinese fonts in two sizes (12pt/16pt), which worked well on FireBeetle 2 ESP32-E, here's the flash usage info when building.
RAM: [===] 31.3% (used 102596 bytes from 327680 bytes)
Flash: [======] 60.6% (used 1905669 bytes from 3145728 bytes)
And if you can crop the font library, I believe this can get much better flash usage.