Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lem SDL2 can now properly display zero-width characters. #1739

Merged
merged 1 commit into from
Jan 9, 2025

Conversation

funk443
Copy link
Contributor

@funk443 funk443 commented Jan 9, 2025

Previously, if someone tried to open a file that contains zero-width characters, such as U+200B, U+200C, U+200D, and U+FEFF, Lem SDL2 will just refuse to display any content, leaving the user with a blank window. This is probably because SDL2 does not like zero-width characters.

So now I try to fix this problem by adding another char-type dedicated for zero-width characters, and display them with a special character.

I've also created a new attribute for zero-width characters and control characters, so users won't confuse them with real contents.

Previously, if someone tried to open a file that contains zero-width
characters, such as `U+200B`, `U+200C`, `U+200D`, and `U+FEFF`, Lem
SDL2 will just refuse to display any content, leaving the user with
a blank window.  This is probably because SDL2 does not like zero-width
characters.

So now I try to fix this problem by adding another `char-type` dedicated
for zero-width characters, and display them with a special character.

I've also created a new attribute for zero-width characters and control
characters, so users won't confuse them with real contents.
@sakurawald
Copy link
Contributor

I think this may be a relative issue, would you like to test it?
#1698

@funk443
Copy link
Contributor Author

funk443 commented Jan 9, 2025

I think this may be a relative issue, would you like to test it? #1698

Interesting, it seems Lem just hang instead of giving out that SDL error while reproducing this issue. Maybe there are other weird characters in these documents which cause these problems.

image

@funk443
Copy link
Contributor Author

funk443 commented Jan 9, 2025

So I just checked if Lem can now open the file that I wanted to open, turns out it still gives out the same zero-width error from SDL2. So I went to see what kind of characters this document has, it turns out it does not contain any zero-width character AT ALL, then it must be some other character causing the issue.

The document: https://raw.githubusercontent.com/dannyvankooten/1brc/refs/heads/main/README.md

@funk443
Copy link
Contributor Author

funk443 commented Jan 9, 2025

I think the problem is cause by Lem not recognising 🏎️, which in terms of unicode characters is \u1F3CE\uFE0F. So maybe #1698 is caused by the same issue?

Edit: yeah, I think the issue is ◌️ (U+FE0F), which also appears in themes/hugo-geekdoc/static/js/116-773fdae5.chunk.min.js from #1698.

@cxxxr cxxxr merged commit 0d2ce9f into lem-project:main Jan 9, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants