I think this is a rare “useful” abuse of font ligatures, and in that respect I think this is a cool project, but it's also a terrible idea that nobody should actually use, because Japanese kanji readings are famously ambiguous and context-dependent. The modal character has two different readings without counting those used in personal names, nor contextual slight changes to pronunciation. Some characters have six different readings, and that's again without counting those used in personal names. There's no way a font can do a good job at it unless you can somehow stuff an entire LLM in there.
For context, it's a hard enough problem that Google Translate's own romanised pronunciation guides are frequently wrong, or disagree with their text-to-speech.
And while you can usually figure out which reading applies based on what follows, there are still many cases where you can't. A standalone 今日 can be any of "konnichi", "kyou" or "konjitsu" depending on context and intent.
Yeah. I also notice that Google Translate's guess at pronunciation frequently has errors, and that IS a LLM.
Someone further downthread said it appears to have about 80% accuracy on a short sentence, which means if you're not already at a level of skill where you can verify, then it's going to teach you the wrong pronunciations constantly. When studying languages myself, I find that learning the wrong pronunciation for something is annoyingly sticky, and unlearning takes more effort than learning something correctly the first time.
Pretty good job, meaning that you get 96% there (if I recall correctly the results for MeCab which uses CRF). But 4% of kanji words is still multiple wrong furigana per page.
> but it's also a terrible idea that nobody should actually use, because Japanese kanji readings are famously ambiguous and context-dependent
Seems a bit drastic. Don't use it as the main font in a japanese textbook, sure, but use it with the disclaimer that it's not perfect and will get some readings wrong? Why not, still pretty useful. Should people also not use Google Translate? (also, you can stuff an entire llm in a font, it's been done).
The point of furigana is to be exact - it's a pronunciation guide, and its purpose is to distinguish or to teach. If it's wrong, then nothing makes sense.
If I asked you 'are you a sewer (pronounced "soo-er")?', would that make sense? Maybe if you looked at it hard enough and thought about it, but in the meantime, I would probably get some very strange looks for asking.
As far as I know, all the automated options out there (wether it's Furiganator, the chrome extension, or Google Translate's pronunciation) are far from 100% accurate. I still find both to be very useful, I just don't rely on them to be the absolute truth; they still help me on a daily basis. The web page for this font mentioned it didn't show any furigana if it was ambiguous (I haven't verified),and someone else here mentioned it was ~80% accurate. I respectfully disagree with the fact that it's useless if not perfect; I've personally found all these tools useful, and have not learned a single "wrong reading" because of them because I don't use them as a tool to learn kanji readings to begin with.
It's funny to see LLMs being mentioned, because this is yet another case where "good enough" is, after all, not good enough once you know how "good enough" should be.
Stumbling on names is fine, name litteraly have no logic, there is no system that will get them 100% correct by definition.
Screwing meaning is on a whole different level and Google Translate or even the built in screen reader of your OS will do an extremely good job given enough context (a whole phrase).
In real world use there is little space for sloppy hacks IMHO.
> Google Translate or even the built in screen reader of your OS will do an extremely good job given enough context
Google translate does not - I see it get readings wrong on very basic sentences with plenty of context on a daily basis. It doesn't make it useless. The website mentioned the font wouldn't show furigana if the reading was ambiguous so I was basing my judgement on that, obviously if it's just always wrong, that's not useful.
You're underestimating how wrong this is going to be on average. This is going to be nonsense like half the time. It's not comparable to Google Translate, it's more comparable to Babel Fish circa 1997.
Really nice font! Native Japanese speaker here, and seeing furigana done purely in the font is very cool.
One small thing on the Japanese version of the page: in the "Ligatures do the work" section, 合字 itself only gets じ over 字, and 合 has no reading. It should be ごうじ. I guess 合字 isn't in the dictionary, so it falls back to single kanji, and 合 is skipped because it has several readings.
Automatic furigana is not really possible to be reliable considering readings heavily rely on context that can't be easily inferred, but I'm sure it will help some people who can't read very well.
You can run LLMs inside the shaping engine, so maybe you can use more advanced segmenting and transliteration algorithms? https://fuglede.github.io/llama.ttf/
There aren't good datasets for that. Even neural TTS for Japanese struggles with it since Japanese transcriptions used to train models don't preserve pronunciations used.
(or maybe it could be theoretically done, just the mechanism to automatically classify wildly different pronunciations mapped to the same literal expressions don't yet exist)
It can depend on implied information, e.g. 方 can be hou or kata depending on if you're talking about about direction or a person. And that information can be before it, after it, on some image next to it, etc.
This is very cool. Unfortunate that it gives the wrong pronunciation when words get split in a line wrap (e.g., 大学 got split between lines and changed to incorrect furigana for me).
This is really nice. I'm able to read Hiragana and Katakana fully after about 4-6 weeks now but then I'm hit with the reality of how many Kanji I have to learn still everytime I want to read something in real life lol
When you control the content, I agree that ruby tags would be better, since they can easily handle alternative readings and also enable smarter automation to infer the correct reading from context.
But you could set this as the default system font for Japanese and enjoy furigana everywhere, even in places that don't support ruby tags at all.
You shouldn’t because this is wrong a huge percent of the time, like 20-50%. It’s an interesting trick but please don’t use it as a learner. Even the demo has mistakes.
Maybe this is less dependent on rendering engine or easier to use. Or maybe because it could be done and was done.
From practicality standpoint, this would not work on unprepared texts, as pronunciation of Japanese words are a bit arbitrary and so correct pronunciations would need to be manually selected for each words.
i.e. "maize" could be read "corn". I believe LLMs don't necessarily have good intuition for that, being trained on plain texts.
As to how could such a situation could ever manifest: I would blame the fact that the official language of historical Japanese governments having been a really creative interpretation of medieval Chinese[1] sprinkled with labels and jumps and supplements[2][3], resulting in people just accepting wild divergence between literal and spoken Japanese. I'm not a linguist or a language major so I'd happy if anyone could further elaborate...
Surprised that most of the furigana on the landing page are accurate. A few basic errors did stand out, though: 行われる (furifont: i-wareru, actual: okona-wareru) and 行に (furifont: i-ni, actual: gyo-ni). The font would work better if it took okurigana into account.
That’s incredible. I’m always impressed with the utility of the TrueType/open font spec - the flexibility there is really an underappreciated work of engineering genius.
I mean, fonts are Turing-complete [0], so it isn’t surprising that this is possible. Turing-completeness is often by accident rather than by genius [1].
Which makes sense, because human language is apparently Turing complete, so anything intended to represent it faithfully would wind up with that characteristic, too.
Human language isn’t a formal language, so this is a category error.
In the present context, an example is that the common reading of a Kanji word isn’t always well-defined, and neither is the decomposition of a sequence of Kanji characters into words.
It's an interesting move to point me to a rigorously written English language definition of a formal language to make the argument that humans don't co-opt our natural languages as formal languages when the need arises.
Bitmap fonts aren't turing complete. The fact that TrueType is built with an arbitrary virtual machine is insane, makes implementations ridiculously complex, and is a huge security liability. The purpose of the VM wasn't to "represent human language faithfully" but to make fonts look a little better when displayed at a variety of resolutions, via hinting. Hinting doesn't necessarily require a full VM, but somebody let the engineers have a little too much fun.
Little VMs are really easy to sandbox if you don't need a billion instructions per second, and compared to complicated math thresholds it's easier and simpler for a VM to implement everything you need.
And yet, actually mapping all written human languages and their ligatures, variations, etc, seems to require this level of complexity. So the choice is either you draw the line and decide which languages you consider legitimate or not, or you build a machine of sufficient complexity to capture all languages, and the tradeoff, I suppose, is how much knowledge you want to actually be able to capture in your computerized system.
Unicode is distinct from, and completely unrelated to, TrueType. Unicode does not require a virtual machine to interpret. You can simply express languages and their variations with Unicode + bitmap fonts.
Do you really believe the only way to implement ligatures is with a virtual machine? Dumb and dirty psuedocode written in 5 seconds of thinking about it but come on, this is not that complicated.
for each character in text
characterToPrint = Parse(character)
if character is LigatureStartCandidate
characterToPrint = LookAheadForNextLigatureCandidateAndResolve(character)
Print(characterToPrint)
And you've now gotten 92% of the way there. Iterate a few dozen more times to get up to 97%, 98%, 99%, etc., and eventually you'll have rewritten OpenType.
Would you like to elaborate on those few dozen more cases rather than handwaving them away? Embedding a VM into a font format is foundationally a ridiculous idea and should require real justification, although I know our industry is in love with anything that allows arbitrary code execution exploits.
I could, or you could open your mind to the idea that the reason opentype wound up being as complex as it is is because the problem space you’re trying to solve is more complicated than you think it is. There is a working reference implementation which does the thing you think can be done more cheaply, you’re just throwing popcorn from the cheap seats here.
I want furigana on display glasses, like Even Realities. Since the application would know my vocabulary, it can show just the words that I'm not likely to know.
Assuming you could hint the furigana from the application, the phonetic transcript would make it more accurate.
This is AWESOME for Japanese learners. There are 46 basic hiragana that represent the sounds of Japanese but many thousands of Kanji, and it’s incredibly hard to read Japanese even if you’ve been studying for years.
A few people here rightly pointing out inaccuracies and other methods of display (ie Ruby tags) but IMO they are somewhat missing the usefulness here - there are many applications (ie e-readers, random web contexts, custom apps etc) where the traditional methods of display are hard or fail. I recently ran into it trying to get custom Japanese ebooks to display furigana with ruby tags on my kindle. It’s a giant nightmare.
Excited to try this out.
For context, it's a hard enough problem that Google Translate's own romanised pronunciation guides are frequently wrong, or disagree with their text-to-speech.
https://en.wiktionary.org/wiki/%E7%94%9F
And while you can usually figure out which reading applies based on what follows, there are still many cases where you can't. A standalone 今日 can be any of "konnichi", "kyou" or "konjitsu" depending on context and intent.
Well, it wouldn't be the first time ;)
https://fuglede.github.io/llama.ttf/
Someone further downthread said it appears to have about 80% accuracy on a short sentence, which means if you're not already at a level of skill where you can verify, then it's going to teach you the wrong pronunciations constantly. When studying languages myself, I find that learning the wrong pronunciation for something is annoyingly sticky, and unlearning takes more effort than learning something correctly the first time.
I’d say you can do a pretty good job with conditional random fields.
Still, very cool demo!
Seems a bit drastic. Don't use it as the main font in a japanese textbook, sure, but use it with the disclaimer that it's not perfect and will get some readings wrong? Why not, still pretty useful. Should people also not use Google Translate? (also, you can stuff an entire llm in a font, it's been done).
If I asked you 'are you a sewer (pronounced "soo-er")?', would that make sense? Maybe if you looked at it hard enough and thought about it, but in the meantime, I would probably get some very strange looks for asking.
Stumbling on names is fine, name litteraly have no logic, there is no system that will get them 100% correct by definition.
Screwing meaning is on a whole different level and Google Translate or even the built in screen reader of your OS will do an extremely good job given enough context (a whole phrase).
In real world use there is little space for sloppy hacks IMHO.
Google translate does not - I see it get readings wrong on very basic sentences with plenty of context on a daily basis. It doesn't make it useless. The website mentioned the font wouldn't show furigana if the reading was ambiguous so I was basing my judgement on that, obviously if it's just always wrong, that's not useful.
One small thing on the Japanese version of the page: in the "Ligatures do the work" section, 合字 itself only gets じ over 字, and 合 has no reading. It should be ごうじ. I guess 合字 isn't in the dictionary, so it falls back to single kanji, and 合 is skipped because it has several readings.
https://dic.pixiv.net/a/%E3%81%93%E3%81%AE%E5%85%88%E7%94%9F...
(or maybe it could be theoretically done, just the mechanism to automatically classify wildly different pronunciations mapped to the same literal expressions don't yet exist)
But you could set this as the default system font for Japanese and enjoy furigana everywhere, even in places that don't support ruby tags at all.
That way choosing a powerful and more accurate engine, or more adapted display of the characters also becomes an option.
From practicality standpoint, this would not work on unprepared texts, as pronunciation of Japanese words are a bit arbitrary and so correct pronunciations would need to be manually selected for each words.
i.e. "maize" could be read "corn". I believe LLMs don't necessarily have good intuition for that, being trained on plain texts.
As to how could such a situation could ever manifest: I would blame the fact that the official language of historical Japanese governments having been a really creative interpretation of medieval Chinese[1] sprinkled with labels and jumps and supplements[2][3], resulting in people just accepting wild divergence between literal and spoken Japanese. I'm not a linguist or a language major so I'd happy if anyone could further elaborate...
1: https://en.wikipedia.org/wiki/Kanbun
2: https://youtu.be/_k0I6bi2pYs?t=248
3: https://d1ysbfo9sxjda0.cloudfront.net/wp-content/uploads/201...
https://www.destroyallsoftware.com/talks/wat
[0] https://litherum.blogspot.com/2019/03/addition-font.html
[1] https://en.wikipedia.org/wiki/Turing_completeness#Unintentio...
In the present context, an example is that the common reading of a Kanji word isn’t always well-defined, and neither is the decomposition of a sequence of Kanji characters into words.
Maybe it is:
With the ligatures to adApt to pronunciation of words.
frequency wise it should be dai-ninki for sure
https://pingtype.github.io/japanese.html
It also supports line breaking, so that words don't get cut in half when wrapping text.
Assuming you could hint the furigana from the application, the phonetic transcript would make it more accurate.
confusing when you don't explain which limktations this is
A few people here rightly pointing out inaccuracies and other methods of display (ie Ruby tags) but IMO they are somewhat missing the usefulness here - there are many applications (ie e-readers, random web contexts, custom apps etc) where the traditional methods of display are hard or fail. I recently ran into it trying to get custom Japanese ebooks to display furigana with ruby tags on my kindle. It’s a giant nightmare. Excited to try this out.