Re: German special Characters ( incompatible character encodings: CP850 and UTF-8)
Posted by
mojavelinux on
May 02, 2013; 2:03am
URL: https://discuss.asciidoctor.org/German-special-Characters-incompatible-character-encodings-CP850-and-UTF-8-tp175p176.html
Long story short, Asciidoctor was properly handling non-Ascii characters *if* the system encoding was set to UTF-8. When the system encoding was changed to something other than UTF-8 (such as US-ASCII or CP850), Asciidoctor was unable to handle a string containing a UTF-8 character because Ruby would complain the encodings are incompatible.
If fixed this problem by forcing the encoding for any input data to UTF-8 if the system encoding is something other than UTF-8. I've also made the tests around string encoding even more rigorous, so if a bug appears, the tests will fail spectacularly.
In the meantime, the workaround is to use the -E flag to explicitly set the default encoding when running Ruby.
ruby -E UTF-8 my_script.rb
This is the recommended practice for running Ruby in production. When the default encoding is set to UTF-8, non of the force encoding code will be used.
Hope that helps!
-Dan
p.s. To emulate the encoding problem on any system (or test the fix), you simply set the encoding to something other than UTF-8 when running Ruby:
ruby -E US-ASCII my_script.rb
or
LANG=C ruby my_script.rb
or
LANG=US-ASCII my_script.rb