Asciidoctor :: Discussion
Login  Register

German special Characters ( incompatible character encodings: CP850 and UTF-8)

‹ Previous Topic Next Topic ›
 
classic Classic list List threaded Threaded
♦
♦
 4 messages Options Options
 
Embed post
Permalink
mbw
Reply | Threaded
Open this post in threaded view
♦
♦
| More
Print post
Permalink
Apr 30, 2013; 2:49pm

German special Characters ( incompatible character encodings: CP850 and UTF-8)

mbw
2 posts
Hi all,

rendering a text with Asciidoctor, I alway get this error - only when using german sepcial characters (äöüßÄÜÖ).
If I don't use german special characters (or html-entities), all works fine.
Windows 7 / 64-Bit

CompatibilityError: incompatible character encodings: CP850 and UTF-8

On rendering with asciidoc - all works fine.

Any Ideas?

thanks, Manfred

mojavelinux
Reply | Threaded
Open this post in threaded view
♦
♦
| More
Print post
Permalink
May 02, 2013; 2:03am

Re: German special Characters ( incompatible character encodings: CP850 and UTF-8)

mojavelinux
Administrator
2681 posts
Manfred,

Thank you for reporting this issue. A similar issue was reported this morning. I decided to take action on it. I've since applied a fix (which is pending review).

https://github.com/asciidoctor/asciidoctor/issues/308

Long story short, Asciidoctor was properly handling non-Ascii characters *if* the system encoding was set to UTF-8. When the system encoding was changed to something other than UTF-8 (such as US-ASCII or CP850), Asciidoctor was unable to handle a string containing a UTF-8 character because Ruby would complain the encodings are incompatible.

If fixed this problem by forcing the encoding for any input data to UTF-8 if the system encoding is something other than UTF-8. I've also made the tests around string encoding even more rigorous, so if a bug appears, the tests will fail spectacularly.

In the meantime, the workaround is to use the -E flag to explicitly set the default encoding when running Ruby.

ruby -E UTF-8 my_script.rb

This is the recommended practice for running Ruby in production. When the default encoding is set to UTF-8, non of the force encoding code will be used.

Hope that helps!

-Dan

p.s. To emulate the encoding problem on any system (or test the fix), you simply set the encoding to something other than UTF-8 when running Ruby:

ruby -E US-ASCII my_script.rb

or

LANG=C ruby my_script.rb

or

LANG=US-ASCII my_script.rb



On Tue, Apr 30, 2013 at 8:49 AM, mbw [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Hi all,

rendering a text with Asciidoctor, I alway get this error - only when using german sepcial characters (äöüßÄÜÖ).
If I don't use german special characters (or html-entities), all works fine.
Windows 7 / 64-Bit

CompatibilityError: incompatible character encodings: CP850 and UTF-8

On rendering with asciidoc - all works fine.

Any Ideas?

thanks, Manfred



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/German-special-Characters-incompatible-character-encodings-CP850-and-UTF-8-tp175.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML
... [show rest of quote]



--
Dan Allen
Principal Software Engineer, Red Hat | Author of Seam in Action
Registered Linux User #231597
mbw
Reply | Threaded
Open this post in threaded view
♦
♦
| More
Print post
Permalink
May 02, 2013; 7:08am

Re: German special Characters ( incompatible character encodings: CP850 and UTF-8)

mbw
2 posts
Hi Dan,

thank's for your quick help!

I changed the asciidoctor.bat - Start-Script and it works! (Windows7 64 Bit,German)
Manfred

echo asciidoctor
@ECHO OFF
IF NOT "%~f0" == "~f0" GOTO :WinNT
echo start
@"ruby.exe"  "C:/SOFTWARE/Ruby200-x64/bin/asciidoctor"  %1 %2 %3 %4 %5 %6 %7 %8 %9
GOTO :EOF
:WinNT
@"ruby.exe"  -E UTF-8    "%~dpn0" %*

mojavelinux
Reply | Threaded
Open this post in threaded view
♦
♦
| More
Print post
Permalink
May 02, 2013; 10:32am

Re: German special Characters ( incompatible character encodings: CP850 and UTF-8)

mojavelinux
Administrator
2681 posts
Great!

I've also merged in support to Asciidoctor to handle this gracefully if the system encoding happens to be something other than UTF-8.

https://github.com/asciidoctor/asciidoctor/pull/310

Enjoy the writing!

-Dan


On Thu, May 2, 2013 at 1:08 AM, mbw [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Hi Dan,

thank's for your quick help!

I changed the asciidoctor.bat - Start-Script and it works! (Windows7 64 Bit,German)
Manfred

echo asciidoctor
@ECHO OFF
IF NOT "%~f0" == "~f0" GOTO :WinNT
echo start
@"ruby.exe"  "C:/SOFTWARE/Ruby200-x64/bin/asciidoctor"  %1 %2 %3 %4 %5 %6 %7 %8 %9
GOTO :EOF
:WinNT
@"ruby.exe"  -E UTF-8    "%~dpn0" %*



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/German-special-Characters-incompatible-character-encodings-CP850-and-UTF-8-tp175p177.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML
... [show rest of quote]



--
Dan Allen
Principal Software Engineer, Red Hat | Author of Seam in Action
Registered Linux User #231597
Free forum by Nabble Edit this page