Does Asciidoctor silently abondon docbook?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Does Asciidoctor silently abondon docbook?

getreu
The question - I would like to discuss with you -  came up in this  thread:
>> Since asciidoctor-fopub is not actively maintained, I would recommend using asciidoctor-pdf instead to generate PDF directly from asciidoc documents.

>I can not. For academic writing I need footnotes and hyphenation.

---

It is true that `asciidoctor-fopub` is neglected by our community.
I fully agree with @mojavelinux  saying: 

"`If the need arises, you can make full use of the huge choice of tools available for a DocBook workflow using Asciidoctor’s DocBook converter. That’s why mapping to an enterprise documentation format like DocBook remains a key use case for AsciiDoc.`"

My personal opinon:
As `asciidotor-fopub` alone keeps our HTML5  backend and `asciidoctor-pdf` in sync with docbook, we definitely should find a maintainer for it. I can contribute my style changes to get the `asciidoctor-fopub` output closer to the HTML5, but my xml knowledge is very limited. Anyway someone needs to interface with the docbook community and defend docbook standards in the asciidoctor community.

What do you think?
Reply | Threaded
Open this post in threaded view
|

Re: Does Asciidoctor silently abondon docbook?

getreu
This is a copy of  comments posted here:

"It's important to keep in mind that technically, asciidoctor-fopub isn't needed. It's just a toolkit that wraps the DocBook toolchain. There are other toolchain interfaces available, including a2x from AsciiDoc Python. asciidoctor-fopub was an experiment to create a toolkit that only relied on Java. I was able to demonstrate that is possible. We needed that before Asciidoctor PDF existed. Then Asciidoctor PDF happened. But that doesn't mean it has to be the only path to creating a PDF. If you take the DocBook files, you can feed them to any toolkit that runs DocBook. That's why I'm not convinced maintaining asciidoctor-pdf is the right effort. Perhaps it needs to evolve. If someone wants to run with it, I'd certainly support that.

And yes, I will get footnotes working in Asciidoctor PDF. It just takes time to figure out and implement. Hyphens can be done today via an extension. On a rainy afternoon, I might find time to contribute that the extensions lab.

Keep in mind that fopub accepts xsl parameters. You can pass additional parameters such as this one to override what is set by default."

---

"@getreu, I understand your needs, since I am also a asciidoctor-fopub user myself. But I agree with @mojavelinux that asciidoctor-fopub isn't needed technically. If you like, you can try pandoc to generate pdf from docbook produced by asciidoctor. It shall work, but I have not tried that.

I use the docbook backend mainly for Word document export (asciidoc -(asciidoctor)-> docbook -(pandoc)-> Word docx). I use fopub to generate pdf with CJK fonts before asciidoctor-pdf-cjk was born."

Reply | Threaded
Open this post in threaded view
|

Re: Does Asciidoctor silently abondon docbook?

getreu
mojavelinux wrote
"It's important to keep in mind that technically, asciidoctor-fopub
isn't needed. It's just a toolkit that wraps the DocBook toolchain.
Alain,

I believe `asciidoctor-fopub` has an important role for asciidoc:

1. It keeps our back-ends in compliance with docbook!

        Lemma: A compliant back-end produces the same output, than the
        adoc->docbook->html/ebook/pdf workflow.
       
        This allows then to test to what extend a certain asciidoc semantic
        is rendered correctly. An example of non-compliance is the
        rendering of xref's in the html5 back end.
   
2. Only as long as our back-ends stay compliant (according to the above
    definition) asciidoctor integrates with the docbook ecosystem.
   
3. Concerning the importance of integration with docbook I cite you:

   "`If the need arises, you can make full use of the huge choice of
   tools available for a DocBook workflow using Asciidoctor’s DocBook
   converter. That’s why mapping to an enterprise documentation format
   like DocBook remains a key use case for AsciiDoc.`"
 
   We want to establish asciidoctor in enterprise technical
   documentation? Real Docbook compliance is the key to it.

With this in mind I sent you a pull request for  `asciidoctor-fopub`.
I reproduced the `asciidoctor -b html5` output. Both can now be used
in parallel. Please have a look.

Best regards


Jens
Reply | Threaded
Open this post in threaded view
|

Re: Does Asciidoctor silently abondon docbook?

mojavelinux
Administrator
In reply to this post by getreu

On Sat, Sep 17, 2016 at 1:51 PM, getreu [via Asciidoctor :: Discussion] <[hidden email]> wrote:
That's why I'm not convinced maintaining asciidoctor-pdf is the right effort

Correction. I meant to say asciidoctor-fopub here. I am maintaining Asciidoctor PDF.

-Dan


--
Dan Allen | @mojavelinux | https://twitter.com/mojavelinux
Reply | Threaded
Open this post in threaded view
|

Re: Does Asciidoctor silently abondon docbook?

mojavelinux
Administrator
In reply to this post by getreu
Here's my short answer to the question in $subject. No, we have not silently abandoned DocBook.

I hold to my viewpoint that DocBook output for AsciiDoc (and Asciidoctor) is critical because it serves as a good exchange and export format without any loss of information. (I think Mallard serves this purpose equally well, but is not accepted as input in as many places).

Being able to produce DocBook and being able to convert DocBook to other formats are entirely separate matters. While we are committed to producing DocBook, it's not our focus to be able to convert DocBook. That's the focus of a DocBook toolchain (and why there is a whole other project for that).

It's true that AsciiDoc Python ships with a DocBook toolchain wrapper / interface called a2x. In addition to setting up the DocBook toolchain, that interface also provided a few XSL patches to bridge the AsciiDoc and DocBook worlds (e.g., a hard line break) and better default styles. {1}

Of course, we need to maintain compliance with DocBook. That's why we have a test suite. We can address compliance in DocBook output without having to take on converting that output.

In conclusion, are a2x and asciidoctor-fopub needed and where do they fit? I do think they are needed because the DocBook toolchain has terrible defaults and is extremely difficult to launch. So it makes complete sense to have a easy to use, customizable interface that provides a nice default style. I'm 100% in support of someone maintaining such a project, perhaps by continuing asciidoctor-fopub. But conversion of DocBook is not my focus and it's not the prime focus of Asciidoctor. If someone wants to lead it, I'll support them. If someone wants to go off and do a totally separate project, I'd be happy to promote that effort.

I'm committed to DocBook compliance. I'm not, however, going to spend my time wrestling with the DocBook toolchain because I don't personally enjoy it.

Cheers,

-Dan

:1: I want to mention that I think these patches should make their way into DocBook so that we don't need them, including a hard line break, a horizontal rule, a page break, and a switch to enable/disable the toc.

--
Dan Allen | @mojavelinux | https://twitter.com/mojavelinux
Reply | Threaded
Open this post in threaded view
|

Re: Does Asciidoctor silently abondon docbook?

getreu
`asciidoctor-fopub` can be considered as a DocBook toolchain wrapper that ships an asciidoctor theme. It is pain-free (for the end user) and it is much more convenient then setting up `a2x`!

I hope we find a docbook-fan who is into these 1000 of parameters. Maybe somebody reads this post?

Unfortunately I am not an docbook expert and I do not have the time to become one in the near future. As I am using it every day, I can offer my contribution in keeping `asciidoctor-fopub` in sync with the rest of the ecosystem by updating its defaults.

Please follow also related discussions on:
 https://github.com/asciidoctor/asciidoctor-fopub/pull/62
https://github.com/asciidoctor/asciidoctor-bibtex/issues/15
Reply | Threaded
Open this post in threaded view
|

Re: Does Asciidoctor silently abondon docbook?

getreu
In reply to this post by mojavelinux
Question:

> Of course, we need to maintain compliance with DocBook. That's why we have
a test suite. We can address compliance in DocBook output without having to
take on converting that output.

INPUT                                                                                        OUTPUT A
Semantics(adoc) ---> Semantics (xml) ---------[docboc]----> presentation(pdf/html/ebook)

For me compliance mean:
INPUT                                                                                         OUTPUT B
Same semantics (adoc) ------[asciidoctor-xzy]-----> Same presentation(pdf/html/ebook)

How can your evaluate compliance of [asciidoctor-xzy] without comparing OUTPUT B to OUTPUT A???
Reply | Threaded
Open this post in threaded view
|

Re: Does Asciidoctor silently abondon docbook?

apnadkarni
Not that I have a say in the matter :-) but as a *user* I do not see why output produced by the asciidoctor-* tools be the same as output produced by docbook. Moreover, I would bet most users would not either. DocBook compliance to me would be exactly what Dan said - asciidoctor should produce output that is accepted by DocBook with the correct semantics.

And as an aside, I would say I much prefer the visual look of the asciidoctor tools to that of DocBook (considering defaults of course)
Reply | Threaded
Open this post in threaded view
|

Re: Does Asciidoctor silently abondon docbook?

getreu
It depends what kind of end-user you are!
Those who want to extend an existing DocBook workflow with asciidoctor, need OUTPUT A / OUTPUT B
content equivalence.
Reply | Threaded
Open this post in threaded view
|

Re: Does Asciidoctor silently abondon docbook?

eskwayrd
In reply to this post by getreu
getreu wrote
How can your evaluate compliance of [asciidoctor-xzy] without comparing OUTPUT B to OUTPUT A???

DocBook, as an XML format, has a DTD. DocBook documents can be validated against the DTD, so you can know they are correct without performing any transformations to reader-consumable output formats.

To produce identical output would require asciidoctor-fopub to understand the configuration and customizations applied within an alternate DocBook toolchain, which is entirely unnecessary for producing valid DocBook. For example, is compliance broken just because asciidoctor-fopub cannot use oXygen XML's transformation configurations (or vice-versa)? No.

Reply | Threaded
Open this post in threaded view
|

Re: Does Asciidoctor silently abondon docbook?

eskwayrd
In reply to this post by getreu
getreu wrote
Those who want to extend an existing DocBook workflow with asciidoctor, need OUTPUT A / OUTPUT B content equivalence.

This kind of compliance (which is not DocBook DTD compliance) is relatively easy to determine:

  1. Generate OUTPUTA from your existing DocBook doc source.
  2. Convert your doc source to Asciidoctor (there can be struggles achieving this part).
  3. Run asciidoctor -b docbook and feed its output into your existing toolchain to produce OUTPUTB.
  4. Run diff OUTPUTA OUTPUTB.
  5. Fixup any markup issues you find, and regenerate OUTPUTB is necessary.
  6. Wash/rinse/repeat until outputs are identical.

Note: some diligence might be required. In some scenarios, such as when output is chunked, DocBook auto-generates xml:id's that can differ from transformation to transformation. You may need to edit your DocBook doc source to eliminate any variance in its output in order to achieve identical output from AsciiDoctor format documents.

At $work, I undertook this exercise in the last year, and we now have 2k-3k pages of technical documentation converted to AsciiDoctor, and we verified that for guides with no content updates that our output was indeed identical. Our writers much prefer to work in AsciiDoctor format (compared to DocBook), aside from a few quibbles (AsciidocFX's live preview helps sort those out quickly). We did have to rewrite select chunks that depended on nested tables, or other formatting scenarios not supported in AsciiDoctor, but those were rare.

Reply | Threaded
Open this post in threaded view
|

Re: Does Asciidoctor silently abondon docbook?

getreu
Thank you for sharing your workflow.

After converting your documents and while still using your existing toolchain producing OUTPUTA you will soon discover that the html 5 back-end produces very nice renditions OUTPUTB. Let's assume you want to use this back-end in addition to your existing tool-chain. Lets compare:

OUTPUTA:  The Figure 3, "The search target mapping" shows ...
OUTPUTB: The The search target mapping shows ...

Concerning the example I chose here, the xref feature of the back-end html5 is not compatible with the docbook toolchain.  The html5 back-end is excellent for many reasons, but you just can not use it!

This is why I think we should define "docbook compatibility" in a stricter way:
1. output can be validated against DTD (this is what we do)
2. content representation equivalence (this is what we should do in addition, see my example above)
Reply | Threaded
Open this post in threaded view
|

Re: Does Asciidoctor silently abondon docbook?

eskwayrd
getreu wrote
After converting your documents and while still using your existing toolchain producing OUTPUTA you will soon discover that the html 5 back-end produces very nice renditions OUTPUTB.

That may be a problem for some, but it is not a problem for us. Our DocBook toolchain is used to produce HTML, PDF, CHM, and I've been working on EPUB output. We saw zero difference in how xrefs worked when switching from DocBook source to AsciiDoctor source.

getreu wrote
This is why I think we should define "docbook compatibility" in a stricter way:
1. output can be validated against DTD (this is what we do)
2. content representation equivalence (this is what we should do in addition, see my example above)

You are already doing as much "strict docbook compatibility" as you can with DTD validation. Everything else is transformation conformance, which is a different layer of the problem and is highly toolchain dependent. There are too many factors outside of the AsciiDoctor devs' control to make #2 happen, realistically.

Reply | Threaded
Open this post in threaded view
|

Candidate definition for "transformation conformance

getreu
> Everything else is transformation conformance, which is a different layer of the problem and is highly toolchain dependent. There are too many factors outside of the AsciiDoctor devs' control to make #2 happen, realistically.

Suggestion:  definition of "transformation conformance":
In order to manifest its full potential, every back-end should produce "similar", compatible output. We could define similarity as "the same graphical characters, i.e. all Unicode code points in the body must be present and in the same order."

But what back-end should be the reference? I believe it should be asciidoctor-fopub because it is our
gateway to docbook and professional applications.

We are actually pretty close already, I could only find very few differences. Unfortunately each of them is exclusionary when you want to use all the back-ends together.
Reply | Threaded
Open this post in threaded view
|

Re: Candidate definition for "transformation conformance

eskwayrd
getreu wrote
Suggestion: definition of "transformation conformance": In order to manifest its full potential, every back-end should produce "similar", compatible output.

The inclusion of "compatible" seems wrong here. HTML and PDF are common outputs that are entirely incompatible, and my toolchain can produce both. I would suggest altering this part of the definition to be something like "every back-end should consistently produce output that represents the content as faithfully as possible." One of my back-burner projects is to support EPUB3 outputs, and with text-to-speech technology being as good as it is, I can see auto-generating audio-only versions, and versions with read-along (or karaoke) extensions within a single EPUB3 file.

getreu wrote
We could define similarity as "the same graphical characters, i.e. all Unicode code points in the body must be present and in the same order."

I agree. Incorrect handling of Unicode would be a notable bug. Even if you get the Unicode correct, one backend not having access to the correct typeface can entirely defeat a translation effort.

getreu wrote
But what back-end should be the reference? I believe it should be asciidoctor-fopub because it is our gateway to docbook and professional applications.

asciidoctor-fopub could be a reference back-end, but remember that it is one of many. I don't use it at all and my ability to publish HTML and PDF is not diminished in the slightest. Also, it is definitely not a "gateway to docbook"; it is a DocBook consumer that helps transform DocBook to FO and then to PDF. asciidoctor -b docbook is the "gateway" to DocBook. To what "professional applications" are you referring?

getreu wrote
We are actually pretty close already, I could only find very few differences. Unfortunately each of them is exclusionary when you want to use all the back-ends together.

I'm afraid I have no idea what uslng "all the back-ends together" means. Perhaps you're referring just to the AsciiDoctor backends that currently exist? My frame of reference in this discussion is about the transformation toolchains that consume DocBook, of which there are many.

Reply | Threaded
Open this post in threaded view
|

Re: Does Asciidoctor silently abondon docbook?

mojavelinux
Administrator
In reply to this post by getreu

On Sun, Sep 18, 2016 at 1:15 AM, getreu [via Asciidoctor :: Discussion] <[hidden email]> wrote:
For me compliance mean:
INPUT                                                                                         OUTPUT B
Same semantics (adoc) ------[asciidoctor-xzy]-----> Same presentation(pdf/html/ebook)

This is where we disagree. I do not consider the pixel-perfect presentation to be part of compliance (because it's not our responsibility to determine how the DocBook toolchain is executed or configured).

There is another question about whether the presentation output of Asciidoctor PDF and EPUB3 should match the presentation output of the DocBook toolchain. But even here, I don't think they should. It's too constraining. In fact, I don't think it even makes sense to talk about compliance of presentation since the whole idea of separating content and presentation is that you can style it (aka present it) however you want.

If someone wants to lead an effort to align the presentation of all the formats, they certainly are welcome to. I have no doubt some users, such as yourself, will find utility in that. That's great. But it's not my goal. I want to encourage freedom and diversity of expression.

Understand that I don't think our goals are at odds, just different paths that embark from core.

-Dan

--
Dan Allen | @mojavelinux | https://twitter.com/mojavelinux
Reply | Threaded
Open this post in threaded view
|

Re: Does Asciidoctor silently abondon docbook?

mojavelinux
Administrator
In reply to this post by getreu

On Wed, Sep 21, 2016 at 8:19 PM, getreu [via Asciidoctor :: Discussion] <[hidden email]> wrote:
 The html5 back-end is excellent for many reasons, but you just can not use it!

While it's true the built-in HTML5 converter does not have some of the formalities out of the box yet, like qualified section references, please keep in mind that every character of HTML that the built-in converter produces can be customized. You can provide a custom template for inline_anchor and change how these are written.

I agree it would be nice to have an option to turn on qualified section references by default. We'll get there. But as it stands now, the capability is not out of reach. You also have the option of just replacing the whole converter (which is basically what the Reveal.js and Bespoke.js converters do).

Cheers,

-Dan


--
Dan Allen | @mojavelinux | https://twitter.com/mojavelinux
Reply | Threaded
Open this post in threaded view
|

Re: Does Asciidoctor silently abondon docbook?

mojavelinux
Administrator
In reply to this post by getreu

On Wed, Sep 21, 2016 at 8:19 PM, getreu [via Asciidoctor :: Discussion] <[hidden email]> wrote:
2. content representation equivalence (this is what we should do in addition, see my example above)

I agree that someone could do this. This is an open source effort, so we encourage efforts, we don't shut them down. I'm just not going to define it as a goal of the core project. My efforts may even deviate from this goal (in other words, I may explore taking the presentation in a different direction in Asciidoctor PDF instead of trying to align them..."breaking the mold" so to speak).

-Dan


--
Dan Allen | @mojavelinux | https://twitter.com/mojavelinux
Reply | Threaded
Open this post in threaded view
|

Re: Does Asciidoctor silently abondon docbook?

mojavelinux
Administrator
In reply to this post by eskwayrd

On Thu, Sep 22, 2016 at 2:11 PM, eskwayrd [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Everything else is transformation conformance, which is a different layer of the problem and is highly toolchain dependent.

I'm with @eskwayrd on this one.

That's not to say that I don't care about consistency of presentation. It's just not something we want to enforce. I think there can be a project that has that goal. But that's not everyone's goal. Some people may want vastly different output that conforms more to the medium than some ideal archetype.

The reason I'm pushing back on this is because I feel like the goal of "presentation equivalence" is too imposing for this ecosystem. What I want this ecosystem to be able to do is enable that sort of goal, but not try to funnel everyone into it. I think enablement (making it possible to achieve) is the right goal.

Cheers,

-Dan


--
Dan Allen | @mojavelinux | https://twitter.com/mojavelinux
Reply | Threaded
Open this post in threaded view
|

Re: Candidate definition for "transformation conformance

mojavelinux
Administrator
In reply to this post by getreu

On Fri, Sep 23, 2016 at 6:39 AM, getreu [via Asciidoctor :: Discussion] <[hidden email]> wrote:
But what back-end should be the reference? I believe it should be asciidoctor-fopub because it is our
gateway to docbook and professional applications.

I just want to point out here that asciidoctor-fopub is not a backend. It's a frontend to the DocBook toolchain (as @eskwayrd pointed out). I don't even think the output it produces is really all that nice, and it's not so easy to make nice (trust me, I tried).

If there's any reference, it would be a native Asciidoctor converter such as the HTML5 converter or the PDF converter. Those are converters we have control over.

Having said all that, I do think we can have a conversation about a common theme effort that gets applied to various converters. In fact, there is an open issue in Asciidoctor PDF to make an "Asciidoctor" theme. https://github.com/asciidoctor/asciidoctor-pdf/issues/60 The problem, however, is that there's really no such thing yet. While default stylesheet for the HTML output presents a familiar Asciidoctor look and feel, that's never really been vetted.

I could certainly get behind an effort to define a common/shared theme and then make sure each of the converters / toolchains has the ability to be configured to implement that theme. In my mind, that's more about an effort to improve the theming engine for each converter / toolchain. Certainly the theme engine in Asciidoctor PDF needs more control.

Cheers,

-Dan


--
Dan Allen | @mojavelinux | https://twitter.com/mojavelinux
12