Renderer to Converter metamorphosis

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Renderer to Converter metamorphosis

mojavelinux
Administrator
I just sent the largest pull request yet to the Asciidoctor repository that includes a complete rewrite of the renderers, which are now called converters.

https://github.com/asciidoctor/asciidoctor/pull/906

Renderers are dead, long live extensible Converters, as the saying goes.

The lack of extensibility in the rendering layer became apparent to me while working on Asciidoctor PDF. The Tilt-supported templates are certainly useful, but suffer from a rather sever bias that they are generating a single text file or stream. Formats such as PDF and ePub don't work in this model. So we needed to have a way to still be able to use Ruby or Java directly to convert the document. Now we do :)

Since we aren't just rendering templates anymore, I decided it's more appropriate to call these objects converters. I've introduced a Converter "interface" and a factory object responsible for selecting and instantiating the implementations.

Here's the hierarchy:

....
Converter (a Ruby module)
  BaseConverter (an abstract Ruby class)
    CompositeConverter (delegates to a chain of converters)
    TemplateConverter (delegates conversion to Tilt-supported templates)
  BuiltInConverter
    Html5Converter (used to convert documents to the html5 backend format)
    DocBook45Converter (used to convert documents to the docbook45 backend format)
    DocBook5Converter (used to convert documents to the docbook5 backend format)
....

If you use a built-in backend, such as html5, then the corresponding BuiltInConverter is used. If you specify a template directory, a CompositeConverter is used that delegates first to the TemplateConverter, then to the BuiltInConverter. Functionally, this is how Asciidoctor always worked. But it was a complete hack before. Now it works in a very logical and transparent way.

If you want to create a custom converter in Ruby, it's very straightforward. Here's an example of a very rough text-only output converter.

```ruby

class TextConverter

  # mix in the converter interface
  include Asciidoctor::Converter

  # associate this converter with the text backend
  register_for 'text'

  # convert each node in the tree
  # recursion is triggered by calls to the #content method
  def convert node, transform = nil
    case (transform ||= node.node_name)
    when 'document'
      node.content
    when 'section'
      [node.title, node.content] * "\n\n"
    when 'paragraph'
      node.content.tr("\n", ' ') << "\n"
    else
      if node.inline?
        node.text
      else
        %(TODO: #{transform}\n)
      end
    end
  end
end

```

Here's how it's used:

```ruby

puts Asciidoctor.convert_file 'sample.adoc', backend: :text

```

Just like with extensions, the converter is registered globally by default. If you want to use the converter explicitly without registering it, you remove the "register_for" line and pass either the class or instance as an option to the API:

```ruby

puts Asciidoctor.convert_file 'sample.adoc', backend: :text, converter: TextConverter

```

You'll notice I'm using "convert" in the API calls instead of "render". Both names are still supported, but "convert" is the terminology I recommend from this point forward.

I documented the API thoroughly, so once the pull request is merged, you'll be able to view the docs at the following URL:

http://rubydoc.info/github/asciidoctor/asciidoctor/frames

Other benefits include more speed (oh yes, always more speed) and a working table of contents in Asciidoctor.js :)

I'll publish a preview3 release asap so you can try it out.

If you have questions or comments, please don't hesitate to ask.
Reply | Threaded
Open this post in threaded view
|

Re: Renderer to Converter metamorphosis

asotobu
That's amazing only two questions:

* Is it time for writing converters in Java? Personally I would wait until 1.6.0 but of course if you think interesting I would work for next release.

* Should I deprecated render and render_file method?

When you release the preview.3 I will start all tests to see if there is some problem.

Reply | Threaded
Open this post in threaded view
|

Re: Renderer to Converter metamorphosis

LightGuardjp
In reply to this post by mojavelinux
Unless I'm mistaken, this should more easily allow us to create an AsciiDoc converter! That would mean we could use the AST methods to programmatically create a document and then have it kick out the AsciiDoc text :) 

Sent from Mailbox for iPhone


On Wed, Feb 26, 2014 at 1:41 AM, mojavelinux [via Asciidoctor :: Discussion] <[hidden email]> wrote:

I just sent the largest pull request yet to the Asciidoctor repository that includes a complete rewrite of the renderers, which are now called converters.

https://github.com/asciidoctor/asciidoctor/pull/906

Renderers are dead, long live extensible Converters, as the saying goes.

The lack of extensibility in the rendering layer became apparent to me while working on Asciidoctor PDF. The Tilt-supported templates are certainly useful, but suffer from a rather sever bias that they are generating a single text file or stream. Formats such as PDF and ePub don't work in this model. So we needed to have a way to still be able to use Ruby or Java directly to convert the document. Now we do :)

Since we aren't just rendering templates anymore, I decided it's more appropriate to call these objects converters. I've introduced a Converter "interface" and a factory object responsible for selecting and instantiating the implementations.

Here's the hierarchy:

....
Converter (a Ruby module)
  BaseConverter (an abstract Ruby class)
    CompositeConverter (delegates to a chain of converters)
    TemplateConverter (delegates conversion to Tilt-supported templates)
  BuiltInConverter
    Html5Converter (used to convert documents to the html5 backend format)
    DocBook45Converter (used to convert documents to the docbook45 backend format)
    DocBook5Converter (used to convert documents to the docbook5 backend format)
....

If you use a built-in backend, such as html5, then the corresponding BuiltInConverter is used. If you specify a template directory, a CompositeConverter is used that delegates first to the TemplateConverter, then to the BuiltInConverter. Functionally, this is how Asciidoctor always worked. But it was a complete hack before. Now it works in a very logical and transparent way.

If you want to create a custom converter in Ruby, it's very straightforward. Here's an example of a very rough text-only output converter.

```ruby

class TextConverter

  # mix in the converter interface
  include Asciidoctor::Converter

  # associate this converter with the text backend
  register_for 'text'

  # convert each node in the tree
  # recursion is triggered by calls to the #content method
  def convert node, transform = nil
    case (transform ||= node.node_name)
    when 'document'
      node.content
    when 'section'
      [node.title, node.content] * "\n\n"
    when 'paragraph'
      node.content.tr("\n", ' ') << "\n"
    else
      if node.inline?
        node.text
      else
        %(TODO: #{transform}\n)
      end
    end
  end
end

```

Here's how it's used:

```ruby

puts Asciidoctor.convert_file 'sample.adoc', backend: :text

```

Just like with extensions, the converter is registered globally by default. If you want to use the converter explicitly without registering it, you remove the "register_for" line and pass either the class or instance as an option to the API:

```ruby

puts Asciidoctor.convert_file 'sample.adoc', backend: :text, converter: TextConverter

```

You'll notice I'm using "convert" in the API calls instead of "render". Both names are still supported, but "convert" is the terminology I recommend from this point forward.

I documented the API thoroughly, so once the pull request is merged, you'll be able to view the docs at the following URL:

http://rubydoc.info/github/asciidoctor/asciidoctor/frames

Other benefits include more speed (oh yes, always more speed) and a working table of contents in Asciidoctor.js :)

I'll publish a preview3 release asap so you can try it out.

If you have questions or comments, please don't hesitate to ask.



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML

Reply | Threaded
Open this post in threaded view
|

Re: Renderer to Converter metamorphosis

mojavelinux
Administrator
In reply to this post by asotobu

Alex,

Thanks!

We could certainly start experimenting with Converters in Java now, but we might decide it best to label it as experimental until 1.6.0.

Yes, the render methods should be deprecated, but kept for backwards compatibility.

I'll merge later today and do the preview3 release shortly after.

-Dan

On Feb 26, 2014 11:07 AM, "asotobu [via Asciidoctor :: Discussion]" <[hidden email]> wrote:
That's amazing only two questions:

* Is it time for writing converters in Java? Personally I would wait until 1.6.0 but of course if you think interesting I would work for next release.

* Should I deprecated render and render_file method?

When you release the preview.3 I will start all tests to see if there is some problem.




If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1546.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML
Reply | Threaded
Open this post in threaded view
|

Re: Renderer to Converter metamorphosis

mojavelinux
Administrator
In reply to this post by LightGuardjp

Jason,

Absolutely correct. In fact, I was thinking of doing a Kramdown to Asciidoctor AST transformation and vice-versa to demonstrate.

Probably time for an asciidoctor-converter-lab repo :)

-Dan

On Feb 26, 2014 11:55 AM, "LightGuardjp [via Asciidoctor :: Discussion]" <[hidden email]> wrote:
Unless I'm mistaken, this should more easily allow us to create an AsciiDoc converter! That would mean we could use the AST methods to programmatically create a document and then have it kick out the AsciiDoc text :) 

Sent from Mailbox for iPhone


On Wed, Feb 26, 2014 at 1:41 AM, mojavelinux [via Asciidoctor :: Discussion] <[hidden email]> wrote:

I just sent the largest pull request yet to the Asciidoctor repository that includes a complete rewrite of the renderers, which are now called converters.

https://github.com/asciidoctor/asciidoctor/pull/906

Renderers are dead, long live extensible Converters, as the saying goes.

The lack of extensibility in the rendering layer became apparent to me while working on Asciidoctor PDF. The Tilt-supported templates are certainly useful, but suffer from a rather sever bias that they are generating a single text file or stream. Formats such as PDF and ePub don't work in this model. So we needed to have a way to still be able to use Ruby or Java directly to convert the document. Now we do :)

Since we aren't just rendering templates anymore, I decided it's more appropriate to call these objects converters. I've introduced a Converter "interface" and a factory object responsible for selecting and instantiating the implementations.

Here's the hierarchy:

....
Converter (a Ruby module)
  BaseConverter (an abstract Ruby class)
    CompositeConverter (delegates to a chain of converters)
    TemplateConverter (delegates conversion to Tilt-supported templates)
  BuiltInConverter
    Html5Converter (used to convert documents to the html5 backend format)
    DocBook45Converter (used to convert documents to the docbook45 backend format)
    DocBook5Converter (used to convert documents to the docbook5 backend format)
....

If you use a built-in backend, such as html5, then the corresponding BuiltInConverter is used. If you specify a template directory, a CompositeConverter is used that delegates first to the TemplateConverter, then to the BuiltInConverter. Functionally, this is how Asciidoctor always worked. But it was a complete hack before. Now it works in a very logical and transparent way.

If you want to create a custom converter in Ruby, it's very straightforward. Here's an example of a very rough text-only output converter.

```ruby

class TextConverter

  # mix in the converter interface
  include Asciidoctor::Converter

  # associate this converter with the text backend
  register_for 'text'

  # convert each node in the tree
  # recursion is triggered by calls to the #content method
  def convert node, transform = nil
    case (transform ||= node.node_name)
    when 'document'
      node.content
    when 'section'
      [node.title, node.content] * "\n\n"
    when 'paragraph'
      node.content.tr("\n", ' ') << "\n"
    else
      if node.inline?
        node.text
      else
        %(TODO: #{transform}\n)
      end
    end
  end
end

```

Here's how it's used:

```ruby

puts Asciidoctor.convert_file 'sample.adoc', backend: :text

```

Just like with extensions, the converter is registered globally by default. If you want to use the converter explicitly without registering it, you remove the "register_for" line and pass either the class or instance as an option to the API:

```ruby

puts Asciidoctor.convert_file 'sample.adoc', backend: :text, converter: TextConverter

```

You'll notice I'm using "convert" in the API calls instead of "render". Both names are still supported, but "convert" is the terminology I recommend from this point forward.

I documented the API thoroughly, so once the pull request is merged, you'll be able to view the docs at the following URL:

http://rubydoc.info/github/asciidoctor/asciidoctor/frames

Other benefits include more speed (oh yes, always more speed) and a working table of contents in Asciidoctor.js :)

I'll publish a preview3 release asap so you can try it out.

If you have questions or comments, please don't hesitate to ask.



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML




If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1547.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML
Reply | Threaded
Open this post in threaded view
|

Re: Renderer to Converter metamorphosis

LightGuardjp
Now  things are getting really interesting :) as soon as we get things running on nashorn you'll have full asciidoctor in one slim jar in Java. You could do asciidoclet easily, really all sorts of things :)

Sent from Mailbox for iPhone


On Wed, Feb 26, 2014 at 3:37 PM, mojavelinux [via Asciidoctor :: Discussion] <[hidden email]> wrote:

Jason,

Absolutely correct. In fact, I was thinking of doing a Kramdown to Asciidoctor AST transformation and vice-versa to demonstrate.

Probably time for an asciidoctor-converter-lab repo :)

-Dan

On Feb 26, 2014 11:55 AM, "LightGuardjp [via Asciidoctor :: Discussion]" <[hidden email]> wrote:
Unless I'm mistaken, this should more easily allow us to create an AsciiDoc converter! That would mean we could use the AST methods to programmatically create a document and then have it kick out the AsciiDoc text :) 

Sent from Mailbox for iPhone


On Wed, Feb 26, 2014 at 1:41 AM, mojavelinux [via Asciidoctor :: Discussion] <[hidden email]> wrote:

I just sent the largest pull request yet to the Asciidoctor repository that includes a complete rewrite of the renderers, which are now called converters.

https://github.com/asciidoctor/asciidoctor/pull/906

Renderers are dead, long live extensible Converters, as the saying goes.

The lack of extensibility in the rendering layer became apparent to me while working on Asciidoctor PDF. The Tilt-supported templates are certainly useful, but suffer from a rather sever bias that they are generating a single text file or stream. Formats such as PDF and ePub don't work in this model. So we needed to have a way to still be able to use Ruby or Java directly to convert the document. Now we do :)

Since we aren't just rendering templates anymore, I decided it's more appropriate to call these objects converters. I've introduced a Converter "interface" and a factory object responsible for selecting and instantiating the implementations.

Here's the hierarchy:

....
Converter (a Ruby module)
  BaseConverter (an abstract Ruby class)
    CompositeConverter (delegates to a chain of converters)
    TemplateConverter (delegates conversion to Tilt-supported templates)
  BuiltInConverter
    Html5Converter (used to convert documents to the html5 backend format)
    DocBook45Converter (used to convert documents to the docbook45 backend format)
    DocBook5Converter (used to convert documents to the docbook5 backend format)
....

If you use a built-in backend, such as html5, then the corresponding BuiltInConverter is used. If you specify a template directory, a CompositeConverter is used that delegates first to the TemplateConverter, then to the BuiltInConverter. Functionally, this is how Asciidoctor always worked. But it was a complete hack before. Now it works in a very logical and transparent way.

If you want to create a custom converter in Ruby, it's very straightforward. Here's an example of a very rough text-only output converter.

```ruby

class TextConverter

  # mix in the converter interface
  include Asciidoctor::Converter

  # associate this converter with the text backend
  register_for 'text'

  # convert each node in the tree
  # recursion is triggered by calls to the #content method
  def convert node, transform = nil
    case (transform ||= node.node_name)
    when 'document'
      node.content
    when 'section'
      [node.title, node.content] * "\n\n"
    when 'paragraph'
      node.content.tr("\n", ' ') << "\n"
    else
      if node.inline?
        node.text
      else
        %(TODO: #{transform}\n)
      end
    end
  end
end

```

Here's how it's used:

```ruby

puts Asciidoctor.convert_file 'sample.adoc', backend: :text

```

Just like with extensions, the converter is registered globally by default. If you want to use the converter explicitly without registering it, you remove the "register_for" line and pass either the class or instance as an option to the API:

```ruby

puts Asciidoctor.convert_file 'sample.adoc', backend: :text, converter: TextConverter

```

You'll notice I'm using "convert" in the API calls instead of "render". Both names are still supported, but "convert" is the terminology I recommend from this point forward.

I documented the API thoroughly, so once the pull request is merged, you'll be able to view the docs at the following URL:

http://rubydoc.info/github/asciidoctor/asciidoctor/frames

Other benefits include more speed (oh yes, always more speed) and a working table of contents in Asciidoctor.js :)

I'll publish a preview3 release asap so you can try it out.

If you have questions or comments, please don't hesitate to ask.



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML




If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1547.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1550.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML

Reply | Threaded
Open this post in threaded view
|

Re: Renderer to Converter metamorphosis

asotobu
Yes. I have already a project with implementations of Nashorn, Rhino and DynJs (they have already fixed a bug caused by opal). So as soon as a new asciidoctor.js with opal.js is been released we could start the integration again with slim jar.

Now after some tests we can release a java version using js but is slower than the JRuby ones.


2014-02-26 23:41 GMT+01:00 LightGuardjp [via Asciidoctor :: Discussion] <[hidden email]>:
Now  things are getting really interesting :) as soon as we get things running on nashorn you'll have full asciidoctor in one slim jar in Java. You could do asciidoclet easily, really all sorts of things :)

Sent from Mailbox for iPhone


On Wed, Feb 26, 2014 at 3:37 PM, mojavelinux [via Asciidoctor :: Discussion] <[hidden email]> wrote:

Jason,

Absolutely correct. In fact, I was thinking of doing a Kramdown to Asciidoctor AST transformation and vice-versa to demonstrate.

Probably time for an asciidoctor-converter-lab repo :)

-Dan

On Feb 26, 2014 11:55 AM, "LightGuardjp [via Asciidoctor :: Discussion]" <[hidden email]> wrote:
Unless I'm mistaken, this should more easily allow us to create an AsciiDoc converter! That would mean we could use the AST methods to programmatically create a document and then have it kick out the AsciiDoc text :) 

Sent from Mailbox for iPhone


On Wed, Feb 26, 2014 at 1:41 AM, mojavelinux [via Asciidoctor :: Discussion] <[hidden email]> wrote:

I just sent the largest pull request yet to the Asciidoctor repository that includes a complete rewrite of the renderers, which are now called converters.

https://github.com/asciidoctor/asciidoctor/pull/906

Renderers are dead, long live extensible Converters, as the saying goes.

The lack of extensibility in the rendering layer became apparent to me while working on Asciidoctor PDF. The Tilt-supported templates are certainly useful, but suffer from a rather sever bias that they are generating a single text file or stream. Formats such as PDF and ePub don't work in this model. So we needed to have a way to still be able to use Ruby or Java directly to convert the document. Now we do :)

Since we aren't just rendering templates anymore, I decided it's more appropriate to call these objects converters. I've introduced a Converter "interface" and a factory object responsible for selecting and instantiating the implementations.

Here's the hierarchy:

....
Converter (a Ruby module)
  BaseConverter (an abstract Ruby class)
    CompositeConverter (delegates to a chain of converters)
    TemplateConverter (delegates conversion to Tilt-supported templates)
  BuiltInConverter
    Html5Converter (used to convert documents to the html5 backend format)
    DocBook45Converter (used to convert documents to the docbook45 backend format)
    DocBook5Converter (used to convert documents to the docbook5 backend format)
....

If you use a built-in backend, such as html5, then the corresponding BuiltInConverter is used. If you specify a template directory, a CompositeConverter is used that delegates first to the TemplateConverter, then to the BuiltInConverter. Functionally, this is how Asciidoctor always worked. But it was a complete hack before. Now it works in a very logical and transparent way.

If you want to create a custom converter in Ruby, it's very straightforward. Here's an example of a very rough text-only output converter.

```ruby

class TextConverter

  # mix in the converter interface
  include Asciidoctor::Converter

  # associate this converter with the text backend
  register_for 'text'

  # convert each node in the tree
  # recursion is triggered by calls to the #content method
  def convert node, transform = nil
    case (transform ||= node.node_name)
    when 'document'
      node.content
    when 'section'
      [node.title, node.content] * "\n\n"
    when 'paragraph'
      node.content.tr("\n", ' ') << "\n"
    else
      if node.inline?
        node.text
      else
        %(TODO: #{transform}\n)
      end
    end
  end
end

```

Here's how it's used:

```ruby

puts Asciidoctor.convert_file 'sample.adoc', backend: :text

```

Just like with extensions, the converter is registered globally by default. If you want to use the converter explicitly without registering it, you remove the "register_for" line and pass either the class or instance as an option to the API:

```ruby

puts Asciidoctor.convert_file 'sample.adoc', backend: :text, converter: TextConverter

```

You'll notice I'm using "convert" in the API calls instead of "render". Both names are still supported, but "convert" is the terminology I recommend from this point forward.

I documented the API thoroughly, so once the pull request is merged, you'll be able to view the docs at the following URL:

http://rubydoc.info/github/asciidoctor/asciidoctor/frames

Other benefits include more speed (oh yes, always more speed) and a working table of contents in Asciidoctor.js :)

I'll publish a preview3 release asap so you can try it out.

If you have questions or comments, please don't hesitate to ask.



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML




If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1547.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1550.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML




If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1551.html
To unsubscribe from Renderer to Converter metamorphosis, click here.
NAML



--
+----------------------------------------------------------+
  Alex Soto Bueno - Computer Engineer
  www.lordofthejars.com
+----------------------------------------------------------+
Reply | Threaded
Open this post in threaded view
|

Re: Renderer to Converter metamorphosis

LightGuardjp
How much slower are we talking? It may be an acceptable trade off for people (size vs speed).


On Thu, Feb 27, 2014 at 12:38 AM, asotobu [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Yes. I have already a project with implementations of Nashorn, Rhino and DynJs (they have already fixed a bug caused by opal). So as soon as a new asciidoctor.js with opal.js is been released we could start the integration again with slim jar.

Now after some tests we can release a java version using js but is slower than the JRuby ones.


2014-02-26 23:41 GMT+01:00 LightGuardjp [via Asciidoctor :: Discussion] <[hidden email]>:
Now  things are getting really interesting :) as soon as we get things running on nashorn you'll have full asciidoctor in one slim jar in Java. You could do asciidoclet easily, really all sorts of things :)

Sent from Mailbox for iPhone


On Wed, Feb 26, 2014 at 3:37 PM, mojavelinux [via Asciidoctor :: Discussion] <[hidden email]> wrote:

Jason,

Absolutely correct. In fact, I was thinking of doing a Kramdown to Asciidoctor AST transformation and vice-versa to demonstrate.

Probably time for an asciidoctor-converter-lab repo :)

-Dan

On Feb 26, 2014 11:55 AM, "LightGuardjp [via Asciidoctor :: Discussion]" <[hidden email]> wrote:
Unless I'm mistaken, this should more easily allow us to create an AsciiDoc converter! That would mean we could use the AST methods to programmatically create a document and then have it kick out the AsciiDoc text :) 

Sent from Mailbox for iPhone


On Wed, Feb 26, 2014 at 1:41 AM, mojavelinux [via Asciidoctor :: Discussion] <[hidden email]> wrote:

I just sent the largest pull request yet to the Asciidoctor repository that includes a complete rewrite of the renderers, which are now called converters.

https://github.com/asciidoctor/asciidoctor/pull/906

Renderers are dead, long live extensible Converters, as the saying goes.

The lack of extensibility in the rendering layer became apparent to me while working on Asciidoctor PDF. The Tilt-supported templates are certainly useful, but suffer from a rather sever bias that they are generating a single text file or stream. Formats such as PDF and ePub don't work in this model. So we needed to have a way to still be able to use Ruby or Java directly to convert the document. Now we do :)

Since we aren't just rendering templates anymore, I decided it's more appropriate to call these objects converters. I've introduced a Converter "interface" and a factory object responsible for selecting and instantiating the implementations.

Here's the hierarchy:

....
Converter (a Ruby module)
  BaseConverter (an abstract Ruby class)
    CompositeConverter (delegates to a chain of converters)
    TemplateConverter (delegates conversion to Tilt-supported templates)
  BuiltInConverter
    Html5Converter (used to convert documents to the html5 backend format)
    DocBook45Converter (used to convert documents to the docbook45 backend format)
    DocBook5Converter (used to convert documents to the docbook5 backend format)
....

If you use a built-in backend, such as html5, then the corresponding BuiltInConverter is used. If you specify a template directory, a CompositeConverter is used that delegates first to the TemplateConverter, then to the BuiltInConverter. Functionally, this is how Asciidoctor always worked. But it was a complete hack before. Now it works in a very logical and transparent way.

If you want to create a custom converter in Ruby, it's very straightforward. Here's an example of a very rough text-only output converter.

```ruby

class TextConverter

  # mix in the converter interface
  include Asciidoctor::Converter

  # associate this converter with the text backend
  register_for 'text'

  # convert each node in the tree
  # recursion is triggered by calls to the #content method
  def convert node, transform = nil
    case (transform ||= node.node_name)
    when 'document'
      node.content
    when 'section'
      [node.title, node.content] * "\n\n"
    when 'paragraph'
      node.content.tr("\n", ' ') << "\n"
    else
      if node.inline?
        node.text
      else
        %(TODO: #{transform}\n)
      end
    end
  end
end

```

Here's how it's used:

```ruby

puts Asciidoctor.convert_file 'sample.adoc', backend: :text

```

Just like with extensions, the converter is registered globally by default. If you want to use the converter explicitly without registering it, you remove the "register_for" line and pass either the class or instance as an option to the API:

```ruby

puts Asciidoctor.convert_file 'sample.adoc', backend: :text, converter: TextConverter

```

You'll notice I'm using "convert" in the API calls instead of "render". Both names are still supported, but "convert" is the terminology I recommend from this point forward.

I documented the API thoroughly, so once the pull request is merged, you'll be able to view the docs at the following URL:

http://rubydoc.info/github/asciidoctor/asciidoctor/frames

Other benefits include more speed (oh yes, always more speed) and a working table of contents in Asciidoctor.js :)

I'll publish a preview3 release asap so you can try it out.

If you have questions or comments, please don't hesitate to ask.



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML




If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1547.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1550.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML




If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1551.html
To unsubscribe from Renderer to Converter metamorphosis, click here.
NAML



--
+----------------------------------------------------------+
  Alex Soto Bueno - Computer Engineer
  www.lordofthejars.com
+----------------------------------------------------------+



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1554.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



--
Reply | Threaded
Open this post in threaded view
|

Re: Renderer to Converter metamorphosis

asotobu


2014-02-27 8:41 GMT+01:00 LightGuardjp [via Asciidoctor :: Discussion] <[hidden email]>:
How much slower are we talking? It may be an acceptable trade off for people (size vs speed).


On Thu, Feb 27, 2014 at 12:38 AM, asotobu [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Yes. I have already a project with implementations of Nashorn, Rhino and DynJs (they have already fixed a bug caused by opal). So as soon as a new asciidoctor.js with opal.js is been released we could start the integration again with slim jar.

Now after some tests we can release a java version using js but is slower than the JRuby ones.


2014-02-26 23:41 GMT+01:00 LightGuardjp [via Asciidoctor :: Discussion] <[hidden email]>:

Now  things are getting really interesting :) as soon as we get things running on nashorn you'll have full asciidoctor in one slim jar in Java. You could do asciidoclet easily, really all sorts of things :)

Sent from Mailbox for iPhone


On Wed, Feb 26, 2014 at 3:37 PM, mojavelinux [via Asciidoctor :: Discussion] <[hidden email]> wrote:

Jason,

Absolutely correct. In fact, I was thinking of doing a Kramdown to Asciidoctor AST transformation and vice-versa to demonstrate.

Probably time for an asciidoctor-converter-lab repo :)

-Dan

On Feb 26, 2014 11:55 AM, "LightGuardjp [via Asciidoctor :: Discussion]" <[hidden email]> wrote:
Unless I'm mistaken, this should more easily allow us to create an AsciiDoc converter! That would mean we could use the AST methods to programmatically create a document and then have it kick out the AsciiDoc text :) 

Sent from Mailbox for iPhone


On Wed, Feb 26, 2014 at 1:41 AM, mojavelinux [via Asciidoctor :: Discussion] <[hidden email]> wrote:

I just sent the largest pull request yet to the Asciidoctor repository that includes a complete rewrite of the renderers, which are now called converters.

https://github.com/asciidoctor/asciidoctor/pull/906

Renderers are dead, long live extensible Converters, as the saying goes.

The lack of extensibility in the rendering layer became apparent to me while working on Asciidoctor PDF. The Tilt-supported templates are certainly useful, but suffer from a rather sever bias that they are generating a single text file or stream. Formats such as PDF and ePub don't work in this model. So we needed to have a way to still be able to use Ruby or Java directly to convert the document. Now we do :)

Since we aren't just rendering templates anymore, I decided it's more appropriate to call these objects converters. I've introduced a Converter "interface" and a factory object responsible for selecting and instantiating the implementations.

Here's the hierarchy:

....
Converter (a Ruby module)
  BaseConverter (an abstract Ruby class)
    CompositeConverter (delegates to a chain of converters)
    TemplateConverter (delegates conversion to Tilt-supported templates)
  BuiltInConverter
    Html5Converter (used to convert documents to the html5 backend format)
    DocBook45Converter (used to convert documents to the docbook45 backend format)
    DocBook5Converter (used to convert documents to the docbook5 backend format)
....

If you use a built-in backend, such as html5, then the corresponding BuiltInConverter is used. If you specify a template directory, a CompositeConverter is used that delegates first to the TemplateConverter, then to the BuiltInConverter. Functionally, this is how Asciidoctor always worked. But it was a complete hack before. Now it works in a very logical and transparent way.

If you want to create a custom converter in Ruby, it's very straightforward. Here's an example of a very rough text-only output converter.

```ruby

class TextConverter

  # mix in the converter interface
  include Asciidoctor::Converter

  # associate this converter with the text backend
  register_for 'text'

  # convert each node in the tree
  # recursion is triggered by calls to the #content method
  def convert node, transform = nil
    case (transform ||= node.node_name)
    when 'document'
      node.content
    when 'section'
      [node.title, node.content] * "\n\n"
    when 'paragraph'
      node.content.tr("\n", ' ') << "\n"
    else
      if node.inline?
        node.text
      else
        %(TODO: #{transform}\n)
      end
    end
  end
end

```

Here's how it's used:

```ruby

puts Asciidoctor.convert_file 'sample.adoc', backend: :text

```

Just like with extensions, the converter is registered globally by default. If you want to use the converter explicitly without registering it, you remove the "register_for" line and pass either the class or instance as an option to the API:

```ruby

puts Asciidoctor.convert_file 'sample.adoc', backend: :text, converter: TextConverter

```

You'll notice I'm using "convert" in the API calls instead of "render". Both names are still supported, but "convert" is the terminology I recommend from this point forward.

I documented the API thoroughly, so once the pull request is merged, you'll be able to view the docs at the following URL:

http://rubydoc.info/github/asciidoctor/asciidoctor/frames

Other benefits include more speed (oh yes, always more speed) and a working table of contents in Asciidoctor.js :)

I'll publish a preview3 release asap so you can try it out.

If you have questions or comments, please don't hesitate to ask.



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML




If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1547.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1550.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML




If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1551.html
To unsubscribe from Renderer to Converter metamorphosis, click here.
NAML



--
+----------------------------------------------------------+
  Alex Soto Bueno - Computer Engineer
  www.lordofthejars.com
+----------------------------------------------------------+



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1554.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



--



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1555.html
To unsubscribe from Renderer to Converter metamorphosis, click here.
NAML



--
+----------------------------------------------------------+
  Alex Soto Bueno - Computer Engineer
  www.lordofthejars.com
+----------------------------------------------------------+
Reply | Threaded
Open this post in threaded view
|

Re: Renderer to Converter metamorphosis

LightGuardjp
How big of a document are you testing? Also that thread hasn't been updated with dynjs numbers. 

Sent from Mailbox for iPhone


On Thu, Feb 27, 2014 at 12:52 AM, asotobu [via Asciidoctor :: Discussion] <[hidden email]> wrote:



2014-02-27 8:41 GMT+01:00 LightGuardjp [via Asciidoctor :: Discussion] <[hidden email]>:
How much slower are we talking? It may be an acceptable trade off for people (size vs speed).


On Thu, Feb 27, 2014 at 12:38 AM, asotobu [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Yes. I have already a project with implementations of Nashorn, Rhino and DynJs (they have already fixed a bug caused by opal). So as soon as a new asciidoctor.js with opal.js is been released we could start the integration again with slim jar.

Now after some tests we can release a java version using js but is slower than the JRuby ones.


2014-02-26 23:41 GMT+01:00 LightGuardjp [via Asciidoctor :: Discussion] <[hidden email]>:

Now  things are getting really interesting :) as soon as we get things running on nashorn you'll have full asciidoctor in one slim jar in Java. You could do asciidoclet easily, really all sorts of things :)

Sent from Mailbox for iPhone


On Wed, Feb 26, 2014 at 3:37 PM, mojavelinux [via Asciidoctor :: Discussion] <[hidden email]> wrote:

Jason,

Absolutely correct. In fact, I was thinking of doing a Kramdown to Asciidoctor AST transformation and vice-versa to demonstrate.

Probably time for an asciidoctor-converter-lab repo :)

-Dan

On Feb 26, 2014 11:55 AM, "LightGuardjp [via Asciidoctor :: Discussion]" <[hidden email]> wrote:
Unless I'm mistaken, this should more easily allow us to create an AsciiDoc converter! That would mean we could use the AST methods to programmatically create a document and then have it kick out the AsciiDoc text :) 

Sent from Mailbox for iPhone


On Wed, Feb 26, 2014 at 1:41 AM, mojavelinux [via Asciidoctor :: Discussion] <[hidden email]> wrote:

I just sent the largest pull request yet to the Asciidoctor repository that includes a complete rewrite of the renderers, which are now called converters.

https://github.com/asciidoctor/asciidoctor/pull/906

Renderers are dead, long live extensible Converters, as the saying goes.

The lack of extensibility in the rendering layer became apparent to me while working on Asciidoctor PDF. The Tilt-supported templates are certainly useful, but suffer from a rather sever bias that they are generating a single text file or stream. Formats such as PDF and ePub don't work in this model. So we needed to have a way to still be able to use Ruby or Java directly to convert the document. Now we do :)

Since we aren't just rendering templates anymore, I decided it's more appropriate to call these objects converters. I've introduced a Converter "interface" and a factory object responsible for selecting and instantiating the implementations.

Here's the hierarchy:

....
Converter (a Ruby module)
  BaseConverter (an abstract Ruby class)
    CompositeConverter (delegates to a chain of converters)
    TemplateConverter (delegates conversion to Tilt-supported templates)
  BuiltInConverter
    Html5Converter (used to convert documents to the html5 backend format)
    DocBook45Converter (used to convert documents to the docbook45 backend format)
    DocBook5Converter (used to convert documents to the docbook5 backend format)
....

If you use a built-in backend, such as html5, then the corresponding BuiltInConverter is used. If you specify a template directory, a CompositeConverter is used that delegates first to the TemplateConverter, then to the BuiltInConverter. Functionally, this is how Asciidoctor always worked. But it was a complete hack before. Now it works in a very logical and transparent way.

If you want to create a custom converter in Ruby, it's very straightforward. Here's an example of a very rough text-only output converter.

```ruby

class TextConverter

  # mix in the converter interface
  include Asciidoctor::Converter

  # associate this converter with the text backend
  register_for 'text'

  # convert each node in the tree
  # recursion is triggered by calls to the #content method
  def convert node, transform = nil
    case (transform ||= node.node_name)
    when 'document'
      node.content
    when 'section'
      [node.title, node.content] * "\n\n"
    when 'paragraph'
      node.content.tr("\n", ' ') << "\n"
    else
      if node.inline?
        node.text
      else
        %(TODO: #{transform}\n)
      end
    end
  end
end

```

Here's how it's used:

```ruby

puts Asciidoctor.convert_file 'sample.adoc', backend: :text

```

Just like with extensions, the converter is registered globally by default. If you want to use the converter explicitly without registering it, you remove the "register_for" line and pass either the class or instance as an option to the API:

```ruby

puts Asciidoctor.convert_file 'sample.adoc', backend: :text, converter: TextConverter

```

You'll notice I'm using "convert" in the API calls instead of "render". Both names are still supported, but "convert" is the terminology I recommend from this point forward.

I documented the API thoroughly, so once the pull request is merged, you'll be able to view the docs at the following URL:

http://rubydoc.info/github/asciidoctor/asciidoctor/frames

Other benefits include more speed (oh yes, always more speed) and a working table of contents in Asciidoctor.js :)

I'll publish a preview3 release asap so you can try it out.

If you have questions or comments, please don't hesitate to ask.



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML




If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1547.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1550.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML




If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1551.html
To unsubscribe from Renderer to Converter metamorphosis, click here.
NAML



--
+----------------------------------------------------------+
  Alex Soto Bueno - Computer Engineer
  www.lordofthejars.com
+----------------------------------------------------------+



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1554.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



--



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1555.html
To unsubscribe from Renderer to Converter metamorphosis, click here.
NAML



--
+----------------------------------------------------------+
  Alex Soto Bueno - Computer Engineer
  www.lordofthejars.com
+----------------------------------------------------------+



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1556.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML

Reply | Threaded
Open this post in threaded view
|

Re: Renderer to Converter metamorphosis

asotobu
It is the asciidoctor quick reference document. Yes with dynjs is not updated because there was an error in dynjs when executing opal scripts. It was fixed 9 years ago https://github.com/dynjs/dynjs/issues/95 . Now I am working in asciidoctorj 1.5.0, and when it is released I will restart again that path.


2014-02-27 15:11 GMT+01:00 LightGuardjp [via Asciidoctor :: Discussion] <[hidden email]>:
How big of a document are you testing? Also that thread hasn't been updated with dynjs numbers. 

Sent from Mailbox for iPhone


On Thu, Feb 27, 2014 at 12:52 AM, asotobu [via Asciidoctor :: Discussion] <[hidden email]> wrote:



2014-02-27 8:41 GMT+01:00 LightGuardjp [via Asciidoctor :: Discussion] <[hidden email]>:

How much slower are we talking? It may be an acceptable trade off for people (size vs speed).


On Thu, Feb 27, 2014 at 12:38 AM, asotobu [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Yes. I have already a project with implementations of Nashorn, Rhino and DynJs (they have already fixed a bug caused by opal). So as soon as a new asciidoctor.js with opal.js is been released we could start the integration again with slim jar.

Now after some tests we can release a java version using js but is slower than the JRuby ones.


2014-02-26 23:41 GMT+01:00 LightGuardjp [via Asciidoctor :: Discussion] <[hidden email]>:

Now  things are getting really interesting :) as soon as we get things running on nashorn you'll have full asciidoctor in one slim jar in Java. You could do asciidoclet easily, really all sorts of things :)

Sent from Mailbox for iPhone


On Wed, Feb 26, 2014 at 3:37 PM, mojavelinux [via Asciidoctor :: Discussion] <[hidden email]> wrote:

Jason,

Absolutely correct. In fact, I was thinking of doing a Kramdown to Asciidoctor AST transformation and vice-versa to demonstrate.

Probably time for an asciidoctor-converter-lab repo :)

-Dan

On Feb 26, 2014 11:55 AM, "LightGuardjp [via Asciidoctor :: Discussion]" <[hidden email]> wrote:
Unless I'm mistaken, this should more easily allow us to create an AsciiDoc converter! That would mean we could use the AST methods to programmatically create a document and then have it kick out the AsciiDoc text :) 

Sent from Mailbox for iPhone


On Wed, Feb 26, 2014 at 1:41 AM, mojavelinux [via Asciidoctor :: Discussion] <[hidden email]> wrote:

I just sent the largest pull request yet to the Asciidoctor repository that includes a complete rewrite of the renderers, which are now called converters.

https://github.com/asciidoctor/asciidoctor/pull/906

Renderers are dead, long live extensible Converters, as the saying goes.

The lack of extensibility in the rendering layer became apparent to me while working on Asciidoctor PDF. The Tilt-supported templates are certainly useful, but suffer from a rather sever bias that they are generating a single text file or stream. Formats such as PDF and ePub don't work in this model. So we needed to have a way to still be able to use Ruby or Java directly to convert the document. Now we do :)

Since we aren't just rendering templates anymore, I decided it's more appropriate to call these objects converters. I've introduced a Converter "interface" and a factory object responsible for selecting and instantiating the implementations.

Here's the hierarchy:

....
Converter (a Ruby module)
  BaseConverter (an abstract Ruby class)
    CompositeConverter (delegates to a chain of converters)
    TemplateConverter (delegates conversion to Tilt-supported templates)
  BuiltInConverter
    Html5Converter (used to convert documents to the html5 backend format)
    DocBook45Converter (used to convert documents to the docbook45 backend format)
    DocBook5Converter (used to convert documents to the docbook5 backend format)
....

If you use a built-in backend, such as html5, then the corresponding BuiltInConverter is used. If you specify a template directory, a CompositeConverter is used that delegates first to the TemplateConverter, then to the BuiltInConverter. Functionally, this is how Asciidoctor always worked. But it was a complete hack before. Now it works in a very logical and transparent way.

If you want to create a custom converter in Ruby, it's very straightforward. Here's an example of a very rough text-only output converter.

```ruby

class TextConverter

  # mix in the converter interface
  include Asciidoctor::Converter

  # associate this converter with the text backend
  register_for 'text'

  # convert each node in the tree
  # recursion is triggered by calls to the #content method
  def convert node, transform = nil
    case (transform ||= node.node_name)
    when 'document'
      node.content
    when 'section'
      [node.title, node.content] * "\n\n"
    when 'paragraph'
      node.content.tr("\n", ' ') << "\n"
    else
      if node.inline?
        node.text
      else
        %(TODO: #{transform}\n)
      end
    end
  end
end

```

Here's how it's used:

```ruby

puts Asciidoctor.convert_file 'sample.adoc', backend: :text

```

Just like with extensions, the converter is registered globally by default. If you want to use the converter explicitly without registering it, you remove the "register_for" line and pass either the class or instance as an option to the API:

```ruby

puts Asciidoctor.convert_file 'sample.adoc', backend: :text, converter: TextConverter

```

You'll notice I'm using "convert" in the API calls instead of "render". Both names are still supported, but "convert" is the terminology I recommend from this point forward.

I documented the API thoroughly, so once the pull request is merged, you'll be able to view the docs at the following URL:

http://rubydoc.info/github/asciidoctor/asciidoctor/frames

Other benefits include more speed (oh yes, always more speed) and a working table of contents in Asciidoctor.js :)

I'll publish a preview3 release asap so you can try it out.

If you have questions or comments, please don't hesitate to ask.



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML




If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1547.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1550.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML




If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1551.html
To unsubscribe from Renderer to Converter metamorphosis, click here.
NAML



--
+----------------------------------------------------------+
  Alex Soto Bueno - Computer Engineer
  www.lordofthejars.com
+----------------------------------------------------------+



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1554.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



--



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1555.html
To unsubscribe from Renderer to Converter metamorphosis, click here.
NAML



--
+----------------------------------------------------------+
  Alex Soto Bueno - Computer Engineer
  www.lordofthejars.com
+----------------------------------------------------------+



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1556.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML




If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1558.html
To unsubscribe from Renderer to Converter metamorphosis, click here.
NAML



--
+----------------------------------------------------------+
  Alex Soto Bueno - Computer Engineer
  www.lordofthejars.com
+----------------------------------------------------------+
Reply | Threaded
Open this post in threaded view
|

Re: Renderer to Converter metamorphosis

LightGuardjp
Even if things are slow, they're probably still faster than the python impl :) once we have the ability to do includes and extensions with the JS version, I think we'll have something viable for a slim jar. 

Sent from Mailbox for iPhone


On Thu, Feb 27, 2014 at 7:18 AM, asotobu [via Asciidoctor :: Discussion] <[hidden email]> wrote:

It is the asciidoctor quick reference document. Yes with dynjs is not updated because there was an error in dynjs when executing opal scripts. It was fixed 9 years ago https://github.com/dynjs/dynjs/issues/95 . Now I am working in asciidoctorj 1.5.0, and when it is released I will restart again that path.


2014-02-27 15:11 GMT+01:00 LightGuardjp [via Asciidoctor :: Discussion] <[hidden email]>:
How big of a document are you testing? Also that thread hasn't been updated with dynjs numbers. 

Sent from Mailbox for iPhone


On Thu, Feb 27, 2014 at 12:52 AM, asotobu [via Asciidoctor :: Discussion] <[hidden email]> wrote:



2014-02-27 8:41 GMT+01:00 LightGuardjp [via Asciidoctor :: Discussion] <[hidden email]>:

How much slower are we talking? It may be an acceptable trade off for people (size vs speed).


On Thu, Feb 27, 2014 at 12:38 AM, asotobu [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Yes. I have already a project with implementations of Nashorn, Rhino and DynJs (they have already fixed a bug caused by opal). So as soon as a new asciidoctor.js with opal.js is been released we could start the integration again with slim jar.

Now after some tests we can release a java version using js but is slower than the JRuby ones.


2014-02-26 23:41 GMT+01:00 LightGuardjp [via Asciidoctor :: Discussion] <[hidden email]>:

Now  things are getting really interesting :) as soon as we get things running on nashorn you'll have full asciidoctor in one slim jar in Java. You could do asciidoclet easily, really all sorts of things :)

Sent from Mailbox for iPhone


On Wed, Feb 26, 2014 at 3:37 PM, mojavelinux [via Asciidoctor :: Discussion] <[hidden email]> wrote:

Jason,

Absolutely correct. In fact, I was thinking of doing a Kramdown to Asciidoctor AST transformation and vice-versa to demonstrate.

Probably time for an asciidoctor-converter-lab repo :)

-Dan

On Feb 26, 2014 11:55 AM, "LightGuardjp [via Asciidoctor :: Discussion]" <[hidden email]> wrote:
Unless I'm mistaken, this should more easily allow us to create an AsciiDoc converter! That would mean we could use the AST methods to programmatically create a document and then have it kick out the AsciiDoc text :) 

Sent from Mailbox for iPhone


On Wed, Feb 26, 2014 at 1:41 AM, mojavelinux [via Asciidoctor :: Discussion] <[hidden email]> wrote:

I just sent the largest pull request yet to the Asciidoctor repository that includes a complete rewrite of the renderers, which are now called converters.

https://github.com/asciidoctor/asciidoctor/pull/906

Renderers are dead, long live extensible Converters, as the saying goes.

The lack of extensibility in the rendering layer became apparent to me while working on Asciidoctor PDF. The Tilt-supported templates are certainly useful, but suffer from a rather sever bias that they are generating a single text file or stream. Formats such as PDF and ePub don't work in this model. So we needed to have a way to still be able to use Ruby or Java directly to convert the document. Now we do :)

Since we aren't just rendering templates anymore, I decided it's more appropriate to call these objects converters. I've introduced a Converter "interface" and a factory object responsible for selecting and instantiating the implementations.

Here's the hierarchy:

....
Converter (a Ruby module)
  BaseConverter (an abstract Ruby class)
    CompositeConverter (delegates to a chain of converters)
    TemplateConverter (delegates conversion to Tilt-supported templates)
  BuiltInConverter
    Html5Converter (used to convert documents to the html5 backend format)
    DocBook45Converter (used to convert documents to the docbook45 backend format)
    DocBook5Converter (used to convert documents to the docbook5 backend format)
....

If you use a built-in backend, such as html5, then the corresponding BuiltInConverter is used. If you specify a template directory, a CompositeConverter is used that delegates first to the TemplateConverter, then to the BuiltInConverter. Functionally, this is how Asciidoctor always worked. But it was a complete hack before. Now it works in a very logical and transparent way.

If you want to create a custom converter in Ruby, it's very straightforward. Here's an example of a very rough text-only output converter.

```ruby

class TextConverter

  # mix in the converter interface
  include Asciidoctor::Converter

  # associate this converter with the text backend
  register_for 'text'

  # convert each node in the tree
  # recursion is triggered by calls to the #content method
  def convert node, transform = nil
    case (transform ||= node.node_name)
    when 'document'
      node.content
    when 'section'
      [node.title, node.content] * "\n\n"
    when 'paragraph'
      node.content.tr("\n", ' ') << "\n"
    else
      if node.inline?
        node.text
      else
        %(TODO: #{transform}\n)
      end
    end
  end
end

```

Here's how it's used:

```ruby

puts Asciidoctor.convert_file 'sample.adoc', backend: :text

```

Just like with extensions, the converter is registered globally by default. If you want to use the converter explicitly without registering it, you remove the "register_for" line and pass either the class or instance as an option to the API:

```ruby

puts Asciidoctor.convert_file 'sample.adoc', backend: :text, converter: TextConverter

```

You'll notice I'm using "convert" in the API calls instead of "render". Both names are still supported, but "convert" is the terminology I recommend from this point forward.

I documented the API thoroughly, so once the pull request is merged, you'll be able to view the docs at the following URL:

http://rubydoc.info/github/asciidoctor/asciidoctor/frames

Other benefits include more speed (oh yes, always more speed) and a working table of contents in Asciidoctor.js :)

I'll publish a preview3 release asap so you can try it out.

If you have questions or comments, please don't hesitate to ask.



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML




If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1547.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1550.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML




If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1551.html
To unsubscribe from Renderer to Converter metamorphosis, click here.
NAML



--
+----------------------------------------------------------+
  Alex Soto Bueno - Computer Engineer
  www.lordofthejars.com
+----------------------------------------------------------+



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1554.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



--



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1555.html
To unsubscribe from Renderer to Converter metamorphosis, click here.
NAML



--
+----------------------------------------------------------+
  Alex Soto Bueno - Computer Engineer
  www.lordofthejars.com
+----------------------------------------------------------+



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1556.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML




If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1558.html
To unsubscribe from Renderer to Converter metamorphosis, click here.
NAML



--
+----------------------------------------------------------+
  Alex Soto Bueno - Computer Engineer
  www.lordofthejars.com
+----------------------------------------------------------+



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1559.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML

Reply | Threaded
Open this post in threaded view
|

Re: Renderer to Converter metamorphosis

asotobu
Correct this is what I have in mind to implement for version 1.6.0, but of course there are some open points like render to disk, extensions, decide if nashorn or dynjs or both, ... I think that before starting to implement a full project we need some kind of sandbox to play around first. 

But I completely agree with you that we need to create an slim jar.

Also I know that JRuby folks are working on that for splitting the whole runtime, so maybe we will be able to generate an slim jar.


2014-02-27 15:25 GMT+01:00 LightGuardjp [via Asciidoctor :: Discussion] <[hidden email]>:
Even if things are slow, they're probably still faster than the python impl :) once we have the ability to do includes and extensions with the JS version, I think we'll have something viable for a slim jar. 

Sent from Mailbox for iPhone


On Thu, Feb 27, 2014 at 7:18 AM, asotobu [via Asciidoctor :: Discussion] <[hidden email]> wrote:

It is the asciidoctor quick reference document. Yes with dynjs is not updated because there was an error in dynjs when executing opal scripts. It was fixed 9 years ago https://github.com/dynjs/dynjs/issues/95 . Now I am working in asciidoctorj 1.5.0, and when it is released I will restart again that path.


2014-02-27 15:11 GMT+01:00 LightGuardjp [via Asciidoctor :: Discussion] <[hidden email]>:

How big of a document are you testing? Also that thread hasn't been updated with dynjs numbers. 

Sent from Mailbox for iPhone


On Thu, Feb 27, 2014 at 12:52 AM, asotobu [via Asciidoctor :: Discussion] <[hidden email]> wrote:



2014-02-27 8:41 GMT+01:00 LightGuardjp [via Asciidoctor :: Discussion] <[hidden email]>:

How much slower are we talking? It may be an acceptable trade off for people (size vs speed).


On Thu, Feb 27, 2014 at 12:38 AM, asotobu [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Yes. I have already a project with implementations of Nashorn, Rhino and DynJs (they have already fixed a bug caused by opal). So as soon as a new asciidoctor.js with opal.js is been released we could start the integration again with slim jar.

Now after some tests we can release a java version using js but is slower than the JRuby ones.


2014-02-26 23:41 GMT+01:00 LightGuardjp [via Asciidoctor :: Discussion] <[hidden email]>:

Now  things are getting really interesting :) as soon as we get things running on nashorn you'll have full asciidoctor in one slim jar in Java. You could do asciidoclet easily, really all sorts of things :)

Sent from Mailbox for iPhone


On Wed, Feb 26, 2014 at 3:37 PM, mojavelinux [via Asciidoctor :: Discussion] <[hidden email]> wrote:

Jason,

Absolutely correct. In fact, I was thinking of doing a Kramdown to Asciidoctor AST transformation and vice-versa to demonstrate.

Probably time for an asciidoctor-converter-lab repo :)

-Dan

On Feb 26, 2014 11:55 AM, "LightGuardjp [via Asciidoctor :: Discussion]" <[hidden email]> wrote:
Unless I'm mistaken, this should more easily allow us to create an AsciiDoc converter! That would mean we could use the AST methods to programmatically create a document and then have it kick out the AsciiDoc text :) 

Sent from Mailbox for iPhone


On Wed, Feb 26, 2014 at 1:41 AM, mojavelinux [via Asciidoctor :: Discussion] <[hidden email]> wrote:

I just sent the largest pull request yet to the Asciidoctor repository that includes a complete rewrite of the renderers, which are now called converters.

https://github.com/asciidoctor/asciidoctor/pull/906

Renderers are dead, long live extensible Converters, as the saying goes.

The lack of extensibility in the rendering layer became apparent to me while working on Asciidoctor PDF. The Tilt-supported templates are certainly useful, but suffer from a rather sever bias that they are generating a single text file or stream. Formats such as PDF and ePub don't work in this model. So we needed to have a way to still be able to use Ruby or Java directly to convert the document. Now we do :)

Since we aren't just rendering templates anymore, I decided it's more appropriate to call these objects converters. I've introduced a Converter "interface" and a factory object responsible for selecting and instantiating the implementations.

Here's the hierarchy:

....
Converter (a Ruby module)
  BaseConverter (an abstract Ruby class)
    CompositeConverter (delegates to a chain of converters)
    TemplateConverter (delegates conversion to Tilt-supported templates)
  BuiltInConverter
    Html5Converter (used to convert documents to the html5 backend format)
    DocBook45Converter (used to convert documents to the docbook45 backend format)
    DocBook5Converter (used to convert documents to the docbook5 backend format)
....

If you use a built-in backend, such as html5, then the corresponding BuiltInConverter is used. If you specify a template directory, a CompositeConverter is used that delegates first to the TemplateConverter, then to the BuiltInConverter. Functionally, this is how Asciidoctor always worked. But it was a complete hack before. Now it works in a very logical and transparent way.

If you want to create a custom converter in Ruby, it's very straightforward. Here's an example of a very rough text-only output converter.

```ruby

class TextConverter

  # mix in the converter interface
  include Asciidoctor::Converter

  # associate this converter with the text backend
  register_for 'text'

  # convert each node in the tree
  # recursion is triggered by calls to the #content method
  def convert node, transform = nil
    case (transform ||= node.node_name)
    when 'document'
      node.content
    when 'section'
      [node.title, node.content] * "\n\n"
    when 'paragraph'
      node.content.tr("\n", ' ') << "\n"
    else
      if node.inline?
        node.text
      else
        %(TODO: #{transform}\n)
      end
    end
  end
end

```

Here's how it's used:

```ruby

puts Asciidoctor.convert_file 'sample.adoc', backend: :text

```

Just like with extensions, the converter is registered globally by default. If you want to use the converter explicitly without registering it, you remove the "register_for" line and pass either the class or instance as an option to the API:

```ruby

puts Asciidoctor.convert_file 'sample.adoc', backend: :text, converter: TextConverter

```

You'll notice I'm using "convert" in the API calls instead of "render". Both names are still supported, but "convert" is the terminology I recommend from this point forward.

I documented the API thoroughly, so once the pull request is merged, you'll be able to view the docs at the following URL:

http://rubydoc.info/github/asciidoctor/asciidoctor/frames

Other benefits include more speed (oh yes, always more speed) and a working table of contents in Asciidoctor.js :)

I'll publish a preview3 release asap so you can try it out.

If you have questions or comments, please don't hesitate to ask.



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML




If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1547.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1550.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML




If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1551.html
To unsubscribe from Renderer to Converter metamorphosis, click here.
NAML



--
+----------------------------------------------------------+
  Alex Soto Bueno - Computer Engineer
  www.lordofthejars.com
+----------------------------------------------------------+



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1554.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



--



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1555.html
To unsubscribe from Renderer to Converter metamorphosis, click here.
NAML



--
+----------------------------------------------------------+
  Alex Soto Bueno - Computer Engineer
  www.lordofthejars.com
+----------------------------------------------------------+



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1556.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML




If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1558.html
To unsubscribe from Renderer to Converter metamorphosis, click here.
NAML



--
+----------------------------------------------------------+
  Alex Soto Bueno - Computer Engineer
  www.lordofthejars.com
+----------------------------------------------------------+



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1559.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML




If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Renderer-to-Converter-metamorphosis-tp1543p1560.html
To unsubscribe from Renderer to Converter metamorphosis, click here.
NAML



--
+----------------------------------------------------------+
  Alex Soto Bueno - Computer Engineer
  www.lordofthejars.com
+----------------------------------------------------------+