AsciidoctorJ Block Processor Problems

Hi Sean,

        AbstractBlock block = createBlock(parent, "literal", reader.readLines(), attributes, new HashMap<Object, Object>());
        System.out.println(block.blocks());
        for(AbstractBlock b : block.blocks()) {
            System.out.println("Block contents:");
            System.out.println(b.content().toString());
        }

This assumes that the lines you pass to the new block get parsed.
But Dan wrote above that they do not get parsed at the moment.
So I fear that this approach does not work at the moment.

But I think it should be possible to iterate over the child blocks of the original tabs block instead of creating a new block with the same string content and trying to get the child blocks from the new one.

Ah, I misunderstood what he said above. Ok, I will give that a shot. Thank you!

I'm now trying to iterate over the child blocks of the parent, but I'm getting the following exception:

org.jruby.exceptions.RaiseException: (TypeError) exception class/object expected
	at RUBY.load(/Users/sean/.rvm/gems/ruby-2.2.1/gems/asciidoctor-1.5.2/lib/asciidoctor.rb:1362)
	at RUBY.convert(/Users/sean/.rvm/gems/ruby-2.2.1/gems/asciidoctor-1.5.2/lib/asciidoctor.rb:1458)
	at RUBY.convert_file(/Users/sean/.rvm/gems/ruby-2.2.1/gems/asciidoctor-1.5.2/lib/asciidoctor.rb:1562)
	at RUBY.convertFile(<script>:68)
	at org.jruby.gen.InterfaceImpl175252318.convertFile(org/jruby/gen/InterfaceImpl175252318.gen:13)

It throws the exception as soon as I start iterating. I can successfully print out the content of the block, for what it's worth. Here's my code:

@Override
    public Object process(AbstractBlock parent, Reader reader, Map<String, Object> attributes) {
        System.out.println(reader.read());
        for(AbstractBlock b : parent.blocks()) {
            System.out.println("Block contents: =====================");
            System.out.println(b.content().toString());
        }
    . . . 
}

Hi Sean,

This won't really help you, but just so you know I've also been working on adding tabs and collapsable accordions to my own document.  I took the route of first hijacking some Slim templates and I'm now instead working on making a standalone Slim template.  Thanks to a lot of help from Dan and others, I got things to work fairly well.

However, at the moment my solution expects the content of the tabs or sections to be available in external files, not entered in-line within the document.  This is very useful for me, but perhaps not always so for others.  Also, I depend on a fair amount of metadata being passed as attributes which is not as clean as I'd like it to be.  Your approach seems pretty good, too, so I'm looking forward to hearing about the outcome!

Cheers!

Sebastien

Thanks Sebastien, is any of your code public for the Slim template? I would be interested in taking a look at it.

For our case, I really wanted to avoid having a separate file, but it does make a lot of sense for some cases. We're using external files for other use cases but not tabs, though maybe I'll have to consider that.

Anyone have any other suggestions on how to process the child blocks? I've tried to get unblocked (no pun intended) but keep getting this exception. Thanks in advance!

Just filed issues https://github.com/asciidoctor/asciidoctorj/issues/325 and https://github.com/asciidoctor/asciidoctorj/issues/326 to follow up on the open points.

Hi Sean,

I haven't made my code available anywhere yet, but it's in the plan.  In the meantime, I can you give the gist here.

First, I created a new extension that I named bootstrap_extension.rb:

require 'asciidoctor/extensions'
Asciidoctor::Extensions.register do
  block do
    named :bootstrap
    on_contexts :paragraph,:open
    name_attributes 'component'
    process do |parent, reader, attrs|
      create_block parent, attrs['component'], reader.lines, attrs
    end
  end
end

Note that the "on_contexts" refers to both :paragraph and :open.  I don't know what other contexts are available, so perhaps this could be improved.  Note that at the moment, the content of the open block or paragraph is completely ignored.  For a collapsable section, this could be easily changed so that it becomes the content (instead of an external file), but for a tab it would require more involved processing (a bit more on that later).

Next, I make sure that this extension is used by the maven plugin by adding this tidbit to its <configuration>:

<requires>
  <require>${project.basedir}/src/main/asciidoc/ext/bootstrap_extension.rb</require>
</requires>

Next, I created Slim templates for two of the Bootstrap components that I wanted to use, namely Tabs and Collapsable sections:

${project.basedir}/src/main/asciidoc/html5/navtabs.html.slim
Code:
----
- if (attr? 'fileprefix') and (attr? 'filedir') and (attr? 'tabnames') and (attr? 'activetab')
  - tabs = (attr 'tabnames').split(",")
  ul id="bs-tab-#{attr 'id'}" class="nav nav-tabs" role="tablist"
    - tabs.each do |tab|
      - itemTarget = "#{attr 'id'}-#{tab.downcase}"
      - isTabActive = (tab == (attr 'activetab'))
      - itemClass = (isTabActive ? "active" : "")
      - tabStatus = (isTabActive ? "true" : "false")
      li role="presentation" class="#{itemClass}"
        a href="##{itemTarget}" id="#{itemTarget}-tab" role="tab" data-toggle="tab" aria-controls="#{itemTarget}" aria-expanded="#{tabStatus}" =tab
  div id="bs-content-#{attr 'id'}" class="tab-content"
    - tabs.each do |tab|
      - itemTarget = "#{attr 'id'}-#{tab.downcase}"
      - isTabActive = (tab == (attr 'activetab'))
      - tabClass = (isTabActive ? " active in" : "")
      div role="tabpanel" class="tab-pane fade#{tabClass}" id="#{itemTarget}" aria-labelledby="#{itemTarget}-tab"
        - filename = "#{attr 'filedir'}/#{attr 'fileprefix'}-#{tab}.adoc"
        - if (File.exist?(filename))
          = Asciidoctor.convert(IO.read(filename), base_dir: @document.base_dir, safe: @document.safe)
        - else
          = filename + " not found"
- else
  = "Missing configuration..."
----

${project.basedir}/src/main/asciidoc/html5/collapse.html.slim
Code:
----
- if (attr? 'id') and (attr? 'file') and (attr? 'title')
  div class='panel-group' id="bs-accordion-#{attr 'id'}" role="tablist" aria-multiselectable="true"
    div class="panel panel-default"
      div class="panel-heading" role="tab" id="bs-heading-#{attr 'id'}"
        a data-toggle="collapse" data-parent="#bs-accordion-#{attr 'id'}" href="#bs-collapse-#{attr 'id'}" aria-expanded="false" aria-controls="bs-collapse-#{attr 'id'}" = "#{attr 'title'}"
      div id="bs-collapse-#{attr 'id'}" class="panel-collapse collapse" role="tabpanel" aria-labelledby="bs-heading-#{attr 'id'}"
        div class="panel-body"
          - if (attr? 'file')
            - filename = "#{attr 'file'}"
            - if (File.file?(filename))
              = Asciidoctor.convert(IO.read(filename), base_dir: @document.base_dir, safe: @document.safe)
            - else
              = filename + " does not exist"
- else
  = "Missing some configuration..."
----

Next, I make the maven plugin aware of these templates by adding this to the <configuration> element:

<templateDir>src/main/asciidoc/html5</templateDir>

So the end result is that I can now use the following syntax to add a collapsable section or tabs:

Collapsable section:
[bootstrap,collapse, id="collapse1" title="Collapsable Section 1" file="{path_to_my_file}"]
--
--

Tabs:
[bootstrap,navtabs, id="getsub" filedir="{file_dir}" fileprefix="{file_prefix}" tabnames="JSON,XML" activetab="JSON"]
--
--

IMPORTANT:  this assumes that you've already included the Bootstrap CSS and JS dependencies to your document, as well as JQuery.  If not, you can add those using the docinfo approach.

Note that for the tab sections, I'm generating files like "http-request-JSON.adoc" and "http-request-XML.adoc" that are located in a common directory.  This makes the file_dir to be that common directory, and the file prefix to be "http-request".

As an aside, you might have noticed that I use some prefixes for the various IDs that are generated (ex: "bs-accordion-#{id}").  That's not necessary and is a bit of an artefact from previous incarnations that I ought to clean up.

Disclaimer:  I'm no expert in Ruby or Slim, so any improvement suggestions are most welcome.  :)

You might have also noticed that in my Slim templates the external files are being converted by Asciidoc.  That's because those external files contain code that I want to have highlighted (or sometimes tables), so a typical file would contain something like this:

[source,http,options="nowrap"]
----
PUT http://localhost:8080/some/relevant/path HTTP/1.1
Accept: application/json
User-Agent: Apache-HttpClient/4.3.6 (java 1.5)
Host: localhost:8080
Content-Length: <content_length>
Proxy-Connection: Keep-Alive
Content-Type: application/json; charset=UTF-8

{
    "key1": "value1",
    "key2": "value2"
}
----

It could be useful to somehow move the configuration of the tabs into the block itself.  Other than parsing the contents itself manually, I don't know how to do it.  However, I just saw Dan's post about child blocks so perhaps I could piggy back on that approach.

Other things that I'd like to improve is to cap the size of various panels so that a vertical scrollbar is made available instead of having my code eat up a lot of space.  I've had weird results up to now (the horizontal scrollbar introduced by the "nowrap" option on the listing block only shows up when you've completed scrolled vertically to the end of the block).  It might be nice to have a "maxlines" or something similar that would natively be available in the "options" of a listing block.

Finally, BIG thanks to Dan and the community for helping me with this.  I'm standing on the shoulders of giants.  :)

I hope that can be of help to you!

Cheers,

Sebastien

Thank you Dan, that mini-tutorial worked great and I've got everything working as expected.

The only issue I'm running into is that I'll get a warning displayed every time one of the child blocks is processed. Specifically:

asciidoctor: WARNING: doc-test.adoc: line 8: invalid style for literal block: tab

A reminder that the syntax I'm using is as follows:

[tabs]
------
[tab,title="foo"]
....
content goes here
....
[tab,title="bar"]
....
content goes here
....
------

Also, to close the loop on my tab implementation -- I'd really like to do block processing in the same manner for the Atom Asciidoctor Previewer that uses Asciidoctor.js. Is it possible to do block processing in the same manner for Asciidoctor.js and to display tabs in the previewer, assuming I can also inject my own CSS?

Thanks a bunch,
Sean

Thank you Sebastien, I'm giving this a shot as well. I think you came up with a novel solution to the same problem, which probably looks a lot better when you have a really long tab section like we have in our documentation. For really long tabs it makes more sense to pull in from an external file. I'm investigating a mix of your solution with my own -- we'll see.

Hi guys, I could use some more help. I'm having new trouble with my block processor -- it fails seemingly randomly on certain files with my tab syntax. The files are using the syntax just fine. In fact, I've used the same file contents and created many files with the same contents, and it will succeed on some of the files and then fail on others. There's no clear pattern of failure to help me debug.

I get the following error:

org.jruby.exceptions.RaiseException: (TypeError) exception class/object expected
	at RUBY.load(/Users/user/.rvm/gems/ruby-2.2.1/gems/asciidoctor-1.5.2/lib/asciidoctor.rb:1362)
	at RUBY.convert(/Users/user/.rvm/gems/ruby-2.2.1/gems/asciidoctor-1.5.2/lib/asciidoctor.rb:1458)
	at RUBY.convert_file(/Users/user/.rvm/gems/ruby-2.2.1/gems/asciidoctor-1.5.2/lib/asciidoctor.rb:1562)
	at RUBY.convertFile(<script>:68)
	at org.jruby.gen.InterfaceImpl1177483828.convertFile(org/jruby/gen/InterfaceImpl1177483828.gen:13)

Here's the ugly code I wrote to get things working and haven't yet refactored. Any help would be greatly appreciated!

public class TabProcessor extends BlockProcessor {

    private AsciiDocParser parser;
    private AsciiDocProcessor processor;

    private static Map<String, Object> CONFIG = new HashMap<String, Object>() {{
        put("contexts", Arrays.asList(":listing"));
        put("content_model", ":compound");
    }};

    public TabProcessor(String name, Map<String, Object> config) {
        super(name, CONFIG);
        IRubyObject parserRubyClass = rubyRuntime.evalScriptlet("Asciidoctor::Parser");
        processor = AsciiDocProcessor.getProcessorInstance();
        this.parser = (AsciiDocParser) JavaEmbedUtils.rubyToJava(rubyRuntime, parserRubyClass, AsciiDocParser.class);
    }

    @Override
    public Object process(AbstractBlock parent, Reader reader, Map<String, Object> attributes) {
        Map<String, Object> inherited = new HashMap<String, Object>();

        AbstractBlock container = createBlock(parent, "open", (String)null, inherited, new HashMap<Object, Object>());
        Map<String, String> titleAndId = new LinkedHashMap<String, String>();

        int count = 0;

        while (reader.hasMoreLines()) {
            Block child = parser.nextBlock(reader, parent, new HashMap<String, Object>());
            if (child != null) {
                String tabTitle = child.title();
                String uniqueId = Utilities.cleanPageFileNames(tabTitle) + "-" +
                        Utilities.getRandomAlphaNumericString(5);
                titleAndId.put(tabTitle, uniqueId);

                String singleTabHtmlOpen;
                if(count == 0) {
                    singleTabHtmlOpen = "<div class=\"tab-pane in active fade no-padding\" id=\"" + uniqueId + "\">";
                } else {
                    singleTabHtmlOpen = "<div class=\"tab-pane fade no-padding\" id=\"" + uniqueId + "\">";
                }

                String converted = processor.convertAsciiDocString(child.content().toString());
                converted = Utilities.getOnlyContentDivFromHtml(converted);
                converted = singleTabHtmlOpen + converted + "</div>";
                Block block = createBlock(parent, "pass", converted, inherited, new HashMap<Object, Object>());

                count++;

                container.blocks().add(block);
            }
        }

        String tabsOpen = "<div class=\"panel panel-default no-padding\">\n" +
                "    <div class=\"panel-heading no-padding\">\n" +
                "        <!-- Nav tabs -->\n" +
                "        <ul class=\"nav nav-tabs\" role=\"tablist\">\n";

        List<Map.Entry<String,String>> randAccess = new ArrayList<Map.Entry<String,String>>(titleAndId.entrySet());

        for(int i = 0; i < titleAndId.size(); i++) {
            if(i == 0) {
                tabsOpen += "<li class=\"active\">";
                tabsOpen += "<a href=\"#" + randAccess.get(i).getValue() +
                        "\" role=\"tab\" data-toggle=\"tab\">" + randAccess.get(i).getKey() + "</a></li>\n";
            } else {
                tabsOpen += "<li><a href=\"#" + randAccess.get(i).getValue() +
                        "\" role=\"tab\" data-toggle=\"tab\">" + randAccess.get(i).getKey() + "</a></li>\n";
            }
        }

        tabsOpen += "</ul>\n</div>\n<div class=\"panel-body tab-content no-padding\">";
        String tabsClose = "</div></div>";

        String containerContents = tabsOpen + container.content().toString() + tabsClose;
        AbstractBlock containerNew = createBlock(parent, "pass", containerContents, inherited, new HashMap<Object, Object>());
        container = containerNew;

        return container;
    }
}

Thanks Dan, I'll try that workaround and see where it leads me.

Sean,

I'll also mention that it's very possible the underlying problem is addressed in AsciidoctorJ 1.6.0 snapshot. Robert has fundamentally changed the way the AST gets mapped to the Ruby objects to achieve a cleaner and smoother integration. I plan to experiment with it myself to try to find out if the problem I'm seeing is still present or whether Robert's changes eliminated the issue.

One thing is for certain. To do advanced-level extensions in AsciidoctorJ (Java or Groovy), it isn't going to work well with 1.5.2. We've learned a ton from the 1.5.2 code base and therefore can look forward to a much better integration in 1.6.0.

Stay tuned for news about the 1.5.3 release, which should at least take care of the swallowing exception problem.

Cheers,

-Dan

On Wed, Jul 22, 2015 at 8:42 AM, sean.osterberg [via Asciidoctor :: Discussion] <[hidden email]> wrote:

Thanks Dan, I'll try that workaround and see where it leads me.

---

If you reply to this email, your message will be added to the discussion below:

http://discuss.asciidoctor.org/AsciidoctorJ-Block-Processor-Problems-tp3150p3543.html

To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML

--

Dan Allen | http://google.com/profiles/dan.j.allen

---

If you reply to this email, your message will be added to the discussion below:

http://discuss.asciidoctor.org/AsciidoctorJ-Block-Processor-Problems-tp3150p3546.html

To unsubscribe from AsciidoctorJ Block Processor Problems, click here.
NAML

Thanks Dan and Robert.

Is it possible for me to get the AsciidoctorJ 1.6.0 snapshot?

Robert, I sent you the code files and an example file. I'm eager to hear if it's resolved in 1.6, because I'm going to have to figure out a workaround for the time being.

Thanks,
Sean

Hi Sean,

I just tried your example with AsciidoctorJ 1.5.2 and it ... worked on my computer!
I am a actually a little bit surprised myself.

I am responding here because your example very nicely shows where we still have some gaps for 1.6.0 that we have to fill.

First you are using the Ruby runtime inside the processor.
Independent of whether we can provide an API for your concrete problem I think there will always be some point where the user requires the Ruby runtime because we have no solution yet in the product.
Therefore I filed https://github.com/asciidoctor/asciidoctorj/issues/364

Second point is you are using the Ruby runtime to get a wrapper around the Asciidoctor parser.
We already have an issue https://github.com/asciidoctor/asciidoctorj/issues/326 and a PR https://github.com/asciidoctor/asciidoctorj/pull/337 for that.
I think the PR would work for that, so that you'd no longer need to wrap the Parser yourself.

Actually your approach would no longer with AsciidoctorJ 1.6.0 though!
The point is that the Parser object wrapped by JRuby will produce JRuby's own proxy objects which have some drawbacks. For example that the `parse()` method will really always return a Block and not a Section if the Block was a Section.
(To be pedantic the parse method should have returned an AbstractBlock, but then JRuby would really give you only an AbstractBlock and not a Block.
That was the reason why we stepped back from this approach in 1.6.0 and now use an explicit mapping between JRuby and Java nodes.)
But as I said above, the PR would help you with that one.

I am super happy that I could have a look at your sources.
Unfortunately I could not reproduce your problem, maybe you can get the full error message with Dan's hint.

Best regards
Robert