AsciidoctorJ Block Processor Problems

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
65 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

AsciidoctorJ Block Processor Problems

sean.osterberg
This post was updated on .
Hi guys, I'm writing a block processor and have been leaning heavily on the AsciidoctorJ examples to help me. After running into some issues, I've also been referring to these "docs" here: https://github.com/asciidoctor/asciidoctor-documentation/issues/30, which outline the content_model and contexts. I can't get the following to process properly, and I don't know why. Specifically, the process method doesn't get called. Could you please tell me what I'm doing wrong? I think the docs need to be updated, and I'm happy to make a contribution once I figure out what's going on here.

Here's my version of the YellBlock processor. I've tried multiple combinations of contexts and content_models, but this seems the most logical given the issue I linked above:

public class YellBlock extends BlockProcessor {
    private final Asciidoctor asciidoctor;

    private static Map<String, Object> configs = new HashMap<String, Object>();

    static {
        configs.put("contexts", Arrays.asList(":listing"));
        configs.put("content_models", ":compound");
    }

    public YellBlock(String name, Asciidoctor asciidoctor) {
        super(name, configs);
        this.asciidoctor = asciidoctor;
    }

    @Override
    public Object process(AbstractBlock parent, Reader reader, Map<String, Object> attributes) {
        List<String> lines = reader.readLines();
        String upperLines = null;
        for (String line : lines) {
            if (upperLines == null) {
                upperLines = line.toUpperCase();
            }
            else {
                upperLines = upperLines + "\n" + line.toUpperCase();
            }
        }

        return createBlock(parent, "paragraph", Arrays.asList(upperLines), attributes, new HashMap<Object, Object>());
    }
}

And here is the sample file that's not getting processed correctly:

= Tabs test

this page is an example of tabs

[[tabs]]
----
Make this text uppercase.
----

My ultimate goal, however, is to process tabs in the following manner, so any guidance is welcome:

[[tabs]]
----
[tab,title="Foo"]
....
This is the foo tab
....
[tab,title="Bar"]
....
This is the bar tab
....
----
Reply | Threaded
Open this post in threaded view
|

Re: AsciidoctorJ Block Processor Problems

Robert.Panzer
Hi Sean,

I think you had some minor errors in your code that made your example not behave as expected.

First, the config option is only
content_model
 instead of
content_models
I tested with
:compound
 and
:simple
, both worked for me, I don't understand yet exactly what that means either, but it must be set apparently ;-)

Then your example text contained
[[tabs]]
, which is actually a reference target, instead of only
[tabs]

So this is in short what I had to do to get your example running.
I got it running on the current master branch and 1.6.0, so it should also run with asciidoctorJ 1.5.2.

Do you want to have multiple source codes in your document at one location switchable via tabs like they do in the Oracle Java magazine?
If so this would be very cool!

Hope I could help you to continue. If you still have problems just post a response, I could provide an example on gist or sth like that.

Best regards,
Robert
Reply | Threaded
Open this post in threaded view
|

Re: AsciidoctorJ Block Processor Problems

mojavelinux
Administrator
Robert.Panzer wrote:

First, the config option is only
content_model
 instead of
content_models
I tested with
:compound
 and
:simple
, both worked for me, I don't understand yet exactly what that means either, but it must be set apparently ;-)

The content model controls how the content of the block is parsed. If the value is :compound, it supports nested blocks. If it is simple, the content is parsed with paragraph rules. Not all values can be used all the time, but when a specific strategy is needed, this option provides the necessary clarity.
 

Then your example text contained
[[tabs]]
, which is actually a reference target, instead of only
[tabs]
Oops, can't believe I missed that one.

Thanks for the input, Robert!

Cheers,

-Dan

--
Reply | Threaded
Open this post in threaded view
|

Re: AsciidoctorJ Block Processor Problems

Robert.Panzer
Dan, I think that these string constants might often cause problems that are hard to find.
What do you think about adding constants for the most common config options for extensions  to the Processor class or the subclass that introduces a constant?
Another advantage besides reducing the number of typos is that we can add a nice javadoc that explains the parameters and their values.

Best regards
Robert
Reply | Threaded
Open this post in threaded view
|

Re: AsciidoctorJ Block Processor Problems

LightGuardjp
Sounds like a good idea to me. 

On Sunday, May 10, 2015, Robert.Panzer [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Dan, I think that these string constants might often cause problems that are hard to find.
What do you think about adding constants for the most common config options for extensions  to the Processor class or the subclass that introduces a constant?
Another advantage besides reducing the number of typos is that we can add a nice javadoc that explains the parameters and their values.

Best regards
Robert


If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/AsciidoctorJ-Block-Processor-Problems-tp3150p3158.html
To start a new topic under Asciidoctor :: Discussion, email <a href="javascript:_e(%7B%7D,&#39;cvml&#39;,&#39;ml-node%2Bs49171n1h37@n6.nabble.com&#39;);" target="_blank">ml-node+s49171n1h37@...
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML


--
Sent from Gmail Mobile
Reply | Threaded
Open this post in threaded view
|

Re: AsciidoctorJ Block Processor Problems

abelsromero
I agree with Robert, I have suffered myself this configuration.
Such constants will not only help to avoid typos but will also make it easier to see the available options.
Reply | Threaded
Open this post in threaded view
|

Re: AsciidoctorJ Block Processor Problems

mojavelinux
Administrator
In reply to this post by LightGuardjp
Absolutely.

Again, the list can be found here:


On Sun, May 10, 2015 at 7:41 AM, LightGuardjp [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Sounds like a good idea to me. 

On Sunday, May 10, 2015, Robert.Panzer [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Dan, I think that these string constants might often cause problems that are hard to find.
What do you think about adding constants for the most common config options for extensions  to the Processor class or the subclass that introduces a constant?
Another advantage besides reducing the number of typos is that we can add a nice javadoc that explains the parameters and their values.

Best regards
Robert


If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/AsciidoctorJ-Block-Processor-Problems-tp3150p3158.html
To start a new topic under Asciidoctor :: Discussion, email <a href="javascript:_e(%7B%7D,&#39;cvml&#39;,&#[hidden email]&#39;);" target="_blank">ml-node+s49171n1h37@...
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML


--
Sent from Gmail Mobile



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/AsciidoctorJ-Block-Processor-Problems-tp3150p3160.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



--
Reply | Threaded
Open this post in threaded view
|

Re: AsciidoctorJ Block Processor Problems

sean.osterberg
In reply to this post by Robert.Panzer
Thank you everyone for the help! I misunderstood how I can use
[[tabs]]
 and
[tabs]
 as block headers.

Robert, yes, the goal is exactly that. I'm finding it difficult to implement, however, mainly because of how the blocks are processed. For example, if you have the following declaration:

[tabs]
----
[tab,title="foo"]
--
here is some example code:
[source,java]
----
Public static void main(String[] args) { //foo; }
----
--
----

The processor can't "look ahead" to see that the second instance of
----
 shouldn't immediately close the block, if that makes sense. So using the syntax above, I don't think I can have a code snippet inside the tab. Anyone have any thoughts, specifically on how I can use a similar syntax above but include code snippets and other Asciidoctor formatting?
Reply | Threaded
Open this post in threaded view
|

Re: AsciidoctorJ Block Processor Problems

mojavelinux
Administrator

On Mon, May 11, 2015 at 1:56 PM, sean.osterberg [via Asciidoctor :: Discussion] <[hidden email]> wrote:
The processor can't "look ahead" to see that the second instance of
----
 shouldn't immediately close the block, if that makes sense. So using the syntax above, I don't think I can have a code snippet inside the tab. Anyone have any thoughts, specifically on how I can use a similar syntax above but include code snippets and other Asciidoctor formatting?

Asciidoctor does support this nesting. There are two ways to do it.

1) If you just have listing blocks inside of another listing block, switch either the parent or child to using the literal block syntax instead.

[tabs]
....
[source]
----
nested listing block
----
....


2) You can use different lengths for the delimited block boundaries to support nesting.

[tabs]
------
[source]
----
nested listing block
----
------

I'm not sure that a literal block is the right choice for the tabs block, though. I recommend either an open block or an example block. Open blocks can't be nested, though, so if there's a chance you have open blocks, then you need to use the example block. We you define the context, we say that this is the "structural container" since it is merely capturing the content to be interpreted by the extension.

Cheers,

-Dan

--
Reply | Threaded
Open this post in threaded view
|

Re: AsciidoctorJ Block Processor Problems

mojavelinux
Administrator
In reply to this post by sean.osterberg

On Mon, May 11, 2015 at 1:56 PM, sean.osterberg [via Asciidoctor :: Discussion] <[hidden email]> wrote:
[[tabs]]
This assigns an id to the block.
 
 and
[tabs]
 
This assigns an internal name (also called a style) to the block. Block extensions are wired to a block name.

-Dan


--
Reply | Threaded
Open this post in threaded view
|

Re: AsciidoctorJ Block Processor Problems

sean.osterberg
Thanks again Dan, the explanation is very helpful. After trying multiple things, I have a dumb question: what's the best way to process child blocks within a block processor?

I just tried to use a StructuredDocument via asciidoctor.readDocumentStructure() method and get the child blocks that way. Unfortunately I get the odd error of "Failed to parse source, undefined method `readDocumentStructure' for {}:Hash". This is my code:
public Object process(AbstractBlock parent, Reader reader, Map<String, Object> attributes) {
        String buffer = reader.read();
        StructuredDocument doc = asciidoctor.readDocumentStructure(buffer, new HashMap<String, Object>());
        . . .
}

Other ways I can think of are less than ideal, such as writing my own regex to process the blocks within the block.

And another question: my goal is to output HTML tabs for this functionality. Is the best way to go about this by injecting passthrough blocks (containing the desired output HTML) while I'm processing the block?
Reply | Threaded
Open this post in threaded view
|

Re: AsciidoctorJ Block Processor Problems

mojavelinux
Administrator

On Mon, May 11, 2015 at 6:31 PM, sean.osterberg [via Asciidoctor :: Discussion] <[hidden email]> wrote:
what's the best way to process child blocks within a block processor?

The simplest way is to create an block (typically an open block) and pass in the source lines captured by the reader (the reader is just the lines inside the block boundaries). Calling this method will force the content to be parsed and you'll be able to access the children before returning.

AbstractBlock block = createBlock(parent, "open", reader.readLines(), attributes, new HashMap<String, Object>());
System.out.println(block.getBlocks());

(we could clean up the createBlock so the attributes and option hashes are optional).

That should get you started.

There's an additional method in the extension API that's not exposed to AsciidoctorJ yet. Here's how it's done in Ruby:

wrapper = create_block parent, :open, [], {}
parse_content wrapper, reader.lines
puts wrapper.blocks

The `parse_content` method uses the low-level Parser in Asciidoctor to parse each block in turn. However, we haven't settled on the name of the method yet (or it's return value), so I haven't recommend it be exposed in AsciidoctorJ.

-Dan
Reply | Threaded
Open this post in threaded view
|

Re: AsciidoctorJ Block Processor Problems

mojavelinux
Administrator
In reply to this post by sean.osterberg

On Mon, May 11, 2015 at 6:31 PM, sean.osterberg [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Other ways I can think of are less than ideal, such as writing my own regex to process the blocks within the block.

This should definitely be avoided. I'd rather make an urgent change to core or AsciidoctorJ than for you to resort to this...no matter what the context :) We want clean APIs.

Reply | Threaded
Open this post in threaded view
|

Re: AsciidoctorJ Block Processor Problems

mojavelinux
Administrator
In reply to this post by sean.osterberg

On Mon, May 11, 2015 at 6:31 PM, sean.osterberg [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Is the best way to go about this by injecting passthrough blocks (containing the desired output HTML) while I'm processing the block?

That's one way. You can see I do that for the gist extension:


Another way is to create a custom block name (called "tab") that maps to your own convert handler for that node name. You'd then need to provide a template that would handle it. The set of node names in Asciidoctor is open. You can add additional ones and it will look for those notes when converting.

...but the easiest way is to start with a pass block and then mature the extension by trying to get it to a proper AST model.

Reply | Threaded
Open this post in threaded view
|

Re: AsciidoctorJ Block Processor Problems

mojavelinux
Administrator
In reply to this post by sean.osterberg

On Mon, May 11, 2015 at 7:16 PM, Dan Allen <[hidden email]> wrote:
Calling `createBlock` will force the content to be parsed and you'll be able to access the children before returning.

Actually, I was wrong. Parsing of the lines is a separate step. AsciidoctorJ does not expose the necessary methods. We need to fix this.

I propose that an option to createBlock is "parse" which can be set to true or false. If false, the method will do the necessary work internally to parse the content into that new block. Can you file an issue to add a parse switch to createBlock?

While we're at it, we should overload createBlock so that the attributes and options arguments are optional.

Reply | Threaded
Open this post in threaded view
|

Re: AsciidoctorJ Block Processor Problems

Robert.Panzer
Reply | Threaded
Open this post in threaded view
|

Re: AsciidoctorJ Block Processor Problems

mojavelinux
Administrator
Nice!

-Dan

On Mon, May 11, 2015 at 11:08 PM, Robert.Panzer [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Options are actually already optional for 1.6.0:-)
https://github.com/asciidoctor/asciidoctorj/blob/asciidoctorj-1.6.0/asciidoctorj-core/src/main/java/org/asciidoctor/extension/Processor.java#L40

Best regards
Robert


If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/AsciidoctorJ-Block-Processor-Problems-tp3150p3192.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



--
Reply | Threaded
Open this post in threaded view
|

Re: AsciidoctorJ Block Processor Problems

Robert.Panzer
Dan, should Attributes be optional too?
If so is the current order of parameters correct?
That is, it is more common to pass attributes as it is to pass a config?

Best regards
Robert 



Am 12.05.2015 um 08:03 schrieb mojavelinux [via Asciidoctor :: Discussion] <[hidden email]>:

Nice!

-Dan

On Mon, May 11, 2015 at 11:08 PM, Robert.Panzer [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Options are actually already optional for 1.6.0:-)
https://github.com/asciidoctor/asciidoctorj/blob/asciidoctorj-1.6.0/asciidoctorj-core/src/main/java/org/asciidoctor/extension/Processor.java#L40

Best regards
Robert


If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/AsciidoctorJ-Block-Processor-Problems-tp3150p3192.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



--



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/AsciidoctorJ-Block-Processor-Problems-tp3150p3193.html
To unsubscribe from AsciidoctorJ Block Processor Problems, click here.
NAML
Reply | Threaded
Open this post in threaded view
|

Re: AsciidoctorJ Block Processor Problems

mojavelinux
Administrator

Yep, I think both should be optional and, indeed, the current order is ideal.

If others disagree, please chime in.

Cheers,

-Dan

Le 12 mai 2015 2:40 AM, "Robert.Panzer [via Asciidoctor :: Discussion]" <[hidden email]> a écrit :
Dan, should Attributes be optional too?
If so is the current order of parameters correct?
That is, it is more common to pass attributes as it is to pass a config?

Best regards
Robert 



Am 12.05.2015 um 08:03 schrieb mojavelinux [via Asciidoctor :: Discussion] <[hidden email]>:

Nice!

-Dan

On Mon, May 11, 2015 at 11:08 PM, Robert.Panzer [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Options are actually already optional for 1.6.0:-)
https://github.com/asciidoctor/asciidoctorj/blob/asciidoctorj-1.6.0/asciidoctorj-core/src/main/java/org/asciidoctor/extension/Processor.java#L40

Best regards
Robert


If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/AsciidoctorJ-Block-Processor-Problems-tp3150p3192.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



--



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/AsciidoctorJ-Block-Processor-Problems-tp3150p3193.html
To unsubscribe from AsciidoctorJ Block Processor Problems, click here.
NAML



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/AsciidoctorJ-Block-Processor-Problems-tp3150p3198.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML
Reply | Threaded
Open this post in threaded view
|

Re: AsciidoctorJ Block Processor Problems

sean.osterberg
Sorry for the late reply -- I've only been able to give this a try today. I've run into some more issues after debugging most of the evening.

I'm seeing some interesting behavior, and it's likely that I'm doing something wrong. I'm able to successfully start processing a block like below:

[tabs]
------
[tab,title="foo"]
....
foo
....
------

When I call
List<String> lines = reader.readLines();
, it successfully gets all of the lines. However, I can't seem to create a block to process child blocks using Dan's code above. This is the complete code of my method:

@Override
    public Object process(AbstractBlock parent, Reader reader, Map<String, Object> attributes) {

        AbstractBlock block = createBlock(parent, "literal", reader.readLines(), attributes, new HashMap<Object, Object>());
        System.out.println(block.blocks());
        for(AbstractBlock b : block.blocks()) {
            System.out.println("Block contents:");
            System.out.println(b.content().toString());
        }

        List<String> lines = reader.readLines();
        String upperLines = null;
        for (String line : lines) {
            if (upperLines == null) {
                upperLines = line.toUpperCase();
            }
            else {
                upperLines = upperLines + "\n" + line.toUpperCase();
            }
        }

        return createBlock(parent, "paragraph", Arrays.asList(upperLines), attributes, new HashMap<Object, Object>());
    }

As soon as I get to the loop for
for(AbstractBlock b : block.blocks())
, it immediately exits. I can't seem to get any children blocks from the new block that I've created. As a result, it makes it hard to process the nested tab blocks. I could use regex but I'm trying to do things in the official manner. :)

Any thoughts?

Also, it took me quite a while to debug this error: (NameError) uninitialized constant <extension-name>. This would appear every time I tried to parse an Asciidoctor file when my extension was registered. Through random luck, I finally got rid of it by putting all of my code into a package. I'm not the best Java dev so there might be something I'm not understanding, but it seems odd that things wouldn't work otherwise.

Thanks for any help you can provide.
1234