Extension API in Java almost done, but I need some help

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Extension API in Java almost done, but I need some help

asotobu
Hello, I have already done almost all extension API for Java part, but I am having two problems with two kind of extensions (BlockProcessor and InlineMacroProcessor), all other ones work as expected. You can see the test here: https://github.com/asciidoctor/asciidoctor-java-integration/blob/extension/src/test/java/org/asciidoctor/extension/WhenExtensionIsRegistered.java

Let me start with first problem of BlockProcessor:

BlockProcessor is an abstract class for all block processors, and among other methods it had one called config which looks like:

public RubyHash config() {
       
        RubySymbol[] values = new RubySymbol[2];
        values[0] = RubyUtils.toSymbol(rubyRuntime, "paragraph");
        values[1] = RubyUtils.toSymbol(rubyRuntime, "open");
       
        Map<String, Object> context = new HashMap<String, Object>();
        context.put("contexts", values);
       
        return RubyHashUtil.convertMapToRubyHashWithSymbols(rubyRuntime,context);
       
    }


It is not important what it is doing but how it is called (which is config), then a block extension is registered as any other extension and using the same approach used in extensions that work perfectly:

def block_processor(blockSymbol, extensionName)
        Asciidoctor::Extensions.register do |document|
            block blockSymbol, extensionName
        end
end


The problem is that when I execute the test, next exception is thrown:

org.jruby.exceptions.RaiseException: (NoMethodError) undefined method `config' for Java::OrgAsciidoctorExtension::YellBlock:Class
        at RUBY.processor_registered_for_block?(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/extensions.rb:217)
        at RUBY.next_block(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/lexer.rb:578)
        at RUBY.next_section(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/lexer.rb:280)
        at RUBY.parse(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/lexer.rb:52)
        at RUBY.initialize(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/document.rb:329)
        at RUBY.load(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor.rb:805)


There is a method called config but I don't know why but it is not found, I suspect that Ruby part is referring to Java Class YellBlock instead of instance of YellBlock but I am not so sure. Moreover other extensions like for example BlockMacro works as expected using the same approach.

The other problem is with InlineMacroProcessors, in this case I think that it is a problem of integration between JRuby and Asciidoctor Ruby code. Again InlineMacroProcessor is almost the same of BlockMacroProcessor (which it works), but when I execute the test next exception is thrown:

org.jruby.exceptions.RaiseException: (RuntimeError) can't modify frozen module
        at org.jruby.RubyModule.extend_object(org/jruby/RubyModule.java:2008)
        at RUBY.load_inline_macro_processor(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/extensions.rb:251)
        at RUBY.load_inline_macro_processors(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/extensions.rb:256)
        at org.jruby.RubyHash.each(org/jruby/RubyHash.java:1332)
        at org.jruby.RubyEnumerable.map(org/jruby/RubyEnumerable.java:713)
        at RUBY.load_inline_macro_processors(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/extensions.rb:255)
        at RUBY.sub_macros(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/substituters.rb:484)
        at RUBY.apply_subs(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/substituters.rb:79)
        at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1617)
        at RUBY.apply_subs(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/substituters.rb:68)
        at RUBY.content(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/block.rb:66)
        at RUBY.result(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/backends/html5.rb:527)
        at RUBY.render(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/backends/base_template.rb:51)
        at RUBY.render(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/renderer.rb:137)
        at RUBY.render(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/abstract_block.rb:54)
        at RUBY.content(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/abstract_block.rb:60)
        at org.jruby.RubyArray.map(org/jruby/RubyArray.java:2417)
        at RUBY.content(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/abstract_block.rb:60)
        at RUBY.content(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/document.rb:788)


Making some research it seems that there is a problem while iterating a Map but because I am not well versed in Ruby I cannot see the problem.

So although I will continue researching a solution for these problems, if something comes to your mind about what it is happening, don't hesitate to write it and I will try.

Thank you so much, we are almost done!!!!



       



Reply | Threaded
Open this post in threaded view
|

Re: Extension API in Java almost done, but I need some help

mojavelinux
Administrator
Hey Alex!

I'll try to get to a reply tomorrow. I know you've been waiting patiently for some input. You are next on my Asciidoctor list :)

-Dan


On Sun, Sep 22, 2013 at 5:58 AM, asotobu [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Hello, I have already done almost all extension API for Java part, but I am having two problems with two kind of extensions (BlockProcessor and InlineMacroProcessor), all other ones work as expected. You can see the test here: https://github.com/asciidoctor/asciidoctor-java-integration/blob/extension/src/test/java/org/asciidoctor/extension/WhenExtensionIsRegistered.java

Let me start with first problem of BlockProcessor:

BlockProcessor is an abstract class for all block processors, and among other methods it had one called config which looks like:

public RubyHash config() {
       
        RubySymbol[] values = new RubySymbol[2];
        values[0] = RubyUtils.toSymbol(rubyRuntime, "paragraph");
        values[1] = RubyUtils.toSymbol(rubyRuntime, "open");
       
        Map<String, Object> context = new HashMap<String, Object>();
        context.put("contexts", values);
       
        return RubyHashUtil.convertMapToRubyHashWithSymbols(rubyRuntime,context);
       
    }


It is not important what it is doing but how it is called (which is config), then a block extension is registered as any other extension and using the same approach used in extensions that work perfectly:

def block_processor(blockSymbol, extensionName)
        Asciidoctor::Extensions.register do |document|
            block blockSymbol, extensionName
        end
end


The problem is that when I execute the test, next exception is thrown:

org.jruby.exceptions.RaiseException: (NoMethodError) undefined method `config' for Java::OrgAsciidoctorExtension::YellBlock:Class
        at RUBY.processor_registered_for_block?(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/extensions.rb:217)
        at RUBY.next_block(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/lexer.rb:578)
        at RUBY.next_section(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/lexer.rb:280)
        at RUBY.parse(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/lexer.rb:52)
        at RUBY.initialize(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/document.rb:329)
        at RUBY.load(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor.rb:805)


There is a method called config but I don't know why but it is not found, I suspect that Ruby part is referring to Java Class YellBlock instead of instance of YellBlock but I am not so sure. Moreover other extensions like for example BlockMacro works as expected using the same approach.

The other problem is with InlineMacroProcessors, in this case I think that it is a problem of integration between JRuby and Asciidoctor Ruby code. Again InlineMacroProcessor is almost the same of BlockMacroProcessor (which it works), but when I execute the test next exception is thrown:

org.jruby.exceptions.RaiseException: (RuntimeError) can't modify frozen module
        at org.jruby.RubyModule.extend_object(org/jruby/RubyModule.java:2008)
        at RUBY.load_inline_macro_processor(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/extensions.rb:251)
        at RUBY.load_inline_macro_processors(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/extensions.rb:256)
        at org.jruby.RubyHash.each(org/jruby/RubyHash.java:1332)
        at org.jruby.RubyEnumerable.map(org/jruby/RubyEnumerable.java:713)
        at RUBY.load_inline_macro_processors(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/extensions.rb:255)
        at RUBY.sub_macros(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/substituters.rb:484)
        at RUBY.apply_subs(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/substituters.rb:79)
        at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1617)
        at RUBY.apply_subs(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/substituters.rb:68)
        at RUBY.content(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/block.rb:66)
        at RUBY.result(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/backends/html5.rb:527)
        at RUBY.render(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/backends/base_template.rb:51)
        at RUBY.render(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/renderer.rb:137)
        at RUBY.render(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/abstract_block.rb:54)
        at RUBY.content(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/abstract_block.rb:60)
        at org.jruby.RubyArray.map(org/jruby/RubyArray.java:2417)
        at RUBY.content(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/abstract_block.rb:60)
        at RUBY.content(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/document.rb:788)


Making some research it seems that there is a problem while iterating a Map but because I am not well versed in Ruby I cannot see the problem.

So although I will continue researching a solution for these problems, if something comes to your mind about what it is happening, don't hesitate to write it and I will try.

Thank you so much, we are almost done!!!!



       






If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Extension-API-in-Java-almost-done-but-I-need-some-help-tp637.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



--
Reply | Threaded
Open this post in threaded view
|

Re: Extension API in Java almost done, but I need some help

asotobu
Don't worry Dan, I know you have been busy so it is normal to not have all the time to review the code. I have almost done the integration, all kind of extensions work except Block and InlineMacro, maybe I will need to debug Java and Ruby part together but before that I would like to have a second opinion about my suspicion.


Alex.
Reply | Threaded
Open this post in threaded view
|

Re: Extension API in Java almost done, but I need some help

mojavelinux
Administrator
In reply to this post by asotobu
Alex,

I have sent a pull request with a working BlockProcessor!


You had identified the main problem in this statement:

> There is a method called config but I don't know why but it is not found, I suspect that Ruby part is referring to Java Class YellBlock instead of instance of YellBlock but I am not so sure. 

Indeed, Ruby looks for a static method named `config` that returns a Map on the BlockProcessor implementation class. This static configuration contains information about whether the processor is relevant for a given block context (e.g., :open, :paragraph, etc).

When writing an extension in Ruby, the static configuration is defined using the static method `option`, giving a DSL-like feel to it:

```ruby
class YellBlock < Extensions::BlockProcessor
  option :contexts, [:paragraph]
  option :content_model, :simple

  ...
end
```

Here's how we might do that in a Java-based extension:

```java
public class YellBlock extends BlockProcessor {
    static {
        config.put("contexts", Arrays.asList(":paragraph"));
        config.put("content_model", ":simple");
    }

   ...
}
```

The static config field is defined in the BlockProcessor class as follows:

```java
protected static final Map<String, Object> config = new HashMap<String, Object>();
    
public static Map<String, Object> config() {
    return config;
}
```

There's only one problem. Ruby is looking for symbol keys and values, but the config Map only contains strings in our example. Unfortunately, we need a Ruby (runtime) instance to convert strings to symbols (as far as I know). What I've proposed in the patch is to add a static method named `setup` that gets invoked in the ExtensionRegistry, where the Ruby instance is available.

Here's the `setup` method I added to the BlockProcessor class:

```java
public static void setup(Ruby rubyRuntime) {
    // replace values by overwriting collection
    Map<String, Object> raw = config();
    Map<String, Object> converted = RubyHashUtil.convertMapToRubyHashWithSymbols(rubyRuntime, raw);
    raw.clear();
    raw.putAll(converted);
        
    // alternate approach: replace values using assignment
    //config = RubyHashUtil.convertMapToRubyHashWithSymbols(rubyRuntime, config());
    }
```

The conversion between strings and symbols is one of the rougher areas of this integration.

The open question at this point is whether the static config in the Java code above is a reasonable API for extension writers. If you see a better way to define it, or you think there should be an abstraction layer that hides the static block in some way, I'm open to that. Another thing to consider is that maybe it's not a good idea to have static configuration on the processor class at all. Maybe we should change the Ruby code so as not to rely on static state. Feel free to propose changes!

I haven't dug into the InlineMacroProcessor, so I can't provide any insight on that problem just yet.

Again, great work Alex!

-Dan



On Sun, Sep 22, 2013 at 5:58 AM, asotobu [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Hello, I have already done almost all extension API for Java part, but I am having two problems with two kind of extensions (BlockProcessor and InlineMacroProcessor), all other ones work as expected. You can see the test here: https://github.com/asciidoctor/asciidoctor-java-integration/blob/extension/src/test/java/org/asciidoctor/extension/WhenExtensionIsRegistered.java

Let me start with first problem of BlockProcessor:

BlockProcessor is an abstract class for all block processors, and among other methods it had one called config which looks like:

public RubyHash config() {
       
        RubySymbol[] values = new RubySymbol[2];
        values[0] = RubyUtils.toSymbol(rubyRuntime, "paragraph");
        values[1] = RubyUtils.toSymbol(rubyRuntime, "open");
       
        Map<String, Object> context = new HashMap<String, Object>();
        context.put("contexts", values);
       
        return RubyHashUtil.convertMapToRubyHashWithSymbols(rubyRuntime,context);
       
    }


It is not important what it is doing but how it is called (which is config), then a block extension is registered as any other extension and using the same approach used in extensions that work perfectly:

def block_processor(blockSymbol, extensionName)
        Asciidoctor::Extensions.register do |document|
            block blockSymbol, extensionName
        end
end


The problem is that when I execute the test, next exception is thrown:

org.jruby.exceptions.RaiseException: (NoMethodError) undefined method `config' for Java::OrgAsciidoctorExtension::YellBlock:Class
        at RUBY.processor_registered_for_block?(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/extensions.rb:217)
        at RUBY.next_block(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/lexer.rb:578)
        at RUBY.next_section(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/lexer.rb:280)
        at RUBY.parse(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/lexer.rb:52)
        at RUBY.initialize(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/document.rb:329)
        at RUBY.load(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor.rb:805)


There is a method called config but I don't know why but it is not found, I suspect that Ruby part is referring to Java Class YellBlock instead of instance of YellBlock but I am not so sure. Moreover other extensions like for example BlockMacro works as expected using the same approach.

The other problem is with InlineMacroProcessors, in this case I think that it is a problem of integration between JRuby and Asciidoctor Ruby code. Again InlineMacroProcessor is almost the same of BlockMacroProcessor (which it works), but when I execute the test next exception is thrown:

org.jruby.exceptions.RaiseException: (RuntimeError) can't modify frozen module
        at org.jruby.RubyModule.extend_object(org/jruby/RubyModule.java:2008)
        at RUBY.load_inline_macro_processor(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/extensions.rb:251)
        at RUBY.load_inline_macro_processors(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/extensions.rb:256)
        at org.jruby.RubyHash.each(org/jruby/RubyHash.java:1332)
        at org.jruby.RubyEnumerable.map(org/jruby/RubyEnumerable.java:713)
        at RUBY.load_inline_macro_processors(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/extensions.rb:255)
        at RUBY.sub_macros(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/substituters.rb:484)
        at RUBY.apply_subs(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/substituters.rb:79)
        at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1617)
        at RUBY.apply_subs(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/substituters.rb:68)
        at RUBY.content(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/block.rb:66)
        at RUBY.result(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/backends/html5.rb:527)
        at RUBY.render(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/backends/base_template.rb:51)
        at RUBY.render(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/renderer.rb:137)
        at RUBY.render(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/abstract_block.rb:54)
        at RUBY.content(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/abstract_block.rb:60)
        at org.jruby.RubyArray.map(org/jruby/RubyArray.java:2417)
        at RUBY.content(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/abstract_block.rb:60)
        at RUBY.content(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/document.rb:788)


Making some research it seems that there is a problem while iterating a Map but because I am not well versed in Ruby I cannot see the problem.

So although I will continue researching a solution for these problems, if something comes to your mind about what it is happening, don't hesitate to write it and I will try.

Thank you so much, we are almost done!!!!



       






If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Extension-API-in-Java-almost-done-but-I-need-some-help-tp637.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



--
Reply | Threaded
Open this post in threaded view
|

Re: Extension API in Java almost done, but I need some help

mojavelinux
Administrator
In reply to this post by asotobu
I've added another commit to the pull request that gets the InlineMacroProcessor working, though not fully implemented.

https://github.com/mojavelinux/asciidoctor-java-integration/commit/0b2107a609821143b391e8e479f8dedaef335217

This problem:

> org.jruby.exceptions.RaiseException: (RuntimeError) can't modify frozen module 

was due to the fact that the constructor parameters were out of order. Once I corrected that problem, everything seemed fine.

The InlineMacroProcessor will need access to a createInline() method that parallels the createBlock() method. For now, I just hard coded some HTML in the ManpageMacro to see it functioning.

Only one more test to get working and we've got the foundation in place!

Cheers!

-Dan


On Wed, Oct 2, 2013 at 2:41 AM, Dan Allen <[hidden email]> wrote:
Alex,

I have sent a pull request with a working BlockProcessor!


You had identified the main problem in this statement:

> There is a method called config but I don't know why but it is not found, I suspect that Ruby part is referring to Java Class YellBlock instead of instance of YellBlock but I am not so sure. 

Indeed, Ruby looks for a static method named `config` that returns a Map on the BlockProcessor implementation class. This static configuration contains information about whether the processor is relevant for a given block context (e.g., :open, :paragraph, etc).

When writing an extension in Ruby, the static configuration is defined using the static method `option`, giving a DSL-like feel to it:

```ruby
class YellBlock < Extensions::BlockProcessor
  option :contexts, [:paragraph]
  option :content_model, :simple

  ...
end
```

Here's how we might do that in a Java-based extension:

```java
public class YellBlock extends BlockProcessor {
    static {
        config.put("contexts", Arrays.asList(":paragraph"));
        config.put("content_model", ":simple");
    }

   ...
}
```

The static config field is defined in the BlockProcessor class as follows:

```java
protected static final Map<String, Object> config = new HashMap<String, Object>();
    
public static Map<String, Object> config() {
    return config;
}
```

There's only one problem. Ruby is looking for symbol keys and values, but the config Map only contains strings in our example. Unfortunately, we need a Ruby (runtime) instance to convert strings to symbols (as far as I know). What I've proposed in the patch is to add a static method named `setup` that gets invoked in the ExtensionRegistry, where the Ruby instance is available.

Here's the `setup` method I added to the BlockProcessor class:

```java
public static void setup(Ruby rubyRuntime) {
    // replace values by overwriting collection
    Map<String, Object> raw = config();
    Map<String, Object> converted = RubyHashUtil.convertMapToRubyHashWithSymbols(rubyRuntime, raw);
    raw.clear();
    raw.putAll(converted);
        
    // alternate approach: replace values using assignment
    //config = RubyHashUtil.convertMapToRubyHashWithSymbols(rubyRuntime, config());
    }
```

The conversion between strings and symbols is one of the rougher areas of this integration.

The open question at this point is whether the static config in the Java code above is a reasonable API for extension writers. If you see a better way to define it, or you think there should be an abstraction layer that hides the static block in some way, I'm open to that. Another thing to consider is that maybe it's not a good idea to have static configuration on the processor class at all. Maybe we should change the Ruby code so as not to rely on static state. Feel free to propose changes!

I haven't dug into the InlineMacroProcessor, so I can't provide any insight on that problem just yet.

Again, great work Alex!

-Dan



On Sun, Sep 22, 2013 at 5:58 AM, asotobu [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Hello, I have already done almost all extension API for Java part, but I am having two problems with two kind of extensions (BlockProcessor and InlineMacroProcessor), all other ones work as expected. You can see the test here: https://github.com/asciidoctor/asciidoctor-java-integration/blob/extension/src/test/java/org/asciidoctor/extension/WhenExtensionIsRegistered.java

Let me start with first problem of BlockProcessor:

BlockProcessor is an abstract class for all block processors, and among other methods it had one called config which looks like:

public RubyHash config() {
       
        RubySymbol[] values = new RubySymbol[2];
        values[0] = RubyUtils.toSymbol(rubyRuntime, "paragraph");
        values[1] = RubyUtils.toSymbol(rubyRuntime, "open");
       
        Map<String, Object> context = new HashMap<String, Object>();
        context.put("contexts", values);
       
        return RubyHashUtil.convertMapToRubyHashWithSymbols(rubyRuntime,context);
       
    }


It is not important what it is doing but how it is called (which is config), then a block extension is registered as any other extension and using the same approach used in extensions that work perfectly:

def block_processor(blockSymbol, extensionName)
        Asciidoctor::Extensions.register do |document|
            block blockSymbol, extensionName
        end
end


The problem is that when I execute the test, next exception is thrown:

org.jruby.exceptions.RaiseException: (NoMethodError) undefined method `config' for Java::OrgAsciidoctorExtension::YellBlock:Class
        at RUBY.processor_registered_for_block?(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/extensions.rb:217)
        at RUBY.next_block(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/lexer.rb:578)
        at RUBY.next_section(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/lexer.rb:280)
        at RUBY.parse(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/lexer.rb:52)
        at RUBY.initialize(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/document.rb:329)
        at RUBY.load(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor.rb:805)


There is a method called config but I don't know why but it is not found, I suspect that Ruby part is referring to Java Class YellBlock instead of instance of YellBlock but I am not so sure. Moreover other extensions like for example BlockMacro works as expected using the same approach.

The other problem is with InlineMacroProcessors, in this case I think that it is a problem of integration between JRuby and Asciidoctor Ruby code. Again InlineMacroProcessor is almost the same of BlockMacroProcessor (which it works), but when I execute the test next exception is thrown:

org.jruby.exceptions.RaiseException: (RuntimeError) can't modify frozen module
        at org.jruby.RubyModule.extend_object(org/jruby/RubyModule.java:2008)
        at RUBY.load_inline_macro_processor(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/extensions.rb:251)
        at RUBY.load_inline_macro_processors(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/extensions.rb:256)
        at org.jruby.RubyHash.each(org/jruby/RubyHash.java:1332)
        at org.jruby.RubyEnumerable.map(org/jruby/RubyEnumerable.java:713)
        at RUBY.load_inline_macro_processors(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/extensions.rb:255)
        at RUBY.sub_macros(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/substituters.rb:484)
        at RUBY.apply_subs(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/substituters.rb:79)
        at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1617)
        at RUBY.apply_subs(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/substituters.rb:68)
        at RUBY.content(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/block.rb:66)
        at RUBY.result(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/backends/html5.rb:527)
        at RUBY.render(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/backends/base_template.rb:51)
        at RUBY.render(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/renderer.rb:137)
        at RUBY.render(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/abstract_block.rb:54)
        at RUBY.content(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/abstract_block.rb:60)
        at org.jruby.RubyArray.map(org/jruby/RubyArray.java:2417)
        at RUBY.content(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/abstract_block.rb:60)
        at RUBY.content(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/document.rb:788)


Making some research it seems that there is a problem while iterating a Map but because I am not well versed in Ruby I cannot see the problem.

So although I will continue researching a solution for these problems, if something comes to your mind about what it is happening, don't hesitate to write it and I will try.

Thank you so much, we are almost done!!!!



       






If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Extension-API-in-Java-almost-done-but-I-need-some-help-tp637.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML





--
Reply | Threaded
Open this post in threaded view
|

Re: Extension API in Java almost done, but I need some help

asotobu
Sorry for that, it was a problem of copy/paste and I have not seen. Thank you so much.

Alex.


2013/10/2 mojavelinux [via Asciidoctor :: Discussion] <[hidden email]>
I've added another commit to the pull request that gets the InlineMacroProcessor working, though not fully implemented.

https://github.com/mojavelinux/asciidoctor-java-integration/commit/0b2107a609821143b391e8e479f8dedaef335217

This problem:

> org.jruby.exceptions.RaiseException: (RuntimeError) can't modify frozen module 

was due to the fact that the constructor parameters were out of order. Once I corrected that problem, everything seemed fine.

The InlineMacroProcessor will need access to a createInline() method that parallels the createBlock() method. For now, I just hard coded some HTML in the ManpageMacro to see it functioning.

Only one more test to get working and we've got the foundation in place!

Cheers!

-Dan


On Wed, Oct 2, 2013 at 2:41 AM, Dan Allen <[hidden email]> wrote:
Alex,

I have sent a pull request with a working BlockProcessor!


You had identified the main problem in this statement:

> There is a method called config but I don't know why but it is not found, I suspect that Ruby part is referring to Java Class YellBlock instead of instance of YellBlock but I am not so sure. 

Indeed, Ruby looks for a static method named `config` that returns a Map on the BlockProcessor implementation class. This static configuration contains information about whether the processor is relevant for a given block context (e.g., :open, :paragraph, etc).

When writing an extension in Ruby, the static configuration is defined using the static method `option`, giving a DSL-like feel to it:

```ruby
class YellBlock < Extensions::BlockProcessor
  option :contexts, [:paragraph]
  option :content_model, :simple

  ...
end
```

Here's how we might do that in a Java-based extension:

```java
public class YellBlock extends BlockProcessor {
    static {
        config.put("contexts", Arrays.asList(":paragraph"));
        config.put("content_model", ":simple");
    }

   ...
}
```

The static config field is defined in the BlockProcessor class as follows:

```java
protected static final Map<String, Object> config = new HashMap<String, Object>();
    
public static Map<String, Object> config() {
    return config;
}
```

There's only one problem. Ruby is looking for symbol keys and values, but the config Map only contains strings in our example. Unfortunately, we need a Ruby (runtime) instance to convert strings to symbols (as far as I know). What I've proposed in the patch is to add a static method named `setup` that gets invoked in the ExtensionRegistry, where the Ruby instance is available.

Here's the `setup` method I added to the BlockProcessor class:

```java
public static void setup(Ruby rubyRuntime) {
    // replace values by overwriting collection
    Map<String, Object> raw = config();
    Map<String, Object> converted = RubyHashUtil.convertMapToRubyHashWithSymbols(rubyRuntime, raw);
    raw.clear();
    raw.putAll(converted);
        
    // alternate approach: replace values using assignment
    //config = RubyHashUtil.convertMapToRubyHashWithSymbols(rubyRuntime, config());
    }
```

The conversion between strings and symbols is one of the rougher areas of this integration.

The open question at this point is whether the static config in the Java code above is a reasonable API for extension writers. If you see a better way to define it, or you think there should be an abstraction layer that hides the static block in some way, I'm open to that. Another thing to consider is that maybe it's not a good idea to have static configuration on the processor class at all. Maybe we should change the Ruby code so as not to rely on static state. Feel free to propose changes!

I haven't dug into the InlineMacroProcessor, so I can't provide any insight on that problem just yet.

Again, great work Alex!

-Dan



On Sun, Sep 22, 2013 at 5:58 AM, asotobu [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Hello, I have already done almost all extension API for Java part, but I am having two problems with two kind of extensions (BlockProcessor and InlineMacroProcessor), all other ones work as expected. You can see the test here: https://github.com/asciidoctor/asciidoctor-java-integration/blob/extension/src/test/java/org/asciidoctor/extension/WhenExtensionIsRegistered.java

Let me start with first problem of BlockProcessor:

BlockProcessor is an abstract class for all block processors, and among other methods it had one called config which looks like:

public RubyHash config() {
       
        RubySymbol[] values = new RubySymbol[2];
        values[0] = RubyUtils.toSymbol(rubyRuntime, "paragraph");
        values[1] = RubyUtils.toSymbol(rubyRuntime, "open");
       
        Map<String, Object> context = new HashMap<String, Object>();
        context.put("contexts", values);
       
        return RubyHashUtil.convertMapToRubyHashWithSymbols(rubyRuntime,context);
       
    }


It is not important what it is doing but how it is called (which is config), then a block extension is registered as any other extension and using the same approach used in extensions that work perfectly:

def block_processor(blockSymbol, extensionName)
        Asciidoctor::Extensions.register do |document|
            block blockSymbol, extensionName
        end
end


The problem is that when I execute the test, next exception is thrown:

org.jruby.exceptions.RaiseException: (NoMethodError) undefined method `config' for Java::OrgAsciidoctorExtension::YellBlock:Class
        at RUBY.processor_registered_for_block?(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/extensions.rb:217)
        at RUBY.next_block(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/lexer.rb:578)
        at RUBY.next_section(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/lexer.rb:280)
        at RUBY.parse(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/lexer.rb:52)
        at RUBY.initialize(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/document.rb:329)
        at RUBY.load(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor.rb:805)


There is a method called config but I don't know why but it is not found, I suspect that Ruby part is referring to Java Class YellBlock instead of instance of YellBlock but I am not so sure. Moreover other extensions like for example BlockMacro works as expected using the same approach.

The other problem is with InlineMacroProcessors, in this case I think that it is a problem of integration between JRuby and Asciidoctor Ruby code. Again InlineMacroProcessor is almost the same of BlockMacroProcessor (which it works), but when I execute the test next exception is thrown:

org.jruby.exceptions.RaiseException: (RuntimeError) can't modify frozen module
        at org.jruby.RubyModule.extend_object(org/jruby/RubyModule.java:2008)
        at RUBY.load_inline_macro_processor(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/extensions.rb:251)
        at RUBY.load_inline_macro_processors(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/extensions.rb:256)
        at org.jruby.RubyHash.each(org/jruby/RubyHash.java:1332)
        at org.jruby.RubyEnumerable.map(org/jruby/RubyEnumerable.java:713)
        at RUBY.load_inline_macro_processors(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/extensions.rb:255)
        at RUBY.sub_macros(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/substituters.rb:484)
        at RUBY.apply_subs(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/substituters.rb:79)
        at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1617)
        at RUBY.apply_subs(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/substituters.rb:68)
        at RUBY.content(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/block.rb:66)
        at RUBY.result(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/backends/html5.rb:527)
        at RUBY.render(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/backends/base_template.rb:51)
        at RUBY.render(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/renderer.rb:137)
        at RUBY.render(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/abstract_block.rb:54)
        at RUBY.content(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/abstract_block.rb:60)
        at org.jruby.RubyArray.map(org/jruby/RubyArray.java:2417)
        at RUBY.content(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/abstract_block.rb:60)
        at RUBY.content(/home/alex/git/asciidoctor-java-integration/target/test-classes/gems/asciidoctor-0.1.4/lib/asciidoctor/document.rb:788)


Making some research it seems that there is a problem while iterating a Map but because I am not well versed in Ruby I cannot see the problem.

So although I will continue researching a solution for these problems, if something comes to your mind about what it is happening, don't hesitate to write it and I will try.

Thank you so much, we are almost done!!!!



       






If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Extension-API-in-Java-almost-done-but-I-need-some-help-tp637.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML





--



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Extension-API-in-Java-almost-done-but-I-need-some-help-tp637p674.html
To unsubscribe from Extension API in Java almost done, but I need some help, click here.
NAML



--
+----------------------------------------------------------+
  Alex Soto Bueno - Computer Engineer
  www.lordofthejars.com
+----------------------------------------------------------+