Include with Glob expression

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Include with Glob expression

asotobu
Hello,

in Asciidoctor we can include external files into one document by using the include:

include::basics.adoc[]

I think it could be a nice addition if we could use a glob expression instance of a concrete file so, for example we can do something like:

include::**/narrative.txt[]

Basically I am thinking about next use case:

Developers are using Thucydides (and JBehave) for BDD. In Thucydides you can add files called narrative.txt where you can describe using plain text a description of the scenario under test. Thucydides supports AsciiDoc format in narrative.txt files but they are rendered when you run Thucydides report tool. The problem is that maybe you want to have all these scenario files being a part of your project documentation. With current include approach, you will need to add one by one all narrative files inside a parent adoc file, which could be very tedious and hard to maintain. On the other hand if you could do something like

include::**/narrative.txt[]

Then every time you render your documentation, you can be assure that the last scenarios are inserted into rendered files.

What do you think? If you find useful I can create an issue on github and implement a possible solution.

Alex.
Reply | Threaded
Open this post in threaded view
|

Re: Include with Glob expression

mojavelinux
Administrator
Alex,

I love this idea.

This feature will be unique to Asciidoctor since there's no way to extend macros like include::[] in AsciiDoc. But, it will be a much welcomed unique feature :)

One way to support this feature is to use the new extension point I've been working on for the include macro in 0.1.4. Here's how it might be implemented just to give you a taste of how the extension point works:

class GlobIncludeHandler
  def handles?(target)
    target.include? '*'
  end

  def process(document, target, attributes)
    lines = []
    matches = Dir.glob target
    matches.each do |match|
      resolved_path = document.normalize_system_path(match)
      lines.concat File.readlines(resolved_path)
      # append blank line so blocks don't get mashed together
      lines.push "\n"
    end
    lines
  end
end

Asciidoctor::Extensions.register_include_handler GlobIncludeHandler

# or...
# Asciidoctor::Extensions.register_handler :include, GlobIncludeHandler

Voila! Glob includes.

I'm fine with baking this functionality into Asciidoctor, though the extension still provides a way to override Asciidoctor's implementation in case someone has different requirements.

-Dan

Btw, I'm still looking for feedback on the include handler extension (and extensions in general) at https://github.com/asciidoctor/asciidoctor/pull/454


On Mon, Jul 1, 2013 at 12:08 AM, asotobu [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Hello,

in Asciidoctor we can include external files into one document by using the include:

include::basics.adoc[]

I think it could be a nice addition if we could use a glob expression instance of a concrete file so, for example we can do something like:

include::**/narrative.txt[]

Basically I am thinking about next use case:

Developers are using Thucydides (and JBehave) for BDD. In Thucydides you can add files called narrative.txt where you can describe using plain text a description of the scenario under test. Thucydides supports AsciiDoc format in narrative.txt files but they are rendered when you run Thucydides report tool. The problem is that maybe you want to have all these scenario files being a part of your project documentation. With current include approach, you will need to add one by one all narrative files inside a parent adoc file, which could be very tedious and hard to maintain. On the other hand if you could do something like

include::**/narrative.txt[]

Then every time you render your documentation, you can be assure that the last scenarios are inserted into rendered files.

What do you think? If you find useful I can create an issue on github and implement a possible solution.

Alex.


If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Include-with-Glob-expression-tp357.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



--
Reply | Threaded
Open this post in threaded view
|

Re: Include with Glob expression

mojavelinux
Administrator
In reply to this post by asotobu
Btw, this extension point can also be used to:

* include the contents from a URL
* include the rendered content of a Markdown file (by detecting the *.md extension, running the Markdown converter on the source and returning the result wrapped in a passthrough block)
* automatically syntax highlight source files (by detecting known source extensions like .java)

-Dan



On Mon, Jul 1, 2013 at 1:22 AM, Dan Allen <[hidden email]> wrote:
Alex,

I love this idea.

This feature will be unique to Asciidoctor since there's no way to extend macros like include::[] in AsciiDoc. But, it will be a much welcomed unique feature :)

One way to support this feature is to use the new extension point I've been working on for the include macro in 0.1.4. Here's how it might be implemented just to give you a taste of how the extension point works:

class GlobIncludeHandler
  def handles?(target)
    target.include? '*'
  end

  def process(document, target, attributes)
    lines = []
    matches = Dir.glob target
    matches.each do |match|
      resolved_path = document.normalize_system_path(match)
      lines.concat File.readlines(resolved_path)
      # append blank line so blocks don't get mashed together
      lines.push "\n"
    end
    lines
  end
end

Asciidoctor::Extensions.register_include_handler GlobIncludeHandler

# or...
# Asciidoctor::Extensions.register_handler :include, GlobIncludeHandler

Voila! Glob includes.

I'm fine with baking this functionality into Asciidoctor, though the extension still provides a way to override Asciidoctor's implementation in case someone has different requirements.

-Dan

Btw, I'm still looking for feedback on the include handler extension (and extensions in general) at https://github.com/asciidoctor/asciidoctor/pull/454


On Mon, Jul 1, 2013 at 12:08 AM, asotobu [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Hello,

in Asciidoctor we can include external files into one document by using the include:

include::basics.adoc[]

I think it could be a nice addition if we could use a glob expression instance of a concrete file so, for example we can do something like:

include::**/narrative.txt[]

Basically I am thinking about next use case:

Developers are using Thucydides (and JBehave) for BDD. In Thucydides you can add files called narrative.txt where you can describe using plain text a description of the scenario under test. Thucydides supports AsciiDoc format in narrative.txt files but they are rendered when you run Thucydides report tool. The problem is that maybe you want to have all these scenario files being a part of your project documentation. With current include approach, you will need to add one by one all narrative files inside a parent adoc file, which could be very tedious and hard to maintain. On the other hand if you could do something like

include::**/narrative.txt[]

Then every time you render your documentation, you can be assure that the last scenarios are inserted into rendered files.

What do you think? If you find useful I can create an issue on github and implement a possible solution.

Alex.


If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Include-with-Glob-expression-tp357.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



--



--
Reply | Threaded
Open this post in threaded view
|

Re: Include with Glob expression

glaforge
In reply to this post by mojavelinux
Perhaps a way to specify that we want a special ordering, like alphabetical ordering would be good?
Because it might be quite random otherwise.
Perhaps even alphabetical by default to have a consistent orderin.

Guillaume


On Mon, Jul 1, 2013 at 9:23 AM, mojavelinux [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Alex,

I love this idea.

This feature will be unique to Asciidoctor since there's no way to extend macros like include::[] in AsciiDoc. But, it will be a much welcomed unique feature :)

One way to support this feature is to use the new extension point I've been working on for the include macro in 0.1.4. Here's how it might be implemented just to give you a taste of how the extension point works:

class GlobIncludeHandler
  def handles?(target)
    target.include? '*'
  end

  def process(document, target, attributes)
    lines = []
    matches = Dir.glob target
    matches.each do |match|
      resolved_path = document.normalize_system_path(match)
      lines.concat File.readlines(resolved_path)
      # append blank line so blocks don't get mashed together
      lines.push "\n"
    end
    lines
  end
end

Asciidoctor::Extensions.register_include_handler GlobIncludeHandler

# or...
# Asciidoctor::Extensions.register_handler :include, GlobIncludeHandler

Voila! Glob includes.

I'm fine with baking this functionality into Asciidoctor, though the extension still provides a way to override Asciidoctor's implementation in case someone has different requirements.

-Dan

Btw, I'm still looking for feedback on the include handler extension (and extensions in general) at https://github.com/asciidoctor/asciidoctor/pull/454


On Mon, Jul 1, 2013 at 12:08 AM, asotobu [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Hello,

in Asciidoctor we can include external files into one document by using the include:

include::basics.adoc[]

I think it could be a nice addition if we could use a glob expression instance of a concrete file so, for example we can do something like:

include::**/narrative.txt[]

Basically I am thinking about next use case:

Developers are using Thucydides (and JBehave) for BDD. In Thucydides you can add files called narrative.txt where you can describe using plain text a description of the scenario under test. Thucydides supports AsciiDoc format in narrative.txt files but they are rendered when you run Thucydides report tool. The problem is that maybe you want to have all these scenario files being a part of your project documentation. With current include approach, you will need to add one by one all narrative files inside a parent adoc file, which could be very tedious and hard to maintain. On the other hand if you could do something like

include::**/narrative.txt[]

Then every time you render your documentation, you can be assure that the last scenarios are inserted into rendered files.

What do you think? If you find useful I can create an issue on github and implement a possible solution.

Alex.


If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Include-with-Glob-expression-tp357.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



--



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Include-with-Glob-expression-tp357p358.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



--
Guillaume Laforge
Groovy Project Manager
Pivotal, Inc.

Reply | Threaded
Open this post in threaded view
|

Re: Include with Glob expression

mojavelinux
Administrator
Guillaume,

Great point!

This is an excellent case where Asciidoctor would implement a certain convention (such as alphabetical), but an extension could be written to override this.

If tailoring the ordering becomes a very common need (and different uses call for different conventions), then we could introduce an order attribute to control it.

-Dan


On Mon, Jul 1, 2013 at 1:30 AM, glaforge [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Perhaps a way to specify that we want a special ordering, like alphabetical ordering would be good?
Because it might be quite random otherwise.
Perhaps even alphabetical by default to have a consistent orderin.

Guillaume


On Mon, Jul 1, 2013 at 9:23 AM, mojavelinux [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Alex,

I love this idea.

This feature will be unique to Asciidoctor since there's no way to extend macros like include::[] in AsciiDoc. But, it will be a much welcomed unique feature :)

One way to support this feature is to use the new extension point I've been working on for the include macro in 0.1.4. Here's how it might be implemented just to give you a taste of how the extension point works:

class GlobIncludeHandler
  def handles?(target)
    target.include? '*'
  end

  def process(document, target, attributes)
    lines = []
    matches = Dir.glob target
    matches.each do |match|
      resolved_path = document.normalize_system_path(match)
      lines.concat File.readlines(resolved_path)
      # append blank line so blocks don't get mashed together
      lines.push "\n"
    end
    lines
  end
end

Asciidoctor::Extensions.register_include_handler GlobIncludeHandler

# or...
# Asciidoctor::Extensions.register_handler :include, GlobIncludeHandler

Voila! Glob includes.

I'm fine with baking this functionality into Asciidoctor, though the extension still provides a way to override Asciidoctor's implementation in case someone has different requirements.

-Dan

Btw, I'm still looking for feedback on the include handler extension (and extensions in general) at https://github.com/asciidoctor/asciidoctor/pull/454


On Mon, Jul 1, 2013 at 12:08 AM, asotobu [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Hello,

in Asciidoctor we can include external files into one document by using the include:

include::basics.adoc[]

I think it could be a nice addition if we could use a glob expression instance of a concrete file so, for example we can do something like:

include::**/narrative.txt[]

Basically I am thinking about next use case:

Developers are using Thucydides (and JBehave) for BDD. In Thucydides you can add files called narrative.txt where you can describe using plain text a description of the scenario under test. Thucydides supports AsciiDoc format in narrative.txt files but they are rendered when you run Thucydides report tool. The problem is that maybe you want to have all these scenario files being a part of your project documentation. With current include approach, you will need to add one by one all narrative files inside a parent adoc file, which could be very tedious and hard to maintain. On the other hand if you could do something like

include::**/narrative.txt[]

Then every time you render your documentation, you can be assure that the last scenarios are inserted into rendered files.

What do you think? If you find useful I can create an issue on github and implement a possible solution.

Alex.


If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Include-with-Glob-expression-tp357.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Include-with-Glob-expression-tp357p358.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



--
Guillaume Laforge
Groovy Project Manager
Pivotal, Inc.




If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Include-with-Glob-expression-tp357p360.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



--
Reply | Threaded
Open this post in threaded view
|

Re: Include with Glob expression

asotobu
In reply to this post by mojavelinux
And now that you comment the extension point, one question related with asciidoctor-java-integration project, and because Guillaume is so interested too, maybe he could give us his opinion too:

do you think that extensions should be provided to asciidoctor-java-integration as Ruby class name, and then Ruby code of extension should be in Java classpath, so in fact Java users should implement the extensions in Ruby.

or we can write an extension in Java, and then from Asciidoctor Ruby code, Java code is executed. We could define a common interface with a method called process. Then the implementation class is also passed to register_handler method, and then in Ruby code we could check if we are in a JRuby environment and if the given class is an instance of common interface, and then only in that case the extension code is registered and executed when it is required.

Also I am thinking that it would be better offer both options, but I don't know the difficulties from the point of view of Asciidoctor project.

What do you think?

Reply | Threaded
Open this post in threaded view
|

Re: Include with Glob expression

mojavelinux
Administrator
I'm confident we can support extensions written in either Ruby or Java (or even Groovy, but let's solve the Java first, then build to Groovy).

Before I go on, I want to say that I consider the ability to extend either in Ruby or in Java (or Groovy) to be very important. (Extensions written in Java could even work in a pure Java implementation, so we'd be making this part of the AsciiDoc contract and not just Asciidoctor).

Ruby can invoke Java (when running in JRuby) just as easily as Java can invoke Ruby (in fact, it's even simpler).

Here's a snippet that I used in a Slim template to prove that I can execute Java code from the Ruby runtime.

- require 'java'
- java_import @document.attr('handler')
=Kernel.const_get(@document.attr('handler').split('.').last).new

- require 'java'
- java_import @document.attr('attributes-class')
/ lookup unqualified class name in the global symbol table and create a new instance
- instance = Kernel.const_get(@document.attr('attributes-class').split('.').last).new('key=value')
=instance
',
=instance.map['key']

When invoked with this attribute declaration:

-a attributes-class=org.asciidoctor.Attributes

Outputs:

org.asciidoctor.Attributes@4d6bb0d1, value

The key goal here is make the language in which the class is written transparent to both Ruby and Java. In other words, they shouldn't know how the class is implemented, only that there is an API that can be invoked. I think that passing handler classes as strings (and maybe instances) is the best way (at least to start).

The Asciidoctor Java integration will only need to add the require 'java' and java_import lines somewhere in the integration code if a Java class is being passed into the processor to make it available in the global symbol table.

When implementing the Ruby code, I just need to remember to be careful not to assume anything about the handler such that it would break the ability to use a Java class under the covers.

Invoking Java from Ruby running on JRuby is covered in this wiki page: https://github.com/jruby/jruby/wiki/CallingJavaFromJRuby

We don't need to use every possible way to execute Java from Ruby, just one that works :)

Once I merge in the include handler pull request, we can start giving this a try in the continuous integration build.

-Dan


On Mon, Jul 1, 2013 at 3:05 AM, asotobu [via Asciidoctor :: Discussion] <[hidden email]> wrote:
And now that you comment the extension point, one question related with asciidoctor-java-integration project, and because Guillaume is so interested too, maybe he could give us his opinion too:

do you think that extensions should be provided to asciidoctor-java-integration as Ruby class name, and then Ruby code of extension should be in Java classpath, so in fact Java users should implement the extensions in Ruby.

or we can write an extension in Java, and then from Asciidoctor Ruby code, Java code is executed. We could define a common interface with a method called process. Then the implementation class is also passed to register_handler method, and then in Ruby code we could check if we are in a JRuby environment and if the given class is an instance of common interface, and then only in that case the extension code is registered and executed when it is required.

Also I am thinking that it would be better offer both options, but I don't know the difficulties from the point of view of Asciidoctor project.

What do you think?




If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Include-with-Glob-expression-tp357p365.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



--
Reply | Threaded
Open this post in threaded view
|

Re: Include with Glob expression

asotobu
Hi Dan,

yes I have already read the post of calling Java from Ruby when I started the AJI project and there would be no problem to  execute Java inside Ruby, for this reason I pointed out both options in previous post. Also I think that if we can implement extensions in Java we are going to have a really good integration with tools like http://ditaa.sourceforge.net/

So no worries, as soon as possible, I will implement the extensions in AJI so we can start playing around it.

Thank you so much for given your opinion about this new feature which I think there will be a before and an after of Asciidoctor life when we release this new feature.

Alex.
Reply | Threaded
Open this post in threaded view
|

Re: Include with Glob expression

mojavelinux
Administrator

On Mon, Jul 1, 2013 at 11:38 PM, asotobu [via Asciidoctor :: Discussion] <[hidden email]> wrote:
I think there will be a before and an after of Asciidoctor life when we release this new feature.

Amen to that!