Ever since I started working on Asciidoctor, I've been eagerly awaiting the chance to work on the extensions API. That time has come. Extensions are the last major blocker for the 0.1.4 release.

I've been looking forward to this enhancement because extensions in AsciiDoc (technically called filters) are very powerful, but also very low-level. What's worse, they rely on system commands to do anything significant.

The goal for Asciidoctor has always been to allow extensions to be written using the full power of a programming language (whether it be Ruby, Java, Groovy or JavaScript), similar to what we've done with backend (rendering) mechanism. That way, you don't have to do yak shaving to get the functionality you want and you can distribute the extension using defacto-standard packaging mechanisms (like RubyGems or JARs).

I've pushed a draft of the API as a pull request for issue #79 (https://github.com/asciidoctor/asciidoctor/issues/79), one of several issues related to extensions.

WARNING: The extension API should be considered experimental and subject to change, even after it's integrated into the 0.1.4 release. It's likely going to take several iterations to get right and we don't want to lock ourselves into an API too early. Early adopters will help us identify and close the gaps and smooth out any awkwardness.

TIP: For those of you on the JVM, yes, you can write extensions in Java. I've prototyped it and it works. We'll need to sort out a few technical challenges and documentation to make it completely smooth, but we'll get there. Part of that is creating abstract extension classes in the Java integration that mirror the classes in the Ruby API...though if you're starting from Groovy that may not even been necessary.

Conceptually, the extensions themselves are rather simple. The hard part came in figuring out how they would be registered in a non-static way and how to weave them into the processor. After studying many other libraries (most notably Middleman, Python Markdown and Kramdown), I think I finally sorted out an elegant approach.

Here's a list of the extension points we have so far.

Preprocessor:: Processes the raw source lines before they are passed to the parser
Treeprocessor:: Processes the Document (AST) once parsing is complete
Postprocessor:: Processes the output after the Document has been rendered, before it's gets written to file
Block processor:: Processes a block of content marked with a custom style (i.e., name) (equivalent to filters in AsciiDoc)
Block macro processor:: Registers a custom block macro and process it (e.g., gist::12345[])
Inline macro processor:: Registers a custom inline macro and process it (e.g., btn:[Save])
Include handler:: - Processes the include::<filename>[] macro

NOTE: The include handler extension is in a separate pull request and will use the same extension mechanism.

These extensions are registered per Document using a callback that feels sort of like a DSL:

```ruby

Asciidoctor::Extensions.register do |document|
  preprocessor SamplePreprocessor
  treeprocessor SampleTreeprocessor
  postprocessor SamplePostprocessor
  block :sample, SampleBlock
  block_macro :sample, SampleBlockMacro
  inline_macro :sampe, SampleInlineMacro
end
```

Each class registered is instantiated when the Document is created. You can register more than one processor for each type, though you can only have one processor per custom block or macro.

I also plan to have an extension point that can process one of the built-in blocks, such as a normal paragraph. I just haven't sorted out the best way to integrate it yet.

For now, you need to use the Asciidoctor API (not the CLI) in order to register the extensions and invoke Asciidoctor. Eventually, we'll be able to load extensions packaged in a RubyGem (Ruby) or JAR (Java) by scanning the LOAD_PATH (Ruby) or classpath (Java), respectively. We may also ship some built-in extensions that can be enabled using an attribute named `extensions`.

Here are some examples of extensions and how they are registered.

== Preprocessor example

Remove front-matter required by site generators like Jekyll.

```ruby
require 'asciidoctor/extensions'
# Sample document:
#
# ---
# tags: [announcement, website]
# ---
# = Document Title
#
# content
#
class FrontMatterPreprocessor < Asciidoctor::Extensions::Preprocessor
  def process lines
    return lines if lines.empty?
    front_matter = []
    if lines.first.chomp == '---'
      original_lines = lines.dup
      lines.shift
      while !lines.empty? && lines.first.chomp != '---'
        front_matter << lines.shift
      end

      if (first = lines.first).nil? || first.chomp != '---'
        lines = original_lines
      else
        lines.shift
        @document.attributes['front-matter'] = front_matter.join.chomp
      end
    end
    lines
  end
end

Asciidoctor::Extensions.register do |document|
  preprocessor FrontMatterPreprocessor
end

Asciidoctor.render_file('sample-with-front-matter.ad', :safe => :safe, :in_place => true)
```

TIP: The processor can be a Java class (see below)

== Block processor example

Register a custom block style that uppercases all the words and converts periods to exclamation points.

```ruby
# Sample:
#
# [yell]
# The time is now. Get a move on.
#
class YellBlock < Asciidoctor::Extensions::BlockProcessor
  option :contexts, [:paragraph]
  option :content_model, :simple
 
  def process parent, reader, attributes
    lines = reader.lines.map {|line|
      line = line.upcase
      line = line.gsub('.', '!')
      line
    }
    Asciidoctor::Block.new(parent, :paragraph, :source => lines, :attributes => attributes)
  end
end

Asciidoctor::Extensions.register do |document|
  block :yell, YelloBlock
end

Asciidoctor.render_file('sample-with-yell-block.ad', :safe => :safe, :in_place => true)
```

== Block macro processor example

Add a macro for embedding a gist.

```ruby
# Sample:
#
# gist::123456[]
#
class GistMacro < Asciidoctor::Extensions::BlockMacroProcessor
  def process parent, target, attributes
    if @document.attr? 'basebackend', 'html'
      Block.new(parent, :pass, :content_model => :raw,
          :source => %(<div class="gistblock">
<div class="title">#{attributes['title']}</div>
<div class="content">
<script src="<a href="https://gist.github.com/#{target}.js">https://gist.github.com/#{target}.js"></script>
</div>
</div>))
    else
      # TODO Download gist and convert it to a listing
      # For now, just create as a link (subject to normal paragraph substitutions)
      Asciidoctor::Block.new(parent, :paragraph, :source => %(<a href="https://gist.github.com/#{target}">https://gist.github.com/#{target}"),
          :attributes => attributes)
    end
  end
end

Asciidoctor::Extensions.register do |document|
  block :gist, GistBlock
end

Asciidoctor.render_file('sample-with-gist.ad', :safe => :safe, :in_place => true)
```

== Java-based preprocessor example

.Extension class from Asciidoctor Java (planned)
```java
import java.util.List;

public abstract class Preprocessor {
  private Object document;

  public Preprocessor(Object document) {
    this.document = document;
  }

  public abstract List<String> process(List<String> lines);
}
```

```java
import java.util.List;

public class JavaBasedPreprocessor extends Preprocessor {
  public JavaBasedPreprocessor(Object document) {
    super(document);
  }

  public List<String> process(List<String> lines) {
    System.out.println("process method called on " + getClass().getName() + " instance with lines:");
    System.out.println(lines);
    // remove first line for fun
    lines.remove(0);
    return lines;
  }

}
```
 javac Preprocessor.java JavaBasedPreprocessor.java

```ruby
$using_jruby = RUBY_ENGINE == 'jruby'
if $using_jruby
  require 'java'
  java_import 'JavaBasedPreprocessor'
end

Asciidoctor::Extensions.register do |document|
  preprocessor JavaBasedPreprocessor if $using_jruby
end

Asciidoctor.render_file('sample.ad', :safe => :safe, :in_place => true
```

We need your feedback! Please post your reply here or comment on issue #79.

--

Dan Allen | http://google.com/profiles/dan.j.allen

---

If you reply to this email, your message will be added to the discussion below:

http://discuss.asciidoctor.org/Finally-a-draft-of-the-Asciidoctor-extension-API-has-landed-tp455.html

To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML