Loading... |
Reply to author |
Edit post |
Move post |
Delete this post |
Delete this post and replies |
Change post date |
Print post |
Permalink |
Raw mail |
Hi list,
I'm trying to use the extension preprocessor to transform the lines in an asciidoc file prior to processing. Although I have something working, I find 'include::' macros are not getting processed, and am not sure how to proceed - I'm perhaps not understanding the rendering process correctly. I'm currently using the 'process' method in a subclass of preprocessor, and changing or inserting lines in the 'lines' array, as appropriate. I then return a new reader with these new lines. def process reader, lines return reader if lines.empty? # change and insert new rows in 'lines' return Asciidoctor::Reader.new lines end This works well with a self-contained asciidoc file, but if the file contains 'include::' macros, these macros are not processed, and their text is simply inserted in the output html without rendering the document to insert. I have tried 'image::' macros, and these are processed and the image added correctly. Is the preprocessor an appropriate way to do a source-to-source transform? I was hoping to slot my rewrite of the asciidoc document in as a first stage, aiming to return a set of lines in correct asciidoc syntax for later rendering. The code I'm working on is here. thanks for any help, Peter. |
Loading... |
Reply to author |
Edit post |
Move post |
Delete this post |
Delete this post and replies |
Change post date |
Print post |
Permalink |
Raw mail |
Preprocessors are executed before starting to render the document, so include macro is not executed, it is executed during render time, for this reason the include content is there during preprocessor. In this case what you is a TreeProcessor or a PostProcessor I guess.
|
Loading... |
Reply to author |
Edit post |
Move post |
Delete this post |
Delete this post and replies |
Change post date |
Print post |
Permalink |
Raw mail |
Thanks for your reply. But the problem is that the 'include' macro is not being executed during render time - I am not processing it during my preprocessing. Below is a screenshot of the output: You can see the include line included verbatim three lines down from the image. Without my preprocessing step, the contents of the included file appear at that point. I believe I have inadvertently 'turned off' processing of the include macro by my use of the preprocessor, and I'm wondering how to correct that mistake. If this is not possible, I may have to explore the other methods, but my transformation needs to occur before any rendering. cheers, Peter. |
Loading... |
Reply to author |
Edit post |
Move post |
Delete this post |
Delete this post and replies |
Change post date |
Print post |
Permalink |
Raw mail |
Administrator
|
Actually, it is possible to read the lines so that the includes get processed. Stay tuned for an example. I'm preparing one. -Dan On Oct 27, 2013 7:09 AM, "plane [via Asciidoctor :: Discussion]" <[hidden email]> wrote:
Thanks for your reply. But the problem is that the 'include' macro is not being executed during render time - I am not processing it during my preprocessing. ... [show rest of quote] |
Loading... |
Reply to author |
Edit post |
Move post |
Delete this post |
Delete this post and replies |
Change post date |
Print post |
Permalink |
Raw mail |
Cool I am expecting it too to learn more about extensions.
The passthrough example works perfectly. I have created an Extension that it is auto-installed and let users use arrows-and-boxes project inside AsciiDoc documents El diumenge 27 d’octubre de 2013, mojavelinux [via Asciidoctor :: Discussion] ha escrit:
... [show rest of quote] -- Enviat amb Gmail Mobile |
Loading... |
Reply to author |
Edit post |
Move post |
Delete this post |
Delete this post and replies |
Change post date |
Print post |
Permalink |
Raw mail |
Administrator
|
In reply to this post by plane
Welcome Peter! We're glad to have you! I'm thrilled to hear that you're working on an integration between Asciidoctor and asciidoc-bib. I've been looking forward to these two projects coming together. I was hopeful the extensions API would open that door, and it seems it has.The preprocessor is the trickiest extension point to understand at first because it has the ability to affect the reader's cursor. Nothing some documentation can't solve :) So let's get to it! You have two options when reading the lines in a preprocessor. You can either work with the raw lines of the main source document, or you can instruct the reader to walk the lines. The second argument to the process method is the raw lines (original source) of the main document. Working with these lines is the most benign since it doesn't affect the reader's cursor. However, as you observed, preprocessor directives in those lines are not processed. When you read the lines from the reader, any preprocessor directives, such as the include directive, will get processed. It will also advance the cursor, effectively discarding those lines. As such, you need to restore the lines by pushing them back onto the reader. You can so using the Reader#push_include method. If the process method returns nil, Asciidoctor will continue processing using the reader in its current state. If you return a reader, it will replace the reader Asciidoctor uses. Here's an example that shows the difference between the raw lines and lines read from the reader. [source,ruby] -- class SamplePreprocessor < Asciidoctor::Extensions::Preprocessor def process reader, raw_lines puts '.Raw lines' puts '....' puts raw_lines.join # <1> puts '....' puts "\n" lines = [] while reader.has_more_lines? lines << reader.read_line # <2> end puts '.Preprocessed lines' puts '....' puts lines.join # <3> puts '....' use_push_include = true if use_push_include reader.push_include lines, '<stdin>', '<stdin>' # <4> nil # <5> else Asciidoctor::Reader.new lines # <6> end end end -- <1> Prints the original source of the main document <2> Consume each line from the reader. The line read is the next line once preprocessor directives on the original line have been evaluated. <3> Prints the effective lines once all preprocessor directives have been evaluated. <4> Push the lines back onto the reader so that they can be interpreted by the AsciiDoc processor. The second and third arguments (file and path, respectively) are required at the moment to work around a bug. <5> A nil return instructs Asciidoctor to continue with the original Reader
<6> Replace the original Reader with a new one that contains the preprocessed lines The only major caveat right now with reading the lines from the Reader is that it messes up the line information for reporting warnings. When you push the preprocessed lines back onto the Reader, Asciidoctor thinks they were all in the original file. I'm working on a way to retain this information in the next version. If you are only reading information from the processed lines, and not modifying them, it's possible to push the raw lines back onto the reader, thus restoring its original state. [source,ruby] -- def process reader, raw_lines lines = reader.read # <1> cursor = reader.cursor reader.push_include raw_lines, (cursor.file || '<stdin>'), (cursor.path || '<stdin>'), 1 # <2> reader.process_lines = true if cursor.file.nil? # <3> nil end -- <1> Preprocess all lines and return as an Array <2> Restore the original lines onto the Reader <3> Work around an assumption that lines should not be processed if the file name is nil I hope that clears some things up. Obviously, the Preprocessor extension needs some polishing, but that's what this discussion is helping to identify. Cheers, -Dan On Sun, Oct 27, 2013 at 6:06 AM, plane [via Asciidoctor :: Discussion] <[hidden email]> wrote: Hi list, ... [show rest of quote] -- Dan Allen | http://google.com/profiles/dan.j.allen |
Loading... |
Reply to author |
Edit post |
Move post |
Delete this post |
Delete this post and replies |
Change post date |
Print post |
Permalink |
Raw mail |
Administrator
|
In reply to this post by plane
[source,ruby] You could assign the file name to a document attribute. That way, you could allow it to be overridden. For example: [source,ruby] -- Asciidoctor::Extensions.register do |document| unless document.attr? 'bibfile' document.set_attribute 'bibfile', 'biblio.bib' # <1> end preprocessor AsciidocBibExtension end -- <1> Set the bibfile attribute on the document. The document may still override this value, unless the attribute was assigned via the CLI or API -Dan On Sun, Oct 27, 2013 at 3:53 PM, Dan Allen <[hidden email]> wrote:
... [show rest of quote] -- Dan Allen | http://google.com/profiles/dan.j.allen |
Loading... |
Reply to author |
Edit post |
Move post |
Delete this post |
Delete this post and replies |
Change post date |
Print post |
Permalink |
Raw mail |
Administrator
|
In reply to this post by plane
# TODO: Manage other asciidoctor options? You may want to delegate to the CLI invoker, essentially creating a cli wrapper: [source,ruby] -- require 'asciidoctor' require 'asciidoc-bib' # includes and registers Asciidoctor extension require 'asciidoctor/cli/options' require 'asciidoctor/cli/invoker' invoker = Asciidoctor::Cli::Invoker.new(ARGV) invoker.invoke! exit invoker.code -- There's an issue [1] to add an argument to the commandline to require additional libraries, such as: $ asciidoctor -r asciidoc-bib sample.adoc If you have other ideas, please feel free to suggest them in that issue. [1] https://github.com/asciidoctor/asciidoctor/issues/574 On Sun, Oct 27, 2013 at 4:03 PM, Dan Allen <[hidden email]> wrote:
... [show rest of quote] -- Dan Allen | http://google.com/profiles/dan.j.allen |
Loading... |
Reply to author |
Edit post |
Move post |
Delete this post |
Delete this post and replies |
Change post date |
Print post |
Permalink |
Raw mail |
Administrator
|
In reply to this post by plane
Peter, If you are willing to modify the syntax of your macro, you could use the inline macro extension point to handle the references: cite:Lane12[] becomes Lane 2012 citenp:Lane12[] becomes Lane (2012) cite:Lane12,59[See] becomes (See Lane 2012, 59) or cite:[Lane12]
citenp:[Lane12] cite:[See:Lane12,59] The idea is to keep in the spirit of the <name>:<target>[<attributes>] convention of AsciiDoc macros. You may need to combine the InlineMacroProcessor with a Treeprocessor, using the tree processor to build a reference table. I'd have to understand better how you do the replacements to know if you need that additional step. Another alternative is to perform all the work in a Treeprocessor. The benefit of the Treeprocessor is that you can work on the chunked content, taking care to avoid any literal blocks. You can find a NodeVisitor I created for this purpose in the following post: http://discuss.asciidoctor.org/Walking-the-Asciidoctor-document-tree-td479.html Cheers, On Sun, Oct 27, 2013 at 4:10 PM, Dan Allen <[hidden email]> wrote:
... [show rest of quote] -- Dan Allen | http://google.com/profiles/dan.j.allen |
Loading... |
Reply to author |
Edit post |
Move post |
Delete this post |
Delete this post and replies |
Change post date |
Print post |
Permalink |
Raw mail |
Administrator
|
In reply to this post by asotobu
On Sun, Oct 27, 2013 at 2:55 PM, asotobu [via Asciidoctor :: Discussion] <[hidden email]> wrote:
That's great news! That's going to open a lot of possibilities for projects using Asciidoctor! You're going to be a step ahead of the Ruby implementation because we've yet to sort out how to auto-register extensions from the LOAD_PATH :) Btw, the Asciidoctor build now uses its own extension (to fix a bug in Yardoc). Check it out! https://github.com/asciidoctor/asciidoctor/blob/master/Rakefile#L67-L74 |
Loading... |
Reply to author |
Edit post |
Move post |
Delete this post |
Delete this post and replies |
Change post date |
Print post |
Permalink |
Raw mail |
In reply to this post by mojavelinux
Thanks Dan for this explanation. The loop to read the lines using 'reader.read_line' seems to have done the trick.
Thanks also for the other suggestions. When I was reading about your extension mechanisms, I did think macro syntax would have been a better way to go, but I'll try to preserve my existing syntax first. Perhaps something to add in a future version? When I have my extension finished, I'll announce it here. cheers, Peter. |
Loading... |
Reply to author |
Edit post |
Move post |
Delete this post |
Delete this post and replies |
Change post date |
Print post |
Permalink |
Raw mail |
In reply to this post by mojavelinux
Amazing, hahahaha I like the Rakefile you showed me :)
|
Free forum by Nabble | Edit this page |