Asciidoc and dependency management

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Asciidoc and dependency management

Michael_M
I'm trying to figure out the best way to include documentation from other (independent) modules. I'm hoping that someone has done this before and can share some pointers.

Let's assume that my project P depends on modules A and B. There are 3 Git repositories: P, A and B each reside in their own Git repository containing the source code and the documentation.

Just as in the code, the dependencies  are specific to a particular version.
So I want all include:: directives that point to A, to point to a specific git branch of A.

There are 2 approaches I am considering:
1. Use attributes in the asciidoc document in P, and point to the specific version.
E.g:
:repo_a: https://raw.githubusercontent.com/username/project_A/v3.5

My concern with this approach: what happens with the includes directives in A? And images?

Approach 2: Use Gradle to handle the dependencies. I assume that Gradle would have to download the correct branch of A and B. I'm not sure how this could be setup with directory structures.

Any thoughts?
Reply | Threaded
Open this post in threaded view
|

Re: Asciidoc and dependency management

nawroth
Hi Michael!

I have every module provide a [modulename]-[version]-docs.jar, just
like normal jars (or sources or test-sources or javadoc jars common in
Java land).
The main documentation build unpacks those jars, and then all of the
documentation is built in one go.

It's quite easy to do this using Maven. For extra convience and some
additional magic, I built a Maven plugin for creating the docs jars.
I wouldn't be surprised if it's even easier using Gradle (we're
considering switching).

One nice feature of using dependency management is that modules can be
moved around without breaking anything.

Here's an outdated writeup of what we're doing:
http://neo4j.com/docs/2.2.5/community-docs.html

The tricky part over time, as the number of modules grow, is to find
the best location for all parts of the documentation.

Feel free to ask any questions you have about our setup!

/anders

2015-09-23 19:24 GMT+02:00 Michael_M [via Asciidoctor :: Discussion]
<[hidden email]>:

> I'm trying to figure out the best way to include documentation from other
> (independent) modules. I'm hoping that someone has done this before and can
> share some pointers.
>
> Let's assume that my project P depends on modules A and B. There are 3 Git
> repositories: P, A and B each reside in their own Git repository containing
> the source code and the documentation.
>
> Just as in the code, the dependencies  are specific to a particular version.
> So I want all include:: directives that point to A, to point to a specific
> git branch of A.
>
> There are 2 approaches I am considering:
> 1. Use attributes in the asciidoc document in P, and point to the specific
> version.
> E.g:
> :repo_a: https://raw.githubusercontent.com/username/project_A/v3.5
>
> My concern with this approach: what happens with the includes directives in
> A? And images?
>
> Approach 2: Use Gradle to handle the dependencies. I assume that Gradle
> would have to download the correct branch of A and B. I'm not sure how this
> could be setup with directory structures.
>
> Any thoughts?
>
> ________________________________
> If you reply to this email, your message will be added to the discussion
> below:
> http://discuss.asciidoctor.org/Asciidoc-and-dependency-management-tp3751.html
> To start a new topic under Asciidoctor :: Discussion, email
> [hidden email]
> To unsubscribe from Asciidoctor :: Discussion, click here.
> NAML
Reply | Threaded
Open this post in threaded view
|

Re: Asciidoc and dependency management

mojavelinux
Administrator
Great insights Anders!

There's no question that tackling this challenge requires more experimentation. And it's still a problem we need to solve to get the documentation for the Asciidoctor family of projects streamlined on asciidoctor.org (right now we're manually copying README files, which is about as far from ideal as you can get). (see https://github.com/asciidoctor/asciidoctor.org/issues/366)

Using a dependency management system is certainly an excellent way to specify and manage the relationships. You build on top of a mature model that already handles concerns like artifact resolution, versioning and resource loading.

For me, the (only) drawback of the setup Anders outlines is that it relies heavily on the build, making it a more formal process. In an ideal world, it should be possible to preview the documents with or without the build (casually)...but that requires a lot more thought and planning to setup (and some AsciiDoc tricks). 

Gradle's continuous build mode does have the potential to address this drawback because you can start the build (much like a server) and just keep it running. As a result, you see changes instantaneously in the output files when you make a change to a source file. But that setup may still break the ability to use an editor like Atom or Brackets to preview the documents in their original form. So I'll finish with...

More research is needed in this area :)

-Dan

On Fri, Sep 25, 2015 at 6:24 PM, nawroth [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Hi Michael!

I have every module provide a [modulename]-[version]-docs.jar, just
like normal jars (or sources or test-sources or javadoc jars common in
Java land).
The main documentation build unpacks those jars, and then all of the
documentation is built in one go.

It's quite easy to do this using Maven. For extra convience and some
additional magic, I built a Maven plugin for creating the docs jars.
I wouldn't be surprised if it's even easier using Gradle (we're
considering switching).

One nice feature of using dependency management is that modules can be
moved around without breaking anything.

Here's an outdated writeup of what we're doing:
http://neo4j.com/docs/2.2.5/community-docs.html

The tricky part over time, as the number of modules grow, is to find
the best location for all parts of the documentation.

Feel free to ask any questions you have about our setup!

/anders

2015-09-23 19:24 GMT+02:00 Michael_M [via Asciidoctor :: Discussion]
<[hidden email]>:

> I'm trying to figure out the best way to include documentation from other
> (independent) modules. I'm hoping that someone has done this before and can
> share some pointers.
>
> Let's assume that my project P depends on modules A and B. There are 3 Git
> repositories: P, A and B each reside in their own Git repository containing
> the source code and the documentation.
>
> Just as in the code, the dependencies  are specific to a particular version.
> So I want all include:: directives that point to A, to point to a specific
> git branch of A.
>
> There are 2 approaches I am considering:
> 1. Use attributes in the asciidoc document in P, and point to the specific
> version.
> E.g:
> :repo_a: https://raw.githubusercontent.com/username/project_A/v3.5
>
> My concern with this approach: what happens with the includes directives in
> A? And images?
>
> Approach 2: Use Gradle to handle the dependencies. I assume that Gradle
> would have to download the correct branch of A and B. I'm not sure how this
> could be setup with directory structures.
>
> Any thoughts?
>
> ________________________________
> If you reply to this email, your message will be added to the discussion
> below:
> http://discuss.asciidoctor.org/Asciidoc-and-dependency-management-tp3751.html
> To start a new topic under Asciidoctor :: Discussion, email
> [hidden email]
> To unsubscribe from Asciidoctor :: Discussion, click here.
> NAML



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Asciidoc-and-dependency-management-tp3751p3755.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



--
Dan Allen | @mojavelinux | http://google.com/profiles/dan.j.allen
Reply | Threaded
Open this post in threaded view
|

Re: Asciidoc and dependency management

mojavelinux
Administrator
In reply to this post by nawroth
Following up on Ander's comments, I will say this. Gradle is *much* better suited for this type of task than Maven. Documentation compilation and publishing is inherently complex. Maven is designed to handle the "happy path", but content is rarely happy. Gradle has the muscle to move files around in arbitrary (read as: flexible) ways, which makes it perfectly suited to solve this type of task. There's no question in my mind that Gradle is the right choice for aggregating and publishing content from multiple sources, repositories and/or channels.

-Dan

On Sat, Sep 26, 2015 at 2:49 PM, Dan Allen <[hidden email]> wrote:
Great insights Anders!

There's no question that tackling this challenge requires more experimentation. And it's still a problem we need to solve to get the documentation for the Asciidoctor family of projects streamlined on asciidoctor.org (right now we're manually copying README files, which is about as far from ideal as you can get). (see https://github.com/asciidoctor/asciidoctor.org/issues/366)

Using a dependency management system is certainly an excellent way to specify and manage the relationships. You build on top of a mature model that already handles concerns like artifact resolution, versioning and resource loading.

For me, the (only) drawback of the setup Anders outlines is that it relies heavily on the build, making it a more formal process. In an ideal world, it should be possible to preview the documents with or without the build (casually)...but that requires a lot more thought and planning to setup (and some AsciiDoc tricks). 

Gradle's continuous build mode does have the potential to address this drawback because you can start the build (much like a server) and just keep it running. As a result, you see changes instantaneously in the output files when you make a change to a source file. But that setup may still break the ability to use an editor like Atom or Brackets to preview the documents in their original form. So I'll finish with...

More research is needed in this area :)

-Dan

On Fri, Sep 25, 2015 at 6:24 PM, nawroth [via Asciidoctor :: Discussion] <[hidden email]> wrote:
Hi Michael!

I have every module provide a [modulename]-[version]-docs.jar, just
like normal jars (or sources or test-sources or javadoc jars common in
Java land).
The main documentation build unpacks those jars, and then all of the
documentation is built in one go.

It's quite easy to do this using Maven. For extra convience and some
additional magic, I built a Maven plugin for creating the docs jars.
I wouldn't be surprised if it's even easier using Gradle (we're
considering switching).

One nice feature of using dependency management is that modules can be
moved around without breaking anything.

Here's an outdated writeup of what we're doing:
http://neo4j.com/docs/2.2.5/community-docs.html

The tricky part over time, as the number of modules grow, is to find
the best location for all parts of the documentation.

Feel free to ask any questions you have about our setup!

/anders

2015-09-23 19:24 GMT+02:00 Michael_M [via Asciidoctor :: Discussion]
<[hidden email]>:

> I'm trying to figure out the best way to include documentation from other
> (independent) modules. I'm hoping that someone has done this before and can
> share some pointers.
>
> Let's assume that my project P depends on modules A and B. There are 3 Git
> repositories: P, A and B each reside in their own Git repository containing
> the source code and the documentation.
>
> Just as in the code, the dependencies  are specific to a particular version.
> So I want all include:: directives that point to A, to point to a specific
> git branch of A.
>
> There are 2 approaches I am considering:
> 1. Use attributes in the asciidoc document in P, and point to the specific
> version.
> E.g:
> :repo_a: https://raw.githubusercontent.com/username/project_A/v3.5
>
> My concern with this approach: what happens with the includes directives in
> A? And images?
>
> Approach 2: Use Gradle to handle the dependencies. I assume that Gradle
> would have to download the correct branch of A and B. I'm not sure how this
> could be setup with directory structures.
>
> Any thoughts?
>
> ________________________________
> If you reply to this email, your message will be added to the discussion
> below:
> http://discuss.asciidoctor.org/Asciidoc-and-dependency-management-tp3751.html
> To start a new topic under Asciidoctor :: Discussion, email
> [hidden email]
> To unsubscribe from Asciidoctor :: Discussion, click here.
> NAML



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/Asciidoc-and-dependency-management-tp3751p3755.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



--
Dan Allen | @mojavelinux | http://google.com/profiles/dan.j.allen



--
Dan Allen | @mojavelinux | http://google.com/profiles/dan.j.allen
Reply | Threaded
Open this post in threaded view
|

Re: Asciidoc and dependency management

mojavelinux
Administrator
In reply to this post by nawroth

On Sat, Sep 26, 2015 at 2:52 PM, Dan Allen <[hidden email]> wrote:
There's no question in my mind that Gradle is the right choice for aggregating and publishing content from multiple sources, repositories and/or channels.

...when compared to Maven. There are other suitable tools that are very suitable, including Rake and Gulp.

-Dan


--
Dan Allen | @mojavelinux | http://google.com/profiles/dan.j.allen
Reply | Threaded
Open this post in threaded view
|

Re: Asciidoc and dependency management

Michael_M
In reply to this post by nawroth
Hi Anders,

Thanks for your suggestions. I like your procedure of publishing a jar file, as that provides a clean separation of responsibilities across projects.
I also agree with Dan, regarding preview while authoring. So I'm playing with this thought:
- The sub-projects deliver the jars as Anders describes
- An author periodically runs a build command which extracts the dependencies into a .gitignore directory in his local Git repository.

In that way, the author does have a "live preview" of the documents.

More research and experimentation needed indeed. I would really like this to result into a standard (default) project and directory layout for AsciiDoc documents. This would greatly benefit the perceived maturity of AsciiDoc and the toolchain.
Reply | Threaded
Open this post in threaded view
|

Re: Asciidoc and dependency management

mojavelinux
Administrator

On Sun, Sep 27, 2015 at 3:24 AM, Michael_M [via Asciidoctor :: Discussion] <[hidden email]> wrote:
More research and experimentation needed indeed. I would really like this to result into a standard (default) project and directory layout for AsciiDoc documents. This would greatly benefit the perceived maturity of AsciiDoc and the toolchain.

+1

-Dan


--
Dan Allen | @mojavelinux | http://google.com/profiles/dan.j.allen
Reply | Threaded
Open this post in threaded view
|

Re: Asciidoc and dependency management

rockyallen
I know this is an old thread, but I just want to give my experience.

We assemble a single large document for each customer from many modules. Each customer can have different modules and different versions of those modules.
We tried the each-module-publishes-a-jar approach, but the editing cycle was too painful. Modules are quite often developed in parallel with the overall document, and you need to edit them seamlessly.

Our solution is to hold the modules in SVN, and create SVN dependencies from the master document project onto the modules it needs. On checking out the master document project, SVN then checks out all the necessary modules into a single folder where the master document can see them, eg:

== Maintenance
include::{moduledir}/engine123/master.adoc[tag=maintenance]
include::{moduledir}/gearbox456/master.adoc[tag=maintenance]

Initially the dependencies are to trunk. When the document is stable, you can change them to tags. And export rather than checkout if you don't want to risk inadvertent changes to the modules.

Note the single master.adoc for each module. I find this makes WYSIWYG editing much easier.
The file has a standard set of tags for "description", "maintenance", "spares" etc. so that the overall master is just a list of include::[tag=], and hardly ever has to be edited.

Good:
- Works well with our Jenkins build server
- Keeps dependencies in 1 place
- Single button build once the dependencies are defined
- Publish a manual for each module with no extra work

Bad:
Very SVN specific
Maintaining the dependencies using Tortoise SVN is ugly.

Ideally the dependencies would be declared in a tool-agnostic way and kept in the build script (Ant in our case) or in the master document. Antora maybe?

This works for us, but any suggestions for stream-lining it would be good.