Generating static REST API docs with Asciidoctor?

Those here who are in software / IT will understand this topic, others may want to look away ;-)

There is a very common requirement these days to document REST APIs in a 'nice way'. There are two 'nice ways' that I know of:

* 'active' JS-enabled page like this example, done with API blueprint, nice for programmers to work with current version of API
* static doc, i.e. a normal 'specification', like you see here, apparently done with Swagger2Markup.

In both kinds we want to be able to put manually authored text, and other static content. In the static specification flavour, we also want typical heading structure (front-page, preface, amendment history, references, logos etc), as we have on our specs (example).

The next question is where is the source content? As far as I can work out there are various choices:

* API blueprint (spec), which is a mixture of Github flavour Markdown and embedded chunks of various kinds of JSON / JSON schema. This format seems to be suitable for included static text in along with the REST API sections. Not sure how much free content can go in there yet.
* Swagger (=OpenAPI 2.0). Format seems to be JSON or YAML. Doesn't look very good for including manual documentation.
* OpenAPI 3.0.  This is still JSON or YAML, so not suitable for manual content.
* others.. commercial things like gelato.io etc...

To create an Asciidoctor-based specification, the following two approaches seem possible:

* using the API blueprint approach, treat the source file like a .adoc file and generate a document from it, by converting it to adoc first. This might not be that hard - e.g. wrapping the JSON bits with [source, json] blocks etc.
* using the Swagger/OpenAPI approach, one would write a normal Asciidoctor set of files, and use includes to pull in bits of the JSON APi file.

BUT. Don't forget, we want to use the same REST API source file in its normal toolchain to generate the 'active' form as well.

Anyone here have any experience or ideas on this question?

thanks

- thomas

The development team I work with decided to use Swagger to document the APIs being developed. As you've noted, Swagger is missing some bits, such as usage examples, but it was close enough to the right thing for the developers; the interactive API made the difference.

Since our product is written in PHP, I wrote a PHP script that scrapes the API source code. It looks for Swagger identifiers and composes an in-memory model of each API. We worked out a number of new Swagger-like identifiers that allow us to provide usage examples, and any descriptive text is marked up as AsciiDoctor right in the API docblocks (it's cool to be able to use tables to describe the permutations of multiple parameters). Once all of the API source has been scraped, the script composes an Asciidoctor file representing a chapter, via a template that includes all of the up-front introduction to the APIs.

The completed Asciidoctor file is included into the overall product guide, and is rendered to static HTML via our DocBook toolchain (because we need the chunking control and presentation customizations afforded by XSLT). Since we also generate PDFs, we haven't made the API docs as fancy as API Blueprint, because the interactive elements just don't work in PDF.

I would imagine Swagger2Markup could be quite useful, but it didn't exist when I started writing my script. You could use it to generate the bulk of the API docs in Asciidoctor markup, and post-processing to inject any additional content you need. However your APIs are implemented, I expect you'll likely find some tool that gets you 80-90% of what you want, and then you'll need to customize it or its output to achieve a complete solution.

Unfortunately, aspects of my solution aren't shareable. But I can certainly provide more details if you feel the need.

thanks for the reply & very interesting.

So you are extracting from code to generate the formal adoc content. I think this is probably equivalent to the Swagger2Markup approach, except that you can add other tags as needed. You said:

eskwayrd wrote

it's cool to be able to use tables to describe the permutations of multiple parameters

Can you share an example of an API docblock like this?

Agree on the 80% + customisation thing - probably unavoidable.

thanks

- thomas

This example demonstrates the Swagger markup, as well as a custom Swagger-like identifier.

My script takes care of stripping off the docblock markup and identifying the indents. The content for a top-level identifier ends when a new top-level identifier (or the end of the docblock) is found. Any descriptive field/block is assumed to contain Asciidoctor markup, and the script applies Asciidoctor markup for everything else to suit our preferred formatting.

    /**
     * @SWG\Api(
     *     path="notify/",
     *     @SWG\Operation(
     *         method="POST",
     *         summary="Send a notification.",
     *         notes="
     *         .Who should expect to receive a notification
     *         [options="header",cols="1a,1a,1a,1a,1a"]
     *         |===
     *         | `event_type`
     *         | `members_only`
     *         | Members
     *         | Followers
     *         | Anonymous
     * 
     *         .2+| `normal` (default)
     *         | `true`
     *         | Yes
     *         | No
     *         | No
     * 
     *         | `false`
     *         | Yes
     *         | No
     *         | No
     * 
     *         .2+| `important`
     *         | `true`
     *         | Yes
     *         | No
     *         | No
     * 
     *         | `false`
     *         | Yes
     *         | Yes
     *         | No
     * 
     *         .2+| `alert`
     *         | `true`
     *         | Yes
     *         | No
     *         | No
     * 
     *         | `false`
     *         | Yes
     *         | Yes
     *         | Yes
     *         |===",
     *         @SWG\Parameter(
     *             name="project",
     *             description="The project id.",
     *             paramType="form",
     *             type="integer",
     *             required=true
     *         ),
     *         @SWG\Parameter(
     *             name="message",
     *             description="The text of the notification.",
     *             paramType="form",
     *             type="string",
     *             required=true
     *         ),
     *         @SWG\Parameter(
     *             name="event_type",
     *             description="The type of event, one of: `regular`, `important`, `alert`",
     *             paramType="form",
     *             type="string",
     *             defaultValue="regular",
     *             required=false
     *         ),
     *         @SWG\Parameter(
     *             name="members_only",
     *             description="Send notification only to members?",
     *             paramType="query",
     *             type="boolean",
     *             defaultValue=false,
     *             required=false
     *         ),
     *     )
     * )
     *
     * @apiUsageExample Send a regular notification.
     *
     *     [source,bash]
     *     curl -d "project:123" -d "message:Testing" http://my.application/api/notify
     *
     * @apiSuccessExample Successful Response:
     *     HTTP/1.1 200 OK
     *
     *     [source,json]
     *     { "success": true }
     *
     * @param  integer  $id            Project id.
     * @param  string   $message       The text of the notification.
     * @param  string   $event_type    The type of event, one of 'regular' (default),
     *                                 'important', 'alert'
     * @param  boolean  $members_only  Send notification only to members?
     * @return boolean  true if notification sent, false otherwise.
     */
   public function notify($id, $message, $event_type, $members_only)
   {
      ...
   }

Not sure if this speaks directly to your question or is only tangentially related, but FWIW...

Our team is using Spring REST Docs toolset (with RestAssured).  (This is what led me to the discovery of asciidoc).

Instead of scraping javadocs or annotations, it uses JUnit to stand up an "instance" of your service and then uses real HTTP request/response content to generate the asciidoc fragments ("snippets") of requests and responses.  The developer's responsibility is to write a "wrapper" document that imports the generated content.

The things I like about it:
- it's "resource-centric", as opposed to "URL-centric".  We are building a hyperlinking API, so publishing URLs all over the place is not really the way we want to go.
- when somebody changes the format of some response, chances are the test will fail.  This will force them to document the updated format (I always found with javadoc-driven documentation, people forgot to update the docs until complaints rolled in from the field)
- we can integrate it into our loadbuild process

Things not so likeable:
- it takes a lot of mocking
- false sense of security:  response format changes are caught... unless the mocked object needed to be changed to force a new property to be included, or it was a simple matter of the range limits on an existing property changed.
- our loadbuild is big and unwieldy so it's been an adventure trying to make JUnit drop the generated doc output in a place where the "wrapper" documents can find it and then having the asciidoctor task able to find everything and link it all together, all while using relative paths which are different in the "development" environment than they are in the "loadbuild" environment... and then make the whole thing work in both the IDE and ant.  (I'm grayer now than I was at the start of the week)
- if you want to include info in your generated docs that's a bit different from the tool's default, the API for that is a bit clunky

Spring REST Docs is still be a bit young (their first release was just this past Christmas, I think, and I'm not sure how big the team is), but I think the approach has promise.

Cheers,
Julie

Julie,

thanks for the reply. We will try to come up with a solution for our environment, and I'll share the results here (and they will be open source as well). Still interested in anyone else's experience.

- thomas