Seeking critique/advice: AsciiDoctor + Git as repository for local legislation

Hello all,

I originally posted this in a forum with the Fifty States Project, but am interested to get feedback here as well.

I am a fellow at the New York State Department of State.  I've got a group of colleagues and municipal officials (attorneys and legislative clerks) interested in a pilot project I am undertaking to build a prototype system to support the legislative process.  In a sentence, we are trying to adapt the continuous delivery approach used in open source software to work for local legislative process.  I am including context below, but to jump straight to the issue: We have a group of informatics students working with us this spring to develop a prototype, and I think our work will benefit from expertise in the technologies we are proposing to work with.

Context

Our hook here is that in New York (unlike any other state I am aware of), local laws must be filed with the Secretary of State (our agency) before they can take effect.  This is now done by snail mail.  Laws are sent uncodified (in a form similiar to session laws at the federal or state level), scanned in our agency, OCR'd, and then published with limited metadata online.  Or they are rejected and sent back if they do not meet validation requirements, which are formal in nature (disputes over substance happen in court).  We have authority to provide an electronic process, which offers an opportunity to launch a new platform.  This idea is inspired both others' observations that law is like source code and will some day be developed more like source code.  Local officials have identified three features they would find valuable to switch to an electronic process that we would like the platform to support:

* revision control -- detailed record of what changed, who did it, when, and why
* automated publication to shared, searchable statewide database in PDF, HTML, ebook formats
* instant validation and acknowledgment on submission

The attached pictures illustrate the concept.  One depicts continuous delivery, an approach to open source software that I understand as the GitHub way of doing things.  Source code is stored in a repo.  Pushes to the repo on GitHub (or similar) trigger continuous integration tests or, when the push is to the production branch, deployment scripts.  It offers an extensible, highly automated framework for software development, and it seems to be the way to develop software (at least open source) these days.

The other is the rough model for the prototype framework I am trying to develop. The steps are largely the same as for continuous delivery, but instead of a programming language like Javascript or Ruby, the source is a narrative text format like AsciiDoc.  And the output is a publication in PDF, HTML, EPUB, etc. instead of a deployed application.  I am proposing using AsciiDoctor as a starting point for representing law for several reasons:

* it's a plain text format that works well with diff tools (revision control)
* it's a mature standard based on more than two decades of refinement
* the AsciiDoctor implementation has a robust array of tools, and is accompanied by a comprehensive suite of tests
* its community has active interest in a Legal AsciiDoctor extension

As a preliminary test, I converted one county's codification into an AsciiDoctor repo and pieced together a script to build it into PDF, EPUB, and HTML.

Help Needed

While I admit it is a stretch to have most municipal attorneys and clerks learn how to use GitHub or GitLab, these platforms have great API's that a more appropriate user experience could be built around.  We will be seeking for our student team to build a prototype for that user experience (unless they have a better suggestion).  I think our chances of success are best if we make connections with others grappling with these issues or familiar with the tools we are planning to use.  So, please let me know:

* Do you have suggestions about connections?
* Are you interested to provide expertise as our work commences?
* Is AsciiDoctor a good tool for this?  Are there alternatives we should consider?
* If you were about to pull stakeholders together for a design studio where you ask them to sketch out their ideal user experience, what prompts would you distribute in advance to help them understand the potential capability of such a framework?

If you have suggestions about connections or are interested to provide expertise as our work commences, please be in touch.

Hi Ari,

I'm not sure you saw it, but I wrote a comment here https://github.com/asciidoctor/asciidoctor/issues/1316#issuecomment-174024449.

I agree Asciidoctor looks like the solution you are looking for to convert your source AsciiDoc files into EPUB, HTML, and .pdf

Also follow Ben Balter: https://twitter.com/benbalter he has several posts on Government using GitHub to host data to be presented to the public.

Thanks for your feedback @abelsromero, I did see it.  I am thinking of AsciiDoctor as the way to represent manifestations of law -- the acts and codifications -- but with other tools providing the indexing, archiving, etc.  I am thinking those tasks can happen as part of a deployment script that would run when legislation is merged into our "official" repo.  Some specific ideas:

* Validation: Use test scripts in a continuous integration testing platform to test submissions before deployment.  Users could also run these tests on each draft or public proposal to make sure what they are developing will meet submission requirements.
* Security: Limit access to the "official" git repositories so changes are made only by pull request and only accepted from the officers of the local government who are authorized to submit.  I think GitHub and GitLab both have capability to provide this sort of access control and integration with our identity management framework.  We could also use GPG signed commits.
* Official copy:  If there is a technical or legal reason that a git repo cannot be official, deploy scripts could generate a PDF expression and drop that into our existing, official ECM system.  We have an ECM system to store and index the scanned PDFs we receive through the current submission process.
* Search: Metadata can be built into the various expressions generated from the AsciiDoctor source.  It can also be parsed out and fed into our own database to provide search.  This could be a simple SQL or NoSQL table that has key metadata fields and links to the published expressions.

For handling links between enactments (the session laws) and codified segments of the law, my thought is to have a hook that runs a diff between the current law from the "official" repository and the working copy of the proposed/adopted version.  It would then generate a series of statements suitable for language in the enactment (session law) version and insert those in an appropriate place (for example, "The following {section} is added to {chapter}:") along with include::[] macros referencing the associated text in the codification.  I think this could be done without abusing the AsciiDoctor language (perhaps it would not be done at all by AsciiDoctor but in a hook run before a commit or push).  Or it could be through additional AsciiDoctor macros embedded in text.

Do those approaches address the concerns you raised?  Do they make AsciiDoctor any more or less attractive in your view as a tool for representing legislation?

Great idea, thanks.