How to track changes with text files?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

How to track changes with text files?

Chuck
(This started as an email to Dan Allen directly and at his suggestion I am posting this up for discussion to this group.)

TL;DR What is an easy way to incorporate the track changes functionality of MS Word to a plain text file writing process?

I'm hoping that you might have a reference or two on a matter that I've run into involving transitioning from writing docs in MS Word to plain text files, and using AsciiDoc markup in particular. I've yet to find anything written about this particular issue via internet searches (possibly due to poor queries).

In general, I have buy-in for making changes to our documentation process to go from our current method of writing/editing in MS Word to using AsciiDoc formatted files. As I'm mapping out the new process I'm having trouble coming up with a method to mimic a key piece of the process which is track changes. Seeing who changed what and, if applicable, with comments why. Keeping the process simple for non-technical people is important. They can be taught commands or things to do for processes but the easier the transition and methods the better.

So the simplified situation is as follows:

<writer > Thsi is the sentance that explains why it happens.
<editor1> This is the sentence that tells of why it occurs.
<writer > This is the sentence that explains why it occurs. //<writer >comment the word explains is important here because blah blah blah...
<editor2> This is the sentence that tells of why it occurs. //<editor2>comment the word tells is correct here because blah blah blah...

After editor2 passes the doc back to the writer, he can see the history of the discussion, see the changes by each user using track changes, and then accepts the verdict or kicks it back for more discussion. Then the person who does the final edit review before publishing has the entire history of edits and updates and accepts all the changes or asks questions from a new perspective.

The only idea I've been able to come up with is to continue the writing and editing in Word then save out what will be published to a text file. My concern is that I've read this can be problematic due to things Word might do to the resulting text file. (Is that true to your knowledge with using AD markup?) The big downside to this idea is if a minor edit is done it has to be done in the Word doc, exported, and handled each time to keep everything in sync. I'd rather just deal with text files to keep the process as simple as possible.

Thanks for your time,

Chuck
Reply | Threaded
Open this post in threaded view
|

Re: How to track changes with text files?

LightGuardjp
As of right now there is nothing out there that does this for AsciiDoc files that in aware of. The best we have currently is GitHub's web interface and online editor. We could build an editor that worked with git, svn, cvs, mercurial, etc. but that hasn't been done and would take some time to roll out a version one. 

That being said, technically it shouldn't be that difficult to do. Any of the developers reading this, you could look at doing something on the JVM (javafx, shoes4, griffon, or others and using the java integration which would also easily give us easy access to fopdf). Perhaps one of the current editors we already have may work well as a base. 

Sent from Mailbox for iPhone


On Tue, Aug 13, 2013 at 7:18 AM, Chuck [via Asciidoctor :: Discussion] <[hidden email]> wrote:

(This started as an email to Dan Allen directly and at his suggestion I am posting this up for discussion to this group.)

TL;DR What is an easy way to incorporate the track changes functionality of MS Word to a plain text file writing process?

I'm hoping that you might have a reference or two on a matter that I've run into involving transitioning from writing docs in MS Word to plain text files, and using AsciiDoc markup in particular. I've yet to find anything written about this particular issue via internet searches (possibly due to poor queries).

In general, I have buy-in for making changes to our documentation process to go from our current method of writing/editing in MS Word to using AsciiDoc formatted files. As I'm mapping out the new process I'm having trouble coming up with a method to mimic a key piece of the process which is track changes. Seeing who changed what and, if applicable, with comments why. Keeping the process simple for non-technical people is important. They can be taught commands or things to do for processes but the easier the transition and methods the better.

So the simplified situation is as follows:

<writer > Thsi is the sentance that explains why it happens.
<editor1> This is the sentence that tells of why it occurs.
<writer > This is the sentence that explains why it occurs. //<writer >comment the word explains is important here because blah blah blah...
<editor2> This is the sentence that tells of why it occurs. //<editor2>comment the word tells is correct here because blah blah blah...

After editor2 passes the doc back to the writer, he can see the history of the discussion, see the changes by each user using track changes, and then accepts the verdict or kicks it back for more discussion. Then the person who does the final edit review before publishing has the entire history of edits and updates and accepts all the changes or asks questions from a new perspective.

The only idea I've been able to come up with is to continue the writing and editing in Word then save out what will be published to a text file. My concern is that I've read this can be problematic due to things Word might do to the resulting text file. (Is that true to your knowledge with using AD markup?) The big downside to this idea is if a minor edit is done it has to be done in the Word doc, exported, and handled each time to keep everything in sync. I'd rather just deal with text files to keep the process as simple as possible.

Thanks for your time,

Chuck


If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/How-to-track-changes-with-text-files-tp453.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML

Reply | Threaded
Open this post in threaded view
|

Re: How to track changes with text files?

Chuck
LightGuardjp wrote
As of right now there is nothing out there that does this for AsciiDoc files that in aware of. The best we have currently is GitHub's web interface and online editor. We could build an editor that worked with git, svn, cvs, mercurial, etc. but that hasn't been done and would take some time to roll out a version one. 
Unfortunately using Github is not an option for the particular project I have in mind. I'm going to see what process I can come up with to make this work without using Word. Its a small enough team that I can probably come up with some git/diff usage that will work.

Meanwhile I'm going to continue looking into options as I work up the idea.
Reply | Threaded
Open this post in threaded view
|

Re: How to track changes with text files?

Chuck
In reply to this post by Chuck
As a follow up to this, it looks like the solution to the track changes issue will be to use a self-hosted etherpad-lite instance.

One thing that I've not looked hard for, but so far have not found, is an Asciidoc plugin for etherpad-lite. If you are aware of one, I'd be grateful for a pointer.

Thanks,

Chuck
Reply | Threaded
Open this post in threaded view
|

Re: How to track changes with text files?

snowolfe
In reply to this post by Chuck
This is not integrated with asciidoc (or asciidoctor) - but looks like it might work generally, particularly if you use a supported text editor that recognises both markups:

http://criticmarkup.com/

Not sure if it offers enough functionality for your needs. Could be worth looking at as an addition to the supported markup for asciidoctor.
Reply | Threaded
Open this post in threaded view
|

Re: How to track changes with text files?

Chuck
snowolfe wrote
This is not integrated with asciidoc (or asciidoctor) - but looks like it might work generally, particularly if you use a supported text editor that recognises both markups:

http://criticmarkup.com/

Not sure if it offers enough functionality for your needs. Could be worth looking at as an addition to the supported markup for asciidoctor.
I do like the idea of this, however as you say, it does not offer all the functionality I need. If it can be expanded to include a comment field it would be better for my needs. That way the comment field can be used to include the user who made the change and reasoning.

The main pitfall that I can see after a quick thought is that if there is a long sentence with changes, and a subset of that needs further editing, can the markup be nested?
Reply | Threaded
Open this post in threaded view
|

Re: How to track changes with text files?

asotobu
How about creating a custom extension? I don't I am thinking while I am writing, but maybe we could write a custom extension which works for people who expects the same behaviour. I am interested on working on it.

Alex.
Reply | Threaded
Open this post in threaded view
|

Re: How to track changes with text files?

snowolfe
In reply to this post by Chuck
The CriticMarkup site suggests using the comment critic {>> <<} along side the edit critic for notation of reasons and commenter e.g. {>>@snowolfe this is my comment<<}.

As for nesting - technically I don't see an immediate reason why this could not not be done. However, that implies that parts of the critic was accepted and part was not. There are potentially several ways to handle that, and I am not sure which way would make the most sense.
Reply | Threaded
Open this post in threaded view
|

Re: Editing markup http://criticmarkup.com/

snowolfe
This post was updated on .
In reply to this post by Chuck
Moved this post here to add my comments in-line.

I think some kind of plugin or extension that works during the conversion process (asciidoctor > output) would be great - assuming we want to add support for this markup format.

I see four output options needed:
1. Generate output with all critic markup shown in the visible output. Output clearly identifies critic's.
Maybe: --cm-showall

2. Generate output with just the latest critic markup clearly identified, embed the history of critic markup as comments in the output, alternatively output the history as linked footnotes.
Maybe: --cm-latest

3.  Generate output with accepted critic, as it should be shown in completed doc, embed the history of critic markup as comments in the output.
Maybe: --cm-history or --cm-embed

4. Generate final output, after critic applied. No embedded history in final output. One potential output option for this, could be to generate "clean" asciidoc" output; that is, remove all critic markup from adoc file, useful for distribution of clean adoc source file (in case it needs to be processed by another tool, and to maintain compatibility).

A variation on the first two options maybe to toggle the 'display' of any associated critic comments, i.e. show just the critic markup for add, change, delete (I think highlight {== ==}{>> <<} always needs the comment, as it probably does not make sense without it) or show the add, change, delete with any immediately trailing comment.

Two things occur to me in relation to the above:
Which is the latest critic? - my suggestion is that it be purely an ordered analysis, last entry is the most recent. This avoids issues with date and time (particularly if you have to manage cross timezone editing).

How to identity accepted/rejected entries? - I think there needs to be an accepted marker, any other markers would depend on all sorts of use-case variations, e.g. definitely rejected, still under consideration, consensus needed.

Suggestions for accepted and rejected:
 These make it possible to use without interfering with the current critic markup spec, but would not be understood by other tools...
   {&&} Accepted, placed in front of the winning entry.
   {!!} Rejected, optional - placed in front of the rejected entry.
 Alternatively, add one of the above to the opening bracket entry, breaks the spec...
  {++& , {++!
  {--& , {--!
  {~~& , {~~!
 Use a special Comment entry, will not break current spec and will not affect other tools, benefits from being able to add additional commentary in the same entry...
  {>>&<<} Accepted, before the winning entry - with comment {>>& @snowolfe This is perfect <<}
  {>>!<<} Rejected, before the entry - with comment {>>! @snowolfe Not a good edit <<}

Comments?
Reply | Threaded
Open this post in threaded view
|

Re: How to track changes with text files?

mojavelinux
Administrator
In reply to this post by Chuck
Chuck et al,

This is an important topic and I'm glad you've brought it. I've been following alone with the conversation, but haven't been able to focus on a reply until now.

I'll start off by saying that I don't condone the use of Word for any purpose, including track changes ;) The migration from Word to plain text should be a one way street, and an exodus at that. I could probably rant about it all day, but instead I'll let this angry rant about the history of Microsoft Word and a call to put it to death say it for me:

I'll try to address your question, "What is an easy way to incorporate the track changes functionality of MS Word to a plain text file writing process?", by sharing some ideas and observations. In a separate post I'll share my thoughts about critic markup.

NFJS, The Magazine, which made the switch to AsciiDoc for authoring at the beginning of this year. I've been observing the editing workflow ever since, and even participated in it when I wrote my article about AsciiDoc. The way the editor is communicating with the authors and vice-versa is by using line comments. Here's an exchange from a recent article:

// MS: Just an aside...have you checked out Gradle's ability to do this kind of thing with a lot less legwork? Quite some goodness there...
// KK: I've only touched the surface of Gradle. I'm currently in the process of migrating several build scripts to Gradle. In time...

That sort of matches what you originally proposed. You could also use comment blocks to write more free-formed text.

Line comments can take you pretty far since they can be used just about anywhere in the document (except in verbatim blocks). If you adopt the sentence-per-line [1] methodology, then the line comments can be as fine grained as a sentence or even a phrase. (I strongly recommend against line wrapping at a fixed column).

The shortcoming of line comments is that they are primarily useful for discussion. It's tedious to use them to explain every little change. They also get noisy after a few rounds of edits. We should consider them a tool, but not the whole solution.

That brings us to the more traditional "track changes" feature (in contrast to document notes).

Track changes is nothing more than a revision system (e.g., git), except it's proprietary and remarkably bad. You only stand to gain by switching to plain text and a real revision system like git. Keep in mind that you can use git without GitHub. You don't even need a web interface, though if you want one there are alternatives to GitHub that you can install on your own servers, such as GitLab [2] or Gitorious [3].

It's easy to get the impression from using GitHub that diffs can only show lines that have changed. The diff view on GitHub is not optimized for prose. In git, each commit stores exactly what characters have changed. The diff algorithm has the capability of showing these changes just like they are shown in Word. They can even been annotated based on who made the change. Here's an example of how a prose diff *could* look:

To compliment this view, a summary of the changes can be communicated using the commit message. Thus, with a revision system alone you can get both inline changes (add, delete, change) and notes. Add line comments into the mix and you've matched the capabilities of track changes.

The only thing that's missing is a proper user interface to display the changes. That's just a matter of looking for a good diff viewer and perhaps some helper scripts. If you want an out of the box experience, take a look at Editorially [4], Authorea [5] and similar online services that are doing exactly this type of thing (but only for Markdown or LaTeX so far). Whatever solution we favor, I'm confident Word isn't a part of it :)

I hope that helps. I'm very interested to hear how well you think this fits with your requirements.

(I'll comment on critic markup next).

-Dan

[1] http://rhodesmill.org/brandon/2012/one-sentence-per-line
[2] http://gitlab.org
[3] https://gitorious.org
[4] https://editorially.com
[5] https://www.authorea.com


On Tue, Aug 13, 2013 at 7:16 AM, Chuck [via Asciidoctor :: Discussion] <[hidden email]> wrote:
(This started as an email to Dan Allen directly and at his suggestion I am posting this up for discussion to this group.)

TL;DR What is an easy way to incorporate the track changes functionality of MS Word to a plain text file writing process?

I'm hoping that you might have a reference or two on a matter that I've run into involving transitioning from writing docs in MS Word to plain text files, and using AsciiDoc markup in particular. I've yet to find anything written about this particular issue via internet searches (possibly due to poor queries).

In general, I have buy-in for making changes to our documentation process to go from our current method of writing/editing in MS Word to using AsciiDoc formatted files. As I'm mapping out the new process I'm having trouble coming up with a method to mimic a key piece of the process which is track changes. Seeing who changed what and, if applicable, with comments why. Keeping the process simple for non-technical people is important. They can be taught commands or things to do for processes but the easier the transition and methods the better.

So the simplified situation is as follows:

<writer > Thsi is the sentance that explains why it happens.
<editor1> This is the sentence that tells of why it occurs.
<writer > This is the sentence that explains why it occurs. //<writer >comment the word explains is important here because blah blah blah...
<editor2> This is the sentence that tells of why it occurs. //<editor2>comment the word tells is correct here because blah blah blah...

After editor2 passes the doc back to the writer, he can see the history of the discussion, see the changes by each user using track changes, and then accepts the verdict or kicks it back for more discussion. Then the person who does the final edit review before publishing has the entire history of edits and updates and accepts all the changes or asks questions from a new perspective.

The only idea I've been able to come up with is to continue the writing and editing in Word then save out what will be published to a text file. My concern is that I've read this can be problematic due to things Word might do to the resulting text file. (Is that true to your knowledge with using AD markup?) The big downside to this idea is if a minor edit is done it has to be done in the Word doc, exported, and handled each time to keep everything in sync. I'd rather just deal with text files to keep the process as simple as possible.

Thanks for your time,

Chuck


If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/How-to-track-changes-with-text-files-tp453.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



--
Reply | Threaded
Open this post in threaded view
|

Re: How to track changes with text files?

mojavelinux
Administrator
In reply to this post by Chuck
I came across a clear and concise blog entry that describes how to enable prose diffs (i.e., word diffs) when using git, with examples. This nicely demonstrates how you can get output in the same spirit as track changes.

http://blogs.atlassian.com/2013/06/git-diff/

Here's the quick command:

 $ git diff --word-diff --color-words

Note that this isn't limited to git. The git example is just an easy way to show what is possible.

-Dan
Reply | Threaded
Open this post in threaded view
|

Re: How to track changes with text files?

mojavelinux
Administrator
In reply to this post by snowolfe
CriticMarkup looks very interesting!

After reviewing it, I followed up with a search to find something similar, but no results turned up. Since CriticMarkup seems to have traction already, and it has a well-defined spec, I'm inclined to say we should just support it.

What I like most about CriticMarkup on a first look is that it uses curly braces, which maps closely to the attribute reference syntax in AsciiDoc. That means it should be easy to parse. It also fits semantically since, like attribute references, it represents a text substitution of some sort.

The timing is good, since I've been sitting on a patch that introduces a proofreading syntax based (e.g., [+]#added#, [-]#removed#). I hadn't yet merged it because I felt it was a bit too complex. I'm glad I waited, because I think CriticMarkup may offer a much cleaner syntax...and clean is what we really want.

We could start by implementing it as an extension, as Alex suggested, then migrate it to the core if we decide we like it. Asciidoctor doesn't have an extension point yet for arbitrary inline syntax, nor for attribute resolution. Thus, this could be handled using a preprocessor.

I'm not sure if CriticMarkup supports nesting, but we could support it in Asciidoctor. AsciiDoc has always recognized nested attribute references (although Asciidoctor doesn't them), so the nesting would at least be consistent.

With a combination of line and block comments, prose diffs and CriticMarkup, I'd say we can certainly achieve something for plain text documentation that can match, or exceed, what track changes offers. Plus, it's just plain text :)

-Dan



On Mon, Oct 14, 2013 at 9:58 PM, snowolfe [via Asciidoctor :: Discussion] <[hidden email]> wrote:
The CriticMarkup site suggests using the comment critic {>> <<} along side the edit critic for notation of reasons and commenter e.g. {>>@snowolfe this is my comment<<}.

As for nesting - technically I don't see an immediate reason why this could not not be done. However, that implies that parts of the critic was accepted and part was not. There are potentially several ways to handle that, and I am not sure which way would make the most sense.


If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/How-to-track-changes-with-text-files-tp453p805.html
To start a new topic under Asciidoctor :: Discussion, email [hidden email]
To unsubscribe from Asciidoctor :: Discussion, click here.
NAML



--
Reply | Threaded
Open this post in threaded view
|

Re: How to track changes with text files?

asotobu
In reply to this post by mojavelinux
Yes, I love the CriticMarkup way too. Also I think that a preprocessor would be the best choice, I think that I will find time from no where to start an implementation. Dan this makes me think that we need a github repo in asciidoctor organization to upload extensions (maybe one for Java, and another one for Ruby).

Alex.


2013/10/21 mojavelinux [via Asciidoctor :: Discussion] <[hidden email]>
I came across a clear and concise blog entry that describes how to enable prose diffs (i.e., word diffs) when using git, with examples. This nicely demonstrates how you can get output in the same spirit as track changes.

http://blogs.atlassian.com/2013/06/git-diff/

Here's the quick command:

 $ git diff --word-diff --color-words

Note that this isn't limited to git. The git example is just an easy way to show what is possible.

-Dan



If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/How-to-track-changes-with-text-files-tp453p838.html
To unsubscribe from How to track changes with text files?, click here.
NAML



--
+----------------------------------------------------------+
  Alex Soto Bueno - Computer Engineer
  www.lordofthejars.com
+----------------------------------------------------------+
Reply | Threaded
Open this post in threaded view
|

Re: How to track changes with text files?

Rob Sykes
asotobu wrote
Yes, I love the CriticMarkup way too. Also I think that a preprocessor
would be the best choice, I think that I will find time from no where to
start an implementation.
How is this going to work—context diff with interactive accept/reject?
Further editing may be needed after reviewing the changes, so I'm not sure that a preprocessor is a good fit.
Reply | Threaded
Open this post in threaded view
|

Re: How to track changes with text files?

asotobu
mmm yes maybe I have been precipitated but accepting/rejecting changes can involve a UI, then we should develop a preprocessor which starts a UI (or console) and asks for each one if user wants to accept or reject the change, or another way is to implement some strategies, like all rejected, all accepted, or the 1,4,5A 2,3R which means the first, fourth and fifth occurrences are accepted, second and third rejected (I don't like this approach but maybe can help to give another idea).

Of course a preprocessor is still possible, of course if it is accepted or rejected, then the document cannot be modified, it is rendered as is. Also this implies to create a temp copy of the document so you can always rollback.

Well it seems that it is harder than I expected but we can think a solution for each case.

As much time I think about it I see more and more challenges.

Alex.

2013/10/22 Rob Sykes [via Asciidoctor :: Discussion] <[hidden email]>
asotobu wrote
Yes, I love the CriticMarkup way too. Also I think that a preprocessor
would be the best choice, I think that I will find time from no where to
start an implementation.
How is this going to work—context diff with interactive accept/reject?
Further editing may be needed after reviewing the changes, so I'm not sure that a preprocessor is a good fit.


If you reply to this email, your message will be added to the discussion below:
http://discuss.asciidoctor.org/How-to-track-changes-with-text-files-tp453p842.html
To unsubscribe from How to track changes with text files?, click here.
NAML



--
+----------------------------------------------------------+
  Alex Soto Bueno - Computer Engineer
  www.lordofthejars.com
+----------------------------------------------------------+
Reply | Threaded
Open this post in threaded view
|

Re: How to track changes with text files?

Chuck
In reply to this post by Chuck
Wow. Lots of good discussion and ideas coming from my question. Thanks to everyone who is taking part in the ideas and spending time working on this concept. After reading through the comments and ideas there's a lot of good things to look into.

This has taken a turn than I had not expected if I'm reading the last few messages correctly. It seems like the idea has become to add support in asciidoctor utilizing the criticmarkup language with an approve/deny/modify marked changes feature as either a pre-processor and/or a part of generating the doc.

As a user of this technology, I would like to see this implemented as a separate tool. This way the editor(s) can use the tool and generate an edited asciidoc document free of the criticmarkup information and which can then be put into the automated tools used to publish.

I'm not a coder, but if there's anything I can do to help in the testing please let me know.