The best change tracking application I have seen so far is FileMerge, which comes free with the dev. tools included with MacOS X (it's an Apple app). It gives a beautiful, graphical presentation of the changes between any two files (e.g. versions of the same file, from a version control system), high-lighting differences down to the character level. It also allows you to edit the resulting "merged" file, but it won't update the display of the comparison.

My main grip with it is that it only handles Latin-1 properly. When used with UTF-8, the character high-lighting goes way off track, since it considers bytes, not characters, when looking for diffs.

After reading through the GroupEdit RFP, the impression is that you want a single application that merge the core aspects of the following existing applications/technologies, with some refinements and tight integration:

- SubEthaEdit (for real-time collaborative writing, highlighting of authors, and tag high-lighting/styling
- FileMerge (for visual tracking and comparing of versions and diffs)
- version control system front end, e.g. CVL (for CVS) - I suggest that GroupEdit is agnostic to which version control system is used, i.e. that it can cooperate with several systems (cvs, subversion, others)
- iChat

The whole should be more than the sum of the above parts. The idea is not to try to integrate each of the above applications with each other, but serve as a starting point for what could be achieved: an integrated whole taking major ideas and features from the above list. It *might* be possible to integrate some of them, for example taking SubEthaEdit as a starting point (given that its developers are interested:-), and integrate chat functionality (I have a vague memory that iChat also provides some sort of framework that can be used by other apps), a version control front-end with features as outlined in the RFP with diff views and version comparison possibly along what is found in FileMerge.

Just my 2 c.

One caution about Darcs -- it is written in a little-used, academic programming language called Haskell. You have to install Haskell as part of installing Darcs. I did this once, long ago, and it was such a horror show that I have blanked out most of the memory.

Yes, I think combining the basic functionality of the apps you mention is roughly in the right direction. But keep in mind that SubEthaEdit is real-time only, which wouldn't be helpful in 99% of the writing situations we envision. And FileMerge, although it can compare files at the character level, doesn't let you edit in a version that shows you the comparisons, making it essentially useless. And as much as making GroupEdit agnostic to the version control system sounds like a good idea, I think it makes much more sense to have it work well with one open source system and to integrate it so the users and admins don't have to fuss with the variability that's guaranteed to come from an agnostic system. We want this to work, not be ultimately flexible.

Yes, I tend to agree, James. In most editing environments, simple check in/out would be fine. Perhaps resolving discrepancies could be something added in a later release.

Having used Subversion a bit, I think it provides everything we'd need on the back end. The real trick is in the front end, since locking, for instance, exists in Subversion, but most clients don't try to prevent users from checking out a locked file, as GroupEdit should. That's because merging changes isn't generally a program with code, whereas it is with prose text.

Yes, we're initially assuming that a major aspect of GroupEdit is to provide a really good front end to Subversion (or some other version control system, but Subversion seems like a good one).

Indeed, I think some careful thought will have to go into how comparisons are displayed between versions, because sometimes you want to see all changes since the beginning, sometimes you want to compare just two arbitrary versions, sometimes you want to see all the changes since some point (when you last saw it, for instance), and sometimes you want to hide all the changes and just see what the current version looks like.

There is indeed some conflict between offline mode and check in/out. My answer for an initial version is that check in/out should be possible only when online. An offline user should be able to open a file, and even make changes, but in such a way that they're entirely outside the system until manually merged back in (and the user should be warned about what will need to happen).

Dear Adam et al. I'm writing from a broadbased publishing perspective, with some limited software development experience, and I have some experience with older editorial systems. And, I'm thinking what would make a sellable product, and I wonder if this would be helpful:
* What output formats do you want to support? You say this is a "text" editing system, though I'm not sure you really mean it. I think you probably mean text (possiblly localized?), at least basic html, setext, and I think some kind of styled text (probably RTF). I would think RTF would make a good common ground, as you could then support a set of paragraph styles appropriate to your needs. Did I say RTF - there's the venerable TextEdit example project sitting in the OS X Developer Tools for anyone to adopt and modify!! Hopefully it includes the spell-checker.
* Workflow: I'm not sure how you work, but I'm guessing you toss stories back and forth among editors and writers until you hit a finish point, and I can see how this would work well for many, while other operations have a more rigid set of steps within a queue. At the very least, though, you have a) a bunch of submissions, or even just ideas, outlines, that are just floating there in various stages, and b) scheduled stories in an in-process queue heading towards a publication of one sort or another. So, this beast needs to be able to support multiple publications neatly, multiple editions, and a bunch of stories with rich labeling as to where they are at in production, as well as a bunch of categories of people assigned to work on them. So, to me, this thing will need a) a window where editors in charge can view clearly what's going on with any one edition in process, who has what part and what stage it is at, and b) an interface containing a list of stories that have their name attached, and labeled so they know it's time to act on it. Again, this is to make it more sellable.
* Version tracking, rollbacks, journaling, individual person-session edit-coloring. It boggles my poor mind. Wonderful, that you are promoting this!

Just checking in. Hadn't seen any postings lately. Wanted to see if there was anything new. :)

Wow! What a great concept. Anything that works better than Word's Track Changes would be a vast improvement. I'm working in the book world and often get files that are full of fancy formatting from authors who want to make it "look pretty" and have to spend a couple of hours stripping all that out before I start with the text edit. Then I send the file to another work station where all my changes in red appear in green and when I send the file back to my original station (don't ask -- I have to work on a Mac and a PC), there are blue and purple but no green or red changes. The actual changes are still there. And don't get me started on paragraphs that auto-renumber. Thank you for what you are doing. Keep up the good work.

UTF-8 and multiple suggestions on wordchoice for me is important. I work on longer documents with a need for 6-8 collaborators to help refine word choice.

It seems to me that Change Tracking and Comments are pretty much the only features you need that couldn't be supported with svn (and a user friendly front end to it) - I note that other people have made similar points.

You'd probably be able to use the svn history data to get all the delta's and assign blame. If you tie comment locations to svn revisions, you should hopefully be able to recalculate the appropriate new offsets for the comments.

Then you "just" need a nice gui for viewing that data and editing the file, delta and comment data.

Yes, I think you're right, but the devil lies in that "just". ;-)

After reviewing Adam's comments in TidBits #842, and reviewing the RFP, here are some additional thoughts. My perspective is influenced by years of publication production work, familiarity with a few high-end editorial systems for newspapers, a lack of knowledge of the software products like SubEthaEdit, some programming experience and website development experience. Let's say the features below are what I would like to see.

If I were going to attempt this project - and I assure you I am not - I would consider using Userland Frontier as the server (but after considering its long-term availability and future development. It has a lot of smart features built in or accessible. Mostly it would just serve and maintain serial versions and data on them such as the production stage, owner, writer, permissons, etc. But it would be a multifunctional server: besides serving story text, it would also serve info on, for example, all stories "owned" by an editor, a writer, etc; the production stage of each; the publication/date each story is intended; and so on. It's the latter capability that will make it a good editorial system.

I now think that stories need to be handled as tagged text with several types of tags. One set of tags would be for paragraph and in-line formatting, and would need to be rich enough to convert well into various user specified formats. The second set of tags would control the text coloring of edits and comments by various people. But I would be wary of trying to be able to see numerous stages of changes all in one version; rather it should be made easy for a user to call up previous versions for reference. Some metadata at the top would define the version, and other potentially useful control information. A possble third set of tags, possbly built into the first, might identify indivdual paragraphs (understand that a headline, subhead, bylines and other user definable styles are paragraphs); this might allow for better management of changes.

UTF-8 and Unicode probably should both be supported, or at least the latter.

The editor would probably need to be built from the ground up. I would like it to display a) styled text, as it is easier to read. That's not to say it should be the final format, just some format that the user can define for their own preferred writing/editing convenience. b) Single and multiple display panels in a single window, where one would always display the current version, and a second could display either a copy of the current version where the reader can scroll up and down to check for consistency between a current paragraph and other parts of the story. Alternatively, the second panel could display a previous version, _or_ extra windows could be opened to display these earlier versions. c) a switch for displaying in narrow or standard column views - I like the narrow for editing and proofreading.

Version control and offline edits. In professional publications I have always worked with fairly strict version/production stage controls. You have to understand that it is during edits and rewrites that a lot of errors get introduced in the copy. Essentially I see all version changes to be offline, but the editor might have a save button that would save to the active version on the server if it is available (ie. the user is internet connected that moment). A version called up from the server (as opposed to some copy on disk), would set a "lock" on the server, so that if say the writer has called the last version for rewriting, a lock is set, and the editor can see right away if the version is out for rewrite. The locks should be something the editor can override when necessary. On the user side, the version he or she has subscribed to such that they own the version is locked for their own use; when they are finished the press a Save and Submit button that releases the lock, and to do further changes they need to resubscribe, which starts a new version.

The pesky part will be the necessary situation where the writer really is offline and cannot access the server to subscribe to a version. Assuming they have the most recent version on disk, they can open it and work on it, but when the eventually submit it, it gets saved as a new version. Occasionally, another person may work on the story at the same time. So the server has to recognize that two current versions exist and communicate it. Here's a case where paragraph labeling could be handy: the editor could call up both versions (note the two display panels above) and step through the two probably doing some cut and paste; or they might choose to join the two, where paragraphs are displayed together, the version x displayed with a version x tag displayed, followed by the version y paragraph with it's tag displayed. I should also mention that the editor should have the ability to roll back to a previous version and make it the current one.

Having built-in chat and internal and external email and other notification cabilities would be a great enhancement. While there may be a set workflow, the editor ultimately controls to whom a version gets directed when he/she submits it (back to writer, to editor, to production); and the reciever should get notification of that the story is in their queue. (others would be locked out presumably.

Live spell checking and regular expression search/replace capabilities as suggested previously by others would be highly desirable as well.

I see all these features as quite capable on the server side using something like Frontier. Doing a good editor would require some real skills. I'll skip any details beyond those mentioned already of the version management features, except to say they should provide a high level of control, communications between a story's team members, and access control to writers or others. I hope these thoughts are helpful.