Tags:
dakar1Add my vote for this tag documentation1Add my vote for this tag extract_doc1Add my vote for this tag process1Add my vote for this tag usability1Add my vote for this tag create new tag
, view all tags

The Dakar documentation model is broken

There are some serious issues with the documentation model of DakarRelease as I pointed out in WhereAreDocsMasters. MattWilkie described the issues in Bugs:Item628, which I copy here for reference. (Matt, I could not have said it better)

-- PeterThoeny - 10 Oct 2005

svn breaks collaboration on docs

Details overheard on IRC:

>We also need to add a note on Plugins.TWiki regarding installing all plugins - saying that after plugin is installed, you have to go back to configure to enable it. Hmmm... Plugins.TWiki has one of those "Do not edit" comments at the end. Guess I can't add a comment about installing the plugins.
> Lynnwood wonders at what point in all this programmers automated efficiency we loose most of the collaborative benefit of a wiki?

A concrete example: JoeUser sees that DakarReleaseNotes does not have a link to TWikiUpgradeGuide. In order to do something about it s/he needs to

  • a) file a bug report Bugs:Item597,
  • b) someone with svn write access needs to edit the .txt file in an svn working copy,
  • c) developer needs to checkin the changes (SVN 6786) (after making sure that r.1.12 is changed to $Rev and JoeDev to TWikiContributor Bugs:Item613),
  • d) wait for automated cron job which runs every X minutes (15?) to update the website

This process is quite a distance from wikidom. "A defining characteristic of wiki technology is the ease with which pages can be created and updated."*

There is a need to keep a formal procedure for svn checkins. It ensures every checkin is documented and increases the quality of code by virture of a few extra moments of reflection. When it comes to documentation though, this procedure sucks rocks. Gone is drive-by fixing of typos, leaving only the determined and committed to keep things up to date while adding more to the inbox of those who would otherwise be spending their time fixing and writing code.

How to fix this?

One possibility:

  • add a mandatory "summary of changes" input to the edit screen of topics in the TWiki web (this is an enhancement request anyway)
  • on save the .txt file is checked into svn using the summary field as the checkin log message (perhaps with a prefix like Doc123: or WikiEdit123: ?)

Other solutions?

-- MattWilkie - 08 Oct 2005

The current documentation model is developer centric, it handles documentation the same as source code. This works fine for developers, but from a tech-writer's point of view it is very combersome to maintain content. As I have seen from a potential tech-writer who got scared away.

Here are some additional issues besides the ones Matt pointed out:

  • Feedback and discussion on a doc topic is not possible
  • TWiki users visit TWiki.org to fix or enhance documentation. They see out of sync content
  • Changes to docs are restricted to a few developers monitoring svn

A consequence of the unresolved WhereAreDocsMasters issue is that the docs on TWiki.org and in develop are out of sync now for about a year. Until Cairo, TWiki.org was roughly in sync with the frequent Beta releases.

Comprehensive docs are an integral part of TWiki. Docs and code are about the same in size (0.9MB each for Cairo, 1.2MB each for Dakar). That is, the current model works for 50% of the pie. We need to address the other 50%.

Actions that need to be taken:

  1. Short-term for DakarRelease
    1. One time sync of docs in TWiki.org's TWiki web into develop's TWiki web
    2. Add contributors to individual doc topics
    3. Update TWiki.org with Dakar
  2. For EdinburghRelease
    • Define and implement a system that addresses the needs of developers and tech-writers:
      • Developers: Source controlled docs
      • Tech writers: Maintain docs in the Wiki way on TWiki.org, including feedbacks

For now, lets concentrate on the short term solution for DakarRelease since this is urgent.

1. One time sync of docs in TWiki.org's TWiki web into develop's TWiki web

TWiki.org docs have been maintained for a year now and only a few changes have been carried over to the develop branch. Changes need to be merged selectively. I prepared for this:

  1. Make TWiki.org docs ready for merge: On TWiki.org we have actually two sets of docs: Beta docs and TWiki.org docs. The Beta docs are in a hidden Beta installation (which was used until Cairo to build releases). In the last few days I spent some time to clean up the docs and to bring the two sets in sync.
  2. Web-based diffs: It is now possible to compare a TWiki.org topic with the develop version, and also with the Cairo baseline version. You can access the diffs at the bottom in the Copyright section (this is disabled for TWikiGuest user).
  3. Record merge progress: We cannot automatically merge the content since some spec changed. I prepared DakarMergeProgress where we can track the merge progress.

I believe each and every of the 192 topics listed in DakarMergeProgress needs to be looked at. What is a good way to merge the docs?

This is tracked in Bugs:Item663.

2. Add contributors to individual doc topics

All signatures have been removed from the doc topics. This makes sense for developer who consider docs just as part of code. The stated reason to remove the sigs is that:

  1. All contributors should be attributed just in one place
  2. Signatures can be confused with users of an installation

There are issues with removing the contributors from the doc topics:

  1. Attributing contributors just in one place as a list of names is bad because you do not know who contributed what.
  2. By lumping doc attribution and code attribution together it is not clear if/how smaller doc fixes should be attributed

I suggest to separate the attribution of code and doc contributions. For code, areas of contribution should be listed in the contributors topic (Bugs:Item664). For docs, contributor names should be added back to the doc topic. A different format than the standard signature should be used to make it obvious that the contribution is from TWiki.org. Example:

Cairo docs   Dakar docs
Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum

-- TWiki:Main/PeterThoeny - 13 Nov 2004
-- TWiki:Main/AndreaSterbini - 29 May 2001
-- TWiki:Main/MikeMannix - 03 Dec 2001

  Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum

-- Contributors: TWiki:Main/PeterThoeny, TWiki:Main/AndreaSterbini, TWiki:Main/MikeMannix

3. Update TWiki.org with Dakar

Udating the code base is no issue, but I am not sure how to accomplish the update of the TWiki web. We cannot simply replace the TWiki web with the Dakar docs. Reason:

  • We would lose all feedback/discussions
  • We need to keep old topics (and mark them as deprecated) for reference
  • We would lose the old revs

Any ideas how to accomplish this?

-- PeterThoeny - 10 Oct 2005

The documentation in develop is what gets distributed to users: a smaller subset of twiki.org's documentation. It's of no use to add feedback (and author names) to the release documentation.

  • Clarification: The released docs should not contain the feedback and discussions, it never has. But they are vital in the maintenance process, thus should be retained for doc authoring. -- PeterThoeny - 10 Oct 2005

for me it is very demotivating to have the signatures removed after spending hundreds of hours on doc writing You are threading people's toes here. I don't see a difference between coding and writing documentation in this respect. Look how many hours get into programming TWiki - while noone sees the names of these contributors, if he is listed anyway. Still I assume that our coders don't get demotivated by this.

  • My apologies for bring up how I feel. I removed my comment since it is irrelevant to actioning on the doc model issues. Lets focus on finding a solution. -- PeterThoeny - 10 Oct 2005

  • Good question how and why he gets listed. I can't tell because it is not documented in the attributions, and I can't tell since there is no trace on TWiki.org and Bugs. On the other hand, other people who spent many many hours (on spec, logo design, docs) are not listed in DakarReleaseNotes. -- PeterThoeny - 10 Oct 2005

We would lose all feedback/discussions That is part of refactoring.

TWiki web docs could go into a CairoDocumentation web. We have to sort out which comments still make sense to keep.

New docs on TWiki should be marked as "For release" to keep the smaller subset.

-- ArthurClemens - 10 Oct 2005

There's a major problem with your scheme for attribution, and that is, what happens when a document is entirely rewritten? Does the original author retain their attribution? And if they are not to be attributed, where does the recognition of the work they put in get recorded? The documentation set in Dakar is ~9500 lines. According to svn praise about 3.500 of these lines are attributable to Peter, 2,500 to me, numbers varying from 1500 to 20 or so to a list of other people. And 7 to Mike Mannix. On that basis, if attributions fairly reflect lines in individual topics, Mike would be all but wiped out of the documentation set. I for one think devalues his contribution.

I guess the proposed approach is to keep a list of contributors at the bottom of each topic, and only ever add names, never remove them. So how does a name get added to that list? Is it just done whenever anyone corrects a spelling error? I have touched every single document in the document set, several times, but I would never claim to have actually made a significant contribution to them all. However, I have made significant contributions to some, including several total rewrites. So should I get equal recognition as co-author of that document with Main.IPNightly, who happens to be able to spell antediluvian? OK, so maybe only significant contributions are recorded by a sig in the topic. Who decides whether a contribution is significant or not? Who is going to stop Main.IPNightly from signing every topic where he makes an edit?

But Main.IPNightly has made a contribution, all right. They need to be recognised somewhere. So, in that case, to avoid the risk of offending anyone, shouldn't you list all documentation contributors at the bottom of each topic? After all, they have all contributed to the documentation set, to an extent where it is really hard to say "she wrote that bit" or "he wrote this bit"? And as Arthur points out, what about the code authors? Don't they get a crack of the whip as well? Shouldn't they be on the list? After all, they designed what is being documented, and in many cases claim original copyrights over the work.

Hmmm. Sounds a bit of a maintenance nightmare. So why not list them all in a separate topic, and %INCLUDE them into each documentation topic? Well, it would mean downloading a huge list of names for every documentation reference. So why not just add a link - for example, to the TWikiContributor page? You could simply sign every topic: -- Main.TWikiContributor

If you want to reflect each individual contriubution - and I absolutely support that (see DakarReleaseNotes) then it would be perfectly approriate to do that in the AUTHORS file, where PeterThoeny can be credited as the author of TWiki and the author of the vast bulk of the documentation, and Mike Mannix can be fairly recognised as having been a major contributor to the reorganisation and structuring or the docs, without the embarassment of having his signature plastered all over topics that he had no part in writing. And all the minor contributors can also be listed, recognising the fact that TWiki is now a true community effort.

-- CrawfordCurrie - 10 Oct 2005

Talk about demoralizing: consider the effect on moral of development team that was working hard finish up a release of:

  • Revisiting at the last moment decisions already made months ago by folks involved in development of Dakar.
    • I am guilty as charged with this one, I could have acted on this much earlier. Apparently it was not enough to lay out the issues in WhereAreDocsMasters. -- PeterThoeny - 10 Oct 2005
  • Proposing a complete and time-consuming overhaul of Dakar docs - actually ignoring all the work that has been done on them.
    • No, that is not the idea. What I meant was to keep Dakar docs as master and merge the additional TWiki.org contributions into Dakar -- PeterThoeny - 10 Oct 2005
    • Ah, thanks for clarifying (or apologies for reading in what wasn't there)! I'm good with that will try to contribute to that effort. -- LynnwoodBrown - 10 Oct 2005

I think we can call agree that the documentation management since the move to SVN needs work. Further discusson on options for crediting contributors is fine. But forcing these issues before completion of Dakar is... I'm lost for words. Let's just say I don't think it's such a good idea.

There has been a lot of work on refactoring, cleaning up, and weeding out un-necessary TWiki web topics in Dakar. And of course, there has been lots of technical changes to parallel changes in code (the bulk of which were done by the coders themselves, btw, and they deserve credit for having done that).

I would make a plea to focus on really essential refinement of the docs for Dakar at this point. If someone wants to incorporate the best of recent addition to TWiki.org docs, then fine. My higher priority is making sure we updated the docs to reflect current code.

I'd also rather err on the side of cutting a bit too much out of the docs shipped with Dakar as I think (and I believe there is near consensus in developer group on this point) that there is way too much is included in TWiki web. We can always add stuff back later - or offer a separate docs installment later. I do believe we are close to achieving a much better (and frequent) beta release system so let's not hold up Dakar release for working out every detail of a docs overhaul.

-- LynnwoodBrown - 10 Oct 2005

For the record, MichaelSparks is credited as a contributor because he contributed. He was listed in the cairo TWikiContributors topic - where his specific contribution was also not documented. Michael was a source of many inspriational ideas over the last year, through his work on O'Wiki. Out of respect for his copyright, however, none of his code (except that already in Cairo) was used in Dakar.

-- CrawfordCurrie - 10 Oct 2005

My goal as everyone else's is to get Dakar out of the virtual door as soon as possible. Hence the suggested split of "A. Short-term for DakarRelease" and B. For EdinburghRelease.

I would like to focus what needs to be done.

For a moment, lets assume that we do not merge the TWiki.org docs back. At the time of the release (or with a very shortly), TWiki.org needs to be upgraded to Dakar because many site admins and users running Dakar will come to TWiki.org for reference and for doc contributions. How can TWiki.org be upgraded with Dakar? Have two TWiki webs, a TWiki web and a TWikiDakar web?

-- PeterThoeny - 10 Oct 2005

Ah Peter, many of us have long ago recommended separate webs for things like each major revision. I'm glad you're coming to see the value of it.

-- AntonAylward - 10 Oct 2005

How about a TWiki web and a TWikiCairo web? That way the TWiki web is always the most "current", and old content is pushed to a different web. Also, you could have a TWikiDevelop web for ongoing work, but that could mean additional effort from time to time to update the SVN with the online version.

-- RafaelAlvarez - 11 Oct 2005

Rafael has a very good point about the need for the a web to be current. It can be very confusing reading and searching though the topics to find many are out of date or inapplicable ebcause of changes and evolution.

-- AntonAylward - 11 Oct 2005

One note of concern over using the TWiki web. The documentation in TWiki should always correspond to the code being run on that platform, so that examples work as advertised. Since TWiki.org doesn't (and probably shouldn't) run the bleeding edge development code, it's hard to see how this works. Of course, careful and consistent management of the content can avoid such problems, but on past performances we're not very good at that.

-- CrawfordCurrie - 11 Oct 2005

To me there is one simple mechanism for us to deal with this issue.

on TWiki.org the TWiki Web is a read only version of the TWiki web for the current release.

each topic points to a feedback page, where changes & improvments for that major release can be discussed, to be used in future.

This leaves us with clear and simple documentation for our users (who don't want to read a huge discussion just in case it applies to them), and makes upgrading the docco easier.

archiving old docco into TWikiCairo etc also suggests that we should archive into CodevCairo - something i and many others really think would be a good idea.

-- SvenDowideit - 11 Oct 2005

That all sounds very sensible.

-- ArthurClemens - 11 Oct 2005

If I were Cpt Picard, I would say: "Make it so!"

-- OliverKrueger - 11 Oct 2005

Sven's solution makes sense for larger changes which typically come about from code changes (we used to do it "like this" and now we do it "like that"). It doesn't address casual drive-by editing (fixing typos and spelling mistakes) which would be discouraged under this model.

Still a big step back to where we belong though!

Later, after having read through the attribution discussion in more detail: Has no one here used or looked at Ducky Sherwood's ContributorsPlugin ? Please do.

-- MattWilkie - 14 Oct 2005

We could make the "read only" docs writable, and send one appointed doc master a notification on a topic change. The doc master is respnsible for synching the topic and the version in the SVN repository. Make it clear in a note that major changes should be discussed on the [Topic]Discussion page.

-- ArthurClemens - 14 Oct 2005

Appointing a "doc master" is the thin end of the wedge that drives "ownership" and "exclusive rights" back into the project, just as we managed to dispense with them.

I would be happy if the "appointed doc master" was the whole of twiki-dev, however, so any responsible merger could merge.

-- CrawfordCurrie - 16 Oct 2005

Alright, ownership is not the idea. How could it be made visible that SVN and doc are out of sync? I see the links at the bottom of this topic, perhaps an 'equality check' could be done on both versions to indicate a difference.

-- ArthurClemens - 16 Oct 2005

there are a number of document management things we can do - i'll elaborate in a week or so when Dakar is out

-- SvenDowideit - 16 Oct 2005

Responding to Bugs:Item663:

I don't understand why Peter is using vi to make his edits. When I want to change the TWiki web I make changes to my copy through the web and then check in the change using SVN commit.

I grant that this shows my WikiName as the contributor rather than TWikiContributor but those are easily cleaned up in batch.

Am I missing something?

-- MartinCleaver - 26 Nov 2005

Following on from what Crawford said on 11 Oct :

There is also the issue that TWiki.org doesn't always run "pure" release code. It often gets extra features added in advance of a release and it's TWiki web has to reflect that. I think we will need TWikiDakar and TWiki webs.

Maybe we should consider having Cario and Dakar webs, each with Codev and TWiki subwebs. (and Bejing\Codev and Athens\Codev webs to archive older topics to.) We could then make a more general form of the oopsdocdiffcairo and oopsdocdiffdevelop pages (nice feature that, how's it work?) for comparing topics and with clever use of IF's in BROADCASTMESSAGE or the WebLeftBar add links between them.

-- SamHasler - 29 Nov 2005

On using vi: It is very iconvenient and inefficient for tech writing, but a less inefficient than (1) changing the docs in the browser, (2) sudo'ing the file permissions, (3) changing the author and version parameters using vi, and (4) checking in the changes with a commit message. Yes, I could create a TWikiContributor user and edit as such, but that does not solve the issue of changing the version parameter back to "$Rev$" each time. So back to steps 1..4.

Rhetorical question: How much Dakar documentation has been done in the last 12 month outside of the DEVELOP branch? Possibly none, probably close to zero. That is, there is one primary place to work on documentation. This place should be on TWiki.org where everybody can work on the docs and discuss the docs. For tech writing I do not want/need a commit message, just work on the doc in a version controlled way (identifying the author). There is no need to work on docs in two places which bring sync issues. Once a release it out there is no need to work on the docs of that release. Also, for doc authoring it does not matter that TWiki.org lags a bit behind the latest DEVELOP version.

So, here is a way that bring the doc authoring back to the wiki-way of doing things:

  • The TWiki web hosts the latest DEVELOP docs
    • I don't think that's practical. As Crawford said earlier "The documentation in TWiki should always correspond to the code being run on that platform, so that examples work as advertised." and as I said "There is also the issue that TWiki.org doesn't always run "pure" release code. It often gets extra features added in advance of a release and it's TWiki web has to reflect that." -- SH - 29 Nov 2005
  • There are %STARTDOC% and %STOPDOC% markers indicating what belongs to the official docs, and what is help text and discussion
  • There is an (semi-)automated sync into the DEVELOP branch
  • The TWiki web in DEVELOP branch is locked down for direct commits
  • There are new TWiki2004x09x01, TWiki2003x02x01 etc webs that contains the offical docs of pervious releases.

We can work on this once DakarRelease is out of the virtual doors.

-- PeterThoeny - 29 Nov 2005

What you propose basically prevent (or make it difficult for ) people that like me are forced to work "offline" for some reason, to contribute to the documentation. I kind of agree with the "everybody should be able to contribute to the Docs", but whatever model we choose should allow contribution from both twiki.org and DEVELOP branch.

OTOH, I agree that the "online method" to update documentation should be done in a different web than TWiki. The TWiki web should remain the "TWiki web of the current release", and then have different webs per version. I we don't do that, we need to answer one question: How are we going to maintain topics such as TWikiPreferences, TWikiRegistrationForm and such?

-- RafaelAlvarez - 29 Nov 2005

"The TWiki web should remain the "TWiki web of the current release"," No, it cannot

I'm going to say this again.

The TWiki web on TWiki.org must correspond to the code being run on twiki.org.

TWiki.org doesn't always run "pure" release code. It often gets extra features added in advance of a release and it's TWiki web has to reflect that.

-- SamHasler - 30 Nov 2005

I totally agree with Sam.

On Peter's rhetorical question. DEVELOP is a branch. The fact that it is a branch allowed us to get on with major doc rewriting without damaging TWiki.org or MAIN. You would have thrown a fit if I had started making some of the massive changes I made on TWiki.org.

I sincerely DEVELOP is not the end of the road; other branches will spawn, and will need to develop other, incompatible, docs. People have to be able to separate out doc work, same as they separate out code work. Another, less rhetorical, question. How much of the public doc work done on TWiki.org during Dakar development was actually of value in the Dakar docs?

Personally, I always work on docs in an external editor. Even when the docs are in TWiki, I copy and paste the raw text into the editor; working in the browser "editor" is too painful for my liking; like keyhole surgery.

Of course that will change with Wysiwyg; but then, that is where Subversive comes into it's own. Documentation should be in subversion, and the webs containing it should use the Subversive store driver. Then you can use TWiki to edit your docs. All that's required to do that is support for selecting the store impl on a web-by-web basis, which is missing in Dakar, but really isn't difficult to do.

As for discussion and comments on twiki.org; I'm sure it's useful, and to be encouraged; but there are other ways to achieve this; for example, comment topics associated with each document (c.f. Plugin dev topics)

CC

It would be helpful if we had something like MediaWiki's way of having talk topics associated with every topic.

-- SamHasler - 02 Dec 2005

I added three webs as a reference of existing release documentation: TWiki01, TWiki02, TWiki03.

I disabled all forms (except search) and added a broadcast message with a note that this is a readonly web for reference, and a note to contribute documentation in the TWiki web.

-- PeterThoeny - 05 Jan 2006

Proposal:

  1. Modify Subversive (the Subversion store implementation) so that it can check in. The main complication here is passing the username, password and checkin comment to the subversion server.
  2. Modify TWiki to support different store implementations on different webs.
  3. Use Subversive on documentation webs.

-- CrawfordCurrie - 20 Feb 2006

I am not sure if this is the best way to go. Here are the requirements as I see them:

  • Keep the existing history of the TWiki topics
  • On same topic, keep the document mode on top, followed by thread mode

I already prepared the DistributionDocument in the TWiki web with %STARTSECTION{"distributiondoc"}% ... %ENDSECTION{"distributiondoc"}%, so that a (to be written) twiki.org specific plugin can synchronize the TWiki web changes with SVN's TWiki web changes.

-- PeterThoeny - 24 Sep 2006

This is still very painful ... I think that one simple model would be to use the SVN backend to TWiki (Subversive) as Crawford suggests, for a special TWiki web that is not just the current TWiki.org docs, but the 'TWiki MAIN branch' docs. However, I'd need to see a lot more detail on how this works with thread-mode discussions. This would support the developers better as they could just commit directly to SVN for docs.

Merging this with Peter's ideas, we could keep the RCS backend but make the web correspond to an SVN branch such as MAIN, then have a special sync plugin that's run on Save to turn the doc changes into an SVN commit. This is better for tech writers.

This leads me towards a two-way sync - if the RCS backend is used, with SVN commits by the sync plugin, and on every commit to SVN by a developer, with an RCS checkin corresponding to the SVN commit, we would have the best of both worlds. There's still an issue if an SVN user checks in at same time as a TWiki RCS user, but that could be treated the same as two people using TWiki interactively (may be too hard - we could revert to locks perhaps)?

I would really like to see a solution here - I do occasionally make doc edits and have just had the pain of having to do an SVN commit just to get some edits into next release.

Also, there needs to be some way to make the Item part of the SVN comment optional if checking in docs - many doc updates are not critical bug fix type changes, and not associated with software updates, so I would make this recommended but not required. If I have to keep logging bugs just to make a doc edit, I will soon give up, and we really don't want such non-bugs clogging Bugs:WebHome. UPDATE: I now realise there is a generic 'doc bug' item, but didn't notice it in the TWiki doc pages I edited via TWiki...

-- RichardDonkin - 12 Mar 2007

I just wrote a script that synchs the content of the 'TWiki' web on this site - it handles the %SECTION tags to update just the document body and not the discussion. The script is written to update a local checkout (it does not check in) but it could easily be extended to check in changes synched from t.o. This could then be run as a cron job.

However it emerges from running the script that the TWiki web on t.o is a long way out of synch with the doc in Subversion. A quick look through shows that the doc in Subversion is more recent and accurate. Before my script can be installed as a cron, the TWiki web on t.o therefore needs to be synched with the latest doc in subversion. I did not make the script change the doc on t.o, as my understanding is that that is the preferred master. So here's what needs doing:

  1. Synch the topic content on t.o with SVN (the bit in the %STARTSECTION{"distributiondoc"}%)
  2. Ensure that the %STARTSECTION{"distributiondoc"}% exists in all t.o versions of distributed docs
  3. Ensure that t.o. versions of all docs in SVN actually exist

Tracked in Bugs:Item628

-- CrawfordCurrie - 19 Apr 2007

Should TWiki contain the most current docs, and TWiki04 and TWiki04x01 the (frozen) distributed docs?

-- ArthurClemens - 19 Apr 2007

That was my understanding; though possibly there should be a web for MAIN and a web for Patch04x01? Dunno.

-- CrawfordCurrie - 19 Apr 2007

That would be more correct. Because documentation might be added for features not yet available. Then TWiki web could become just a gateway for distributed docs.

-- ArthurClemens - 19 Apr 2007

Correct, the TWiki web is intended to be the latest working documents, should be in sync with the MAIN branch.

Thanks Crawford for the first step in automation, once implemented we will be more productive again with doc writing (it seems I am the only one who cares to sync doc contributions from the TWiki web to the MAIN branch, and I have a backlog.)

As for functionality, here is what I envision:

  • Semi-automatic sync, per topic:
    • Automatic update of SVN changes to TWiki web (but only after brought in sync)
      • That is best done as a script that runs on t.o, which I'm sure you could write in a few minutes (svn co, then svn up and merge the changes to the topics; just hack mergeT.O.pl)
    • Push button update from TWiki topic to SVN, with diff to verify difference. No automatic update, this is to avoid checkins of unwanted content from trolls and spammers.
      • This is originally why I made the update script only update an svn checkout. By working that way you can simply svn diff locally to see what has changed before committing changes.
  • Sync done only for distribution topics in TWiki web that have a %STARTSECTION{"distributiondoc"}% marker.
    • Exclude other webs such as Main, _default, etc.
    • Exclude selected distribution topics, such as plugin topics.
    • That's already done by my script, as described above
  • Each distribution topic shows the sync status at the place of the %STARTSECTION{"distributiondoc"}% %ENDSECTION{"distributiondoc"}% markers, with link to sync and diffs. Example:
    SVN sync status: out of sync (sync now)
    • I think that's something that has to be done on t.o.
  • Report table containing all distribution topics in TWiki web, with sync status for each. Example:
    Topic SVN Sync Status
    ATasteOfTWiki SVN out of sync (sync now)
    ATasteOfTWikiTemplate SVN in sync
    AccessKeys SVN in sync
    ... ... ...
    • Note: Something similar was done in temporary ZzzZ topic with twiki.org specific plugin variable %TWIKIORG{"mstat"}% to sync TWiki04 with TWiki web.
    • Ditto.

-- PeterThoeny - 20 Apr 2007

Comments in red . My view is that the script I have given you is already a significant improvement on what you had. I get the impression from the above that you want to wait for the functionality you describe before doing anything about synching the topics on t.o?

-- CrawfordCurrie - 20 Apr 2007

Crawford, I was re-reading your 19 Apr 2007 intro of your script several times, but I am still a bit confused what it does and how to use it. Where is your script is supposed to run? "Synchs the content of the 'TWiki' web on this site" suggests twiki.org. But looking at the source it seems somewhere else since there is an http access on twiki.org topics.

I intended the script to synch from t.o. to a local svn checkout, so that a developer (any developer) could then review the changes using svn diff and check in. Note that the getUrl calls I used are as efficient as possible (they use ?raw=all). I used getUrl because I, like most developers, can't get at the t.o topics any other way, but it would be trivial to make it work on a web instead. There is no support for synching from a local svn to t.o, though that is a simple reversal of the process currently coded in the script.

TWiki.org can't handle the load of a nested loop that iterates over all webs and topics. We probably need to do the sync only on twiki.org, with a local svn checkout.

That's fine, but only if the checkins are fully automated. Otherwise we are always dependent on someone with login access to t.o. to check in. Of course you could write a CGI script to do an svn commit of your local checkout on t.o., but it would be very fragile.

I suggested the "push button update" because I would like to avoid any command line operation, e.g. anyone in the TWikiCommunityGroup should be able to compare and sync topics in the TWiki web, also people without svn access.

Personally I think a push button update is too risky. The risks associated with it are:

  1. incorrect "fixes" getting into final doc without review. The people who tend to contribute to doc on t.o. are those working on the most recently released code. If you allow these people to modify the doc of the MAIN branch, you run the risk that they remove correct doc, because their TWiki doesn't work that way yet.
  2. Only one copy of the doc on t.o. Changes cannot be checked into MAIN and the current patch branch, because they are documenting two different codebases.
  3. spamming.
I think a checkin to subversion requires at least a cursory review from a developer before the change is accepted. While checkins are notified to twiki-dev, if the checkins are automated there is a high risk of people filtering out those checkins from their normal reviews.

-- PeterThoeny - 20 Apr 2007

Comments in red .

-- CrawfordCurrie - 21 Apr 2007

I find the proposed way that developers control the documentation too developer centric. I would like to bring the documentation back to the bigger TWikiCommunityGroup in the wiki-way, e.g. open up the documentation. By restricting the checkin to that TWikiCommunityGroup we have enough protection against spammers. After all, it worked well like this until Cairo (actually much better than the current situation).

The TWiki web is intended to be in sync with the MAIN branch. Doc writers are not used to work in multiple branches, and they do not need to. Developers can carry over docs into a patch branch.

-- PeterThoeny - 22 Apr 2007

Edit | Attach | Watch | Print version | History: r48 < r47 < r46 < r45 < r44 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r48 - 2007-04-22 - PeterThoeny
 
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2016 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.