create new tag
, view all tags

Cleaning Up Codev

Note: I rewrote the original plan and made some other changes to this topic. The previous version can be found at rev 7

While reading the TWikiIRC log I found that the "Cleaning out Codev" theme appeared again. This time KennethLavrsen said:

No matter what - cleaning out Codev does not start with a new web. It starts with a plan and commitment.

And I agree 100% with him. So, here is the plan I propose.

The Problem, The solution, The Goal

Kenneth Lavrsen thanks Rafael for following up on the discussion from IRC

The Problem

  • It is a general observation that it is difficult to find anything in Codev.
  • Codev is the web for developers. Developers feel that they should have a common ownership of the web.
  • All previous attempts have failed because of lack of commitment to actually put a lot of work hours into it.
  • Individuals don't really know where to begin and where to end.
  • The development team probably doesn't know or agree how to do it.

The solution

  • Define a goal which is simple and achievable
  • Divide the work into little pieces and get commitment from named individuals
  • Agree on this plan (this topic) and start doing it nice and easy.

The Goal

  • Reduce the number of gateway topics to the few that each match the different purposes of Codev.
  • %YES% Move open bugs reports to the much better suited and already existing and well working Bugs web on "develop".
  • Handle stale content so it does not become a cloud of noise or cloud of smole.
      • Do we delete anything?
      • Do we move it to alternative attic web?
      • Do we in some other way mark those topics so they are not per default showing up in searches etc.?

The Plan

The original plan was partially achieved.

Here is a new plan for which I commit myself. For the next two weeks, I'll travel the current WebTopicList, top to bottom, performing the following activities:

  • Tag with "archive_me" all the closed items (see definition below).
  • Tag with "stale" documents that seems to be interesting, but their content is stale.
  • Tag with "delete_me" topics belonging to the old workflow.
  • Tag with "extract_stuff" topics that have interesting tidbits that could be extracted or summarized.
  • Tag with "gateway" the gateway topics (some have different classification than GatewayTopic).
  • Tag with "multiple_sites" topics regarding using one codebase on multiple sites.
  • Tag with "multiple_forms" topics regarding multiple TWikiForms in the same topic.
  • Tag with "skin_consolidation" topics regarding skin functionality consolidation, simplification of default skin, define a Skin API.
  • Tag with "cas_tom" topics regarding Content Access Syntax/Topic Object Model
  • Tag with "marketing" topics that has useful information for twiki marketing, or is related to it.
  • Tag with "database_store" topics related to using a database to store topics

And, in general, some more tagging of the topics.

I'll work mostly with tagging because it does not mess with WebChanges.

-- RafaelAlvarez - 12 Aug 2008

I have discovered some patterns in my work, so a script can be built to automate some of it. In those cases, I'm leaving the topics as they are.

See CleaningCodev for the next block of topics to be processed. I process the whole block at a time.



  • CrawfordCurrie
    • My personal goal is to collect cookbooks, FAQs and other support resources in one place.
  • KennethLavrsen
    • Note from RafaelAlvarez: I listed Kenneth here because he supported the original initiative to clean up Codev.
  • SvenDowideit - 12 Aug 2008 - Rafael, an big massive thankyou for undertaking this task
  • MartinSeibert


Future actions

  • Identify and move relevant bug reports to the Bugs or Support web

Script to reorganize Codev

Most of the work can be easily automated, and a run of a single script can help a lot. This script will NOT move any topic around, just automate the reorganization I'm manually performing, which is the basis to a "true" reorganization of the content (which means archive, delete and move stuff around).

This scritp will tag as "archive_me" those topics that are considered Closed Items (See below) and change the forms from the old workflow to the new one (when it gives some value). It should report back a complete log of the process: for each topic which action was taken, and also a complete list of unprocessed topics.

Closed Items definition

Closed Items can be tagged as "archive_me"

Map between different Forms

WebForm TopicClassification ChangeProposalForm TopicClassification ChangeProposalForm CurrentState
FeatureDone FeatureRequest MergedToCore
FeatureHack FeatureRequest UnderInvestigation (if not tagged "brainstorming")
FeatureUnderConstruction FeatureRequest UnderInvestigation (if not tagged "brainstorming")
BugReport BugReport UnderInvestigation

WebForm TopicClassification BasicForm TopicClassification
FeatureHack BrainstormingIdea(if tagged "brainstorming")
FeatureUnderConstruction BrainstormingIdea(if tagged "brainstorming")
FeatureEnhancementRequest BrainstormingIdea
FeatureBrainstorming BrainstormingIdea

The rest of WebForm can be migrated to BasicForm, and if some topic has a missing TopicClassification it can be fixed by hand.

ChangeProposalForm with state NeedsARethink must change to state UnderInvestigation

Topics tagged as "Definition" but without a form, must have the BasicForm attached with TopicClassification DefineTerm

A tarball of Codev's content is useful to test the script and to measure it's effectivenes.

To verify results, the script must produce a list of topics affected and not affected by it. The final verification must be a manual check between the sandboxed run and the "production" run.

-- RafaelAlvarez


Rafael, an big massive thankyou for undertaking this task - I guess you'll know alot of twiki-history by the end of it smile

-- SvenDowideit - 12 Aug 2008

I'd like to help but there's some contradictory comments above (from Sam); specifically, should the forms be converted to the ChangeProposalForm system, or not?

-- CrawfordCurrie - 13 Aug 2008

While digging through the topics, I notices two things:

  • There is no way to mark that a proposal is duplicated/overlaps with another.
  • There is no way to mark a topic as a generic "Discussion" (ie, AddingNewWebsGuidelines is not a BrainstormingIdea, as is not a feature, and the result is most likely to be moved out of Codev)

-- RafaelAlvarez - 14 Aug 2008

After having reviewed more than 300 topics, I definitively need a script to "pre-cleanup" Codev so I can focus on those topics that actually need manual intervention. I can do the script, but I need a tarball of Codev to test it.

-- RafaelAlvarez - 18 Aug 2008


-- RafaelAlvarez - 21 Aug 2008

I'm all for doing what we can with scripted clean-up but I still believe that 90% of the benefit of more complex solutions could be obtained by a fairly crude archiving of old/stale content into an attic web that's not part of main site navigation but available for search on the rare occasion that it's needed. I would also support simply creating a new Codev web, designed newly from the grown up and simply retire/hide the current codev. It would still be available for reclaiming or referencing content but not be visible to new and casual users. Just like when you store a bunch of stuff in the attic and then realize 6 months later that you don't miss any of it, I think the old Codev web would pretty quickly fall into almost complete disuse.

Any of these solutions are far superior to the continued embarrassment of the current state of Codev. We've been waiting so long for some real action on this. Thank-you Rafael for pushing it again!!

-- LynnwoodBrown - 22 Aug 2008

Lynnwood, my original idea is to make a "pre-cleanup" to tag those topics that could be archived without risks, recognize which topics should be left in Codev and to identify which topics need a "human desition" on what to do. Then we can decide what to do with the topics marked as "archived".

OTOH, the more I read in Codev, the more I think that starting a new Web from scratch, moving the relevant topics (those linked from the main page plus new feature request) and make current Codev the "archive" is the way to go. But, hey... CoolURIsDontChange ....

-- RafaelAlvarez - 22 Aug 2008

I registered a new user, TWikiJanitor, and modified WebChangesForAllWebs to filter out topics modified last by him. If this experiment goes well, I'll also modify WebChanges and the rss topics, to prevent the barrage of change notification that occurs when I'm reviewing the topics. Also, this will separate the cleaning activities from my actual contributions (I'm at the top of the top of contributors this month smile )

-- RafaelAlvarez - 22 Aug 2008

Or, make a Codev2 and leave the old stuff where it is?

-- MartinCleaver - 25 Aug 2008

I would support that initiative, that has been mentioned several times before. Right now I'm just classifying topics with well-known tags and fixing forms so when the time is ready we can do "something", either move the garbage out or move the good stuff out. But first, the CoolURIsDontChange mantra must die (or at least define what a cool uri is)

-- RafaelAlvarez - 25 Aug 2008

I appreciate this cleaning work a lot. Is there a possibility to automate that and to use the scripts in corporate Wikis also? A marking of all "old entries" is a good thing to do in my opinion. smile

-- MartinSeibert - 25 Aug 2008

A big step forward would also be, if one could find www.twiki.org-pages in Google. It is not the case at the moment. See here: http://www.google.com/search?hl=en&q=cleaningupcodevrevisited&btnG=Google+Search

-- MartinSeibert - 25 Aug 2008

I restored WebChangesForAllWebs, as it no longer viewed changes on TDO what I (and I assume lots of others) are used to.

It showed changes made by TWikiJanitor which what I think it should not show!

-- MatthiasThullner - 28 Aug 2008

I thought that the change I did to WebChangesForAllWebs would effectively filter out TWikiJanitor, while still showing all the changes made in the pertinent webs. I compared WebChangesForAllWebs with WebChanges to make sure that it was ok.

Matthias, can you explain what changes where not being shown?

-- RafaelAlvarez - 29 Aug 2008


I did some changes which I locked for, but they have not been displayed. Just compare the current version of WebChangesForAllWebs with http://twiki.org/cgi-bin/view/Codev/WebChangesForAllWebs?rev=67.

  1. There are lots of differences between this versions
  2. rev67 displays lots of topics changed by TWikiJanitor (but no others)

-- MatthiasThullner - 29 Aug 2008

oops, I forgot to answer... I changed the query to use the latest QuerySearch feature, and now it is working as it should.

-- RafaelAlvarez- 12 Sep 2008

And it came to pass that the dutiful TWikiJanitor reached the thousand milestone in his road.

Yes, 1000 topics already processed... 4000+ to go.

I got some ideas while traversing the topic list. I'll list them here, and then move them to the proper place later:

  • We need a TopicClassification for Translations.
  • We need to rething the Taxonomy of Codev topics.
  • There is no way to mark a bug as duplicated.
  • We should really consider an archive web for each release.
  • We have way too many BrainstormingTopics that are forgotten.
  • We have many, many recurring themes (DatabaseStore being just one of them).
  • We have had way too many "political" fights on TDO.

I'll take a two week holiday from cleaning up, to work on my feature proposals.

-- RafaelAlvarez - 17 Sep 2008

awesome work Rafael!

-- CarloSchulz - 17 Sep 2008

Edit | Attach | Watch | Print version | History: r28 < r27 < r26 < r25 < r24 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r28 - 2008-09-17 - CarloSchulz
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2017 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.