Session Start: Mon May 28 22:02:04 2007 Session Ident: #twiki_release [22:02] * Now talking in #twiki_release [22:02] * Topic is 'http://twiki.org/cgi-bin/view/Codev/FreetownReleaseMeeting2007x05x28' [22:02] * Set by CDot on Mon May 28 19:30:06 [22:02] hi arthur, crawford, kwang, sven, will, kenneth [22:02] Good evening all. [22:02] hello [22:03] I have been trying to save Bug item 3347 for half an hour now. Does anyone here have access to the temporary bugs machine? [22:03] 'l [22:03] we have a holiday in the usa today [22:03] 'lo [22:04] we have public holiday here too [22:04] what are you up to today, will? [22:04] well, they. not me :( [22:04] twiki meeting ;) then going to a friend's bbq in mountain view around 5ish [22:05] just booked my trip to amsterdam :D [22:05] we will have a bbq ouside on the veranda [22:05] later today [22:05] nice [22:06] Can anyone else save on bugs web right now? [22:06] how much longer shall we wait? not so many people today [22:11] +10 min, time to start [22:11] CDot, sven__ one of you must have access to the wikiring server. [22:11] as usual, kenneth for minutes, pth for facilitation? [22:11] bugs.wikiring.com is on sven's dreamhost account [22:12] afaik, he is the only with with access to it [22:12] ah, i thin ksven is away for a few days [22:12] any idea when he is back? [22:13] This has to change. We should not be dependent on a bugs server and svn server that only Sven admins. We need the entire core team to have access. [22:13] Peter when the new machine is installed at our normal host please make sure we all get access. [22:13] this is normally the case [22:13] sven moved it temporarily away from develop.t.o due to server issue [22:13] We did not have access to d.t.o either. [22:14] Only CDot and Sven did as far as I know [22:14] And this is not acceptable. [22:14] i have access too [22:14] ok, we should give access to all core team members, as on twiki.org [22:15] action item for sven: create accounts for remaining active core team members on develop.twiki.org [22:16] I am on the minutes. I lost the 15 lines of test results I had added to a bug item because so I am pissed. [22:17] we need to get the develop.t.o server issue solved asap [22:17] i will call the isp again, but sven needs to handle the technical details [22:18] I do not fancy this devision of responsibility. it does not work. You two have been pointing fingers at each other for two weeks now. [22:19] what technical details? i thought it's been through that it's hardware failure or somewhere along those lines [22:19] not so much finger pointing, more an issue of miss-communication [22:19] i assumed sven was on top of the server issue since he filed the support request [22:20] i followed up with the isp, asking them to raise the issue to urgent [22:20] he was, he did mentioned that he went through all possible tests [22:20] they checked the server on sat [22:20] and determined that the hardware is fine [22:20] sven also said the test ran fine [22:20] except that the read/write benchmarks weren't sane [22:21] let's not spend more time on this, i will handle this with the isp and sven [22:21] lets start with the meeting: [22:21] # 1. Action Item Review [22:21] # 2. Review Proposed Features [22:21] # 3. Coordinate TWiki Release 4.2 [22:21] # 4. Change "Proposed For" field [22:21] # 5. Need to increase the code quality [22:22] anything to add? [22:22] ---+ 1. Action Item Review [22:22] # All. Implement accepted feature proposals before 28 May 2007 - or defer them to Georgetown (5.0) [22:23] let's defer discussion to agenda item 2 [22:23] # Peter: Create a TWikiBetaTesting topic in Codev, with info for beta testers. [22:23] done [22:23] please review http://twiki.org/cgi-bin/view/Codev/TWikiBetaTesting [22:23] this is the process topic [22:24] actual tracking will be done in the FreetownRelease topic [22:24] all: please review the proposed process [22:24] and change as needed [22:25] we need to recruit a beta testing coordinator for the freetown release [22:25] i do not expect this to be a time consuming role [22:25] i can't take the role, but i can help out in recruiting beta testers [22:26] I am not going to take additional roles either. I now go into the release manager role (you have seen me do that this weekend) and that takes a lot of energy and time. [22:26] agreed [22:26] what difference would it make compared to mass email to announce a beta release? [22:27] yes, i can do that at the time of announciong the first beta [22:27] to all twiki-announce subscribers [22:27] i think that's more than enough [22:27] but we need someone who coordinates the beta testing program [22:28] in what sense? [22:28] We need someone to commit to install and run 4.2.0 beta live. People are lazy. They sit and wait for the real release and let other do the hard work of testing. We need some of those activated to give a little back to the community. [22:28] yes [22:28] We typically end up with a massive 4.X.1 because many new bugs surface. [22:28] xcatly [22:28] the reason why we have this initiative [22:29] i'm guessing that would require real data with good enough customisation on twiki to weed out the bugs? [22:29] to hopefully reduce emegency releases once a month after a production release [22:30] yes [22:30] hm [22:30] i'm out of it. since i haen't even fully upgrade to 4.2.1 yet :/ [22:30] us developers are too busy fixing bugs to test them [22:30] the process topic shuld also state some info on how to install beta twiki pointing to existing production data [22:31] upgrade process you mean? [22:31] i think if we extend the beta stage to over a month, that would be suffice [22:31] especialyl since these days, beta stage is like months [22:32] and have a proper upgrade process to use the beta version. have pointers on how to rollback, etc.. [22:32] ok, i think we need to recruit a person who is not an active developer for the coordinator role [22:32] e.g. somebody else than today's meeting's participants [22:33] any idea how to recruit that person? [22:34] let's move on, and please leave feedback on the beta testing topic [22:34] ---+ 2. Review Proposed Features [22:34] kenneth? [22:34] We have ONE [22:35] SearchWithTWikiQueryLanguage - Peter is still showing concern it is time to vote because we cannot seem to reach agreement. [22:35] i stayed up until 2:30 and intended to provide feedback this morning [22:35] but ran out of time for the meeting [22:36] i prepared a detailed reply, but it is about 50% done [22:36] peter should be able to post his feedback [22:36] (not posed yet) [22:36] My position is that the advanced syntax we have now is only acceptable if we also have the shortcut syntax which is also implemented. [22:36] besides, i'm not really sure we have enough people to vote? [22:36] i really think we should not rush in a syntax that bites us later on [22:37] I find the new possibilities important enough to accept current proposal [22:37] Then we will vote on the proposal topic. This needs to get settled now. Code is there and if it has to be rewritten then it is now we make the decision and not in a month. [22:37] i would like to have this vote done next week [22:38] CDot this is your proposal. [22:38] CDot ping [22:39] i just added my anaysis to http://twiki.org/cgi-bin/view/Codev/SearchWithTWikiQueryLanguage [22:39] but the detailed proposal is pending (run out of time for the meeting) [22:41] i would like to ask the community to defer the decision to next week, e.g. not vote today [22:41] this can be done next week. Lavr__ , surely you must think it safe to push everything out by a week [22:41] b/c of so many bugs still, no? [22:41] the qtl is isolated in search, e.g. does not affect other parts of twiki, e.g. is a save code change if syntax change is done [22:42] because we need to free new features today and start fixing bugs. This opens a door to rewriting the whole search code again. [22:42] freeze not free [22:42] We have deferred the feature freeze several times already. [22:43] And it seems to me that CDot and Peter are miles apart and not willing to work hard on getting to a compromize. [22:43] This is the only proposal we have for today. Let us discuss. That is if CDot is awake. [22:45] I get the feeling that CDots strategy is to just drag it so the already implemented code gets accepted because noone bothers to fight the fight. [22:45] ok, we have choices: [22:45] 1. vote today (likely accepted) and live with limiations (not easy to create auto-completion in enhanced WYSIWYG editor; complete new & esoteric synatx to learn (not DOM, not regex, not SQL); need to be a programmer to grok) [22:45] 2. freeze code today except for tql, and decide next week [22:45] 3. freeze next week [22:45] i'm cool with either 2 or [22:45] 3 [22:45] 3 would be better [22:46] I am not going to manage another week of a bag of fleas of feature proposals. [22:46] kenneth: i have changed my proposal several times already, e.g. no more $formfield() syntax [22:46] based on crawford's feedback [22:46] oh wait, i read wrongly [22:47] 2 would be nice [22:47] we've been looking at freezing it for quite a long time IIRC [22:47] yes, i think 2 is a good way to go [22:47] feature freeze for all but the tql [22:47] and if it's only the search thing that's the main problem, we can decide that next week [22:47] yea [22:48] I can live with that - but it bugs me that after a week there has been so little focus on getting to an agreement. And we cannot even get a discussion going about it tonight. [22:48] CDot wake up. [22:48] okay [22:48] then let's go for 2 [22:50] kenneth, any other proposals we should discuss? [22:50] No. [22:50] And all other proposals that are not yet agreed and not yet implemented are deferred to Georgetown. [22:50] or in other words, anything that's not implemented as of now, is automatically deferred to 5.0 [22:51] yes. [22:51] :-) [22:51] although there are a number of easy to implement features i'd like to see in 4.2 [22:51] glad I could squeeze out the last ones [22:51] but the train left the station [22:52] I do not think we will have a situation at any given time where there are no new proposals that would be nice to have. [22:52] under " Completed Proposals" I see quite a number of implemented proposals that have Freetown as release version [22:53] PeterThoeny: we can save easy ones for 4.2.1 [22:53] Yes. I will change those to Georgetown. Or whatever the field will be called. [22:53] (ref next agenda point) [22:53] sorry, Edin [22:53] true, although features such as case insensitive linking and jump really decrease the geek factor of twiki [22:53] http://twiki.org/cgi-bin/view/Codev/TWikiFeature04x02 [22:53] That is because in many are old carry over proposals from last year [22:53] right [22:54] another one is search pagination [22:54] anyway, we missed the train on those ones [22:55] yes. next train [22:55] Or plugin [22:55] or 4.3 [22:56] if the market demands it [22:56] let's move on [22:56] ---+ 3. Coordinate TWiki Release 4.2 [22:57] we just decided to confirm today's code freeze date [22:57] with one exception: tql [22:57] Unless someone disagrees I will release manage it. [22:58] excellent, thank you thank you thank you kenneth! [22:58] you are the best man for this! [22:58] Don't thank me yet. I am going to bitch all of you - no exception - no special treatment - to get the quality up before release. [22:58] :-) [22:59] that's what we need [22:59] ok, on dates [22:59] in last meeting we stated beta release on 17 Jun 2007 [22:59] still shoot for this? [22:59] Yes. We need to keep this date if possible. But I fear that there will be two betas and a release a month after [23:00] .. after what we currently think. [23:00] beta testers will be involved in beta1 beta2, rc1, rc2,... [23:00] Unless new developers that normally only focus on their pet plugins step in and help with the bug fixing. [23:01] that will be an excellent opportunity for ktwilight :-) [23:01] time check: +60 min [23:01] :) [23:01] yes for sure. [23:01] have a bug day and everyone do it together, working on different bugs of course [23:02] am i right that web will work in the main branch until the day of release? [23:02] ktwilight if you want an easy job - then run the test cases in the TestCases web. And fix them if you can see that the new HTML code is OK but just different. [23:02] e.g. no new feature checkins until release? [23:02] this gives time to focus on bug fixing [23:02] ah, TestCases, didn't know about that one. will look into that [23:02] Yes. We work on the MAIN branch only. [23:03] good [23:03] ktwilight - if you have time after the meeting I can give you a quick intro how it works. it is simpler than it looks at first glance. [23:03] I need to do a couple of half-feature / half-bug things related to recent template and javascript refactoring [23:03] sure Lavr__, that'll be great :) [23:03] nothing large [23:03] and I plan to tackle TablePlugin [23:04] that needs to run at release [23:04] Yes. There are TWO issues related to tables that are hot. [23:04] The TablePlugin that does not show all grid lines in IE. [23:05] And the EditTablePlugin where the thead/tfoot broke the new Javascript move and delete feature. [23:05] yep, a lot of work ahead [23:06] too bad we have to edit table plugins [23:06] on terminating little kritters [23:06] 2, not to [23:06] yes [23:06] i hope the sponsor of crawford funds 100% feature set in his new plugin [23:06] so that his plugin can replace the current edittableplugin [23:07] it does not make sense to maintain two almost identical plugins :-( [23:07] Yes. That would be better for all. [23:07] ok, anything else on relase dates / coordination? [23:08] ---+ 4. Change "Proposed For" field [23:08] CDots proposal. [23:08] CDot: ping [23:08] I am OK. I just need two good English words for "next release" and "next next release". [23:09] i do not really see the point of changing the explicit names [23:10] forthcoming and future [23:10] i guess there is a point for patch releases [23:10] new features are not supposed to go into patch releases anyway [23:11] so may be add just one for patch releases, and keep explicit code names for minor and major? [23:11] Patch releases are for bug fixes only. [23:11] as you know, there is always a push to get a small feature into a patch [23:11] as long as this is done sensibly i do not see an issue [23:12] Patch releases has to be releases you can install on top of a running server and then you should need to update nothing afterwards. [23:12] that is the criteria, yes [23:12] Otherwise the admins will not upgrade and end up with bugs and security issues unresolved. [23:13] So if a proposal is so small that it can go into a patch release - then it only requires a bug item anyway [23:13] so, for example, if a small fix to a template needs to be done for the next twikidrawplugin release, i see not why it should not be done for a patch release [23:13] but isn't the question to make feature requests more aligned with Bugs? [23:14] each feature has one for more bug items, for tracking [23:14] that works well i think [23:15] the question is, do we want to provide a way to accept a feature proposal as a patch item? [23:15] i mean, provide support for it in the feature proposal form [23:15] ? [23:15] If we do that then we are back to the 4th level of hotfixes again. We have minor release for this purpose. [23:16] I meant Bugs has patch, minor and major [23:16] You used as an example a small fully compatible change to a template (one that does not brake normal customizations). That you can do on a simple Bugs item. [23:16] And that can go in a patch release. [23:17] But for example the refactoring Arthur has done the past days do not belong in a patch release. [23:17] ok, makes sense [23:18] what do other folks think about changing the "proposed for" field? [23:18] So I would like the feature proposal TWiki app to signal that this is for the real features or changes to current specs that need discussion and accept. [23:18] ok, it looks we are in agreement that "patch" does not belong into the feature tracker on twiki.org [23:19] No. And there is no need to see things as major or minor either. People mean next release or the one that follows if you know it cannot be made in time for next. [23:19] question now is, do we want to change the explicit "freetown" and "georgetown" to "next" and "next next"? [23:19] That is the practical horizon people can cope with [23:19] so we can call them 'minor' and 'major' [23:20] so they won't age [23:20] well, actually minor and major only happens to fit the current and next release [23:20] You cannot assume that a major comes after a minor. If next release is 5.0 it is major. If it is 4.3 it is minor. [23:20] maintenance release, regular release? (blech) [23:20] Right now our roadmap says 5.0 but we can change our minds. [23:20] launchpad.net has a way to do it, they call it essential, high, medium, low or something similar [23:21] that is what it should mean [23:21] ktwilight that sounds like a priority field. [23:21] i think that is related but not the same [23:21] * ktwilight checks again [23:21] priority and scheduled for are two different animals [23:22] oh yea... [23:22] my bad [23:22] they have no schedule field, they got a delivery field though [23:22] The intention of the ProposedFor is to signal if this is a proposal for the release we all work on now, or it is a proposal that will not be implemented until a later release - what that release may be - minor or major [23:22] time check: +80 min [23:23] i have a hard stop at +120 min [23:23] Arthurs "forthcoming release" and "future release" is the best proposal I have seen so far [23:24] it is doable, but i find explicit names more crisp [23:24] makes it also easier to run reports [23:24] e.g. "show me all features that are in freetown" [23:24] a bit misleading though, forthcoming and future makes no distinct difference unless you define 'em clearly [23:25] giving explicit names is better, but would mean more maintainance? [23:25] but means a bit more work at the time of a release when features are deferred (as is the case now) [23:25] Are we in reality more comfortable with just keeping what we have? [23:25] I do not see it as a big issue to manually update 20 proposals that did not make it. [23:25] i feel more comfortable if we keep it the way it is [23:25] same [23:26] because it is only the accepted and committed proposals that we are talking about [23:26] ye [23:26] yes [23:26] ok, seems we have an agreement [23:26] last chance: opinions? [23:26] OK. we keep what we have and after this meeting I change all the not yet implemented to Georgetown. [23:27] AI for me [23:27] sounds like a plan [23:27] ---+ 5. Need to increase the code quality [23:27] kenneth, the floor is your's [23:27] Well. it is hard to say these things without sounding like an old woman. [23:28] But we have had a very unstable MAIN branch lately and it has been difficult to develop on it because so many other things were broken. [23:28] So it is difficult to see if your own new code is working or buggy. [23:29] And to me the main reason is that we do not follow our own code of conduct as given in DeveloperResponsibilities. [23:29] A topic I did not write! [23:29] i think we all agree so far [23:29] So what I would like developers to consider in future is the bullet points I have stated. [23:30] The first - that all need to fix more bugs - is obvious. And the feature freeze should help getting that focus the next two months [23:30] "Features are not tested properly before they are checked in" is probably the one that is hard for some developers to accept. [23:31] But that is the way I see it. Most are too fast on the "svn commit" trigger button. [23:31] i think we can attack the issue in two phases: [23:31] 1. short term (for freetown release) - catch up on quakity, e.g. focus on big fixing [23:31] 2. improve process - toward's the and & after the release [23:31] Just a little more testing before commit. We are sometimes talking maybe 15 minutes - would make a huge difference [23:31] s/quakity/quality/ [23:32] We have plenty of quakity :-)))) [23:32] Oh blast I need a drink after that one. [23:33] http://images.google.com/images?q=quack+cartoon [23:34] i can hardly disagree with anything you've written. i would like to add that svn commits (unlike cvs) are atomic and multiple files can actually get checked in as one 'rev'. this means that all files need to be checked in together when a change is implemented. i've messed this up recently,t ho i'm usually pretty good about it. but really, that's a small problem. if the changes aren't done (not the i thought it was done, but [23:34] we found some bugs :( 'done' ) then long WIP projects should be worked on in their own feature branch. either a branch within twiki.org's own svn or by using svk externally (svk is quite easy to easy and makes updates happen in seconds rather than minutes) [23:34] (as an additional benefit) [23:35] The next point is similar. Many features seem to be committed in small chunks. I am not talking about a few commits over a few hours. I am talking about the situation where a feature takes 2-4 weeks and we have weeks of broken or half finished code on MAIN [23:35] but really it is all of our faults, too, because none of us have backed out any changes... [23:35] perhaps soon i can have a tinderbox and we can automatically reject commits which break the unit tests... [23:36] what's the purpose of svn? commit latest changes or commit stable changes? [23:36] i would differenciate: if an implementation is done in steps, and tested in steps, then i see no issue with multiple checkins [23:36] No. If each step is a working step then there is no issue. [23:36] the criterium really is: *DONT BREAK THE BUILD* :) [23:36] snap [23:37] what hurts is checkings of preliminary code implementations that are not tested [23:38] and it is good for developers to have internal version control while they're developing the feature. again, i will advocate more use of svn branches while working on longer (for some value of long) project/features [23:38] for example i have seen many smaller checkins by arthur; they are usually quite stable [23:38] I understand that sometimes some developers may want half way review of their large code changes. Then we should use the branching more. Create a branch instead (all can do that) and run on that for a week or two and merge the changes back to MAIN and abbandon the branch [23:38] that sounds good too [23:39] The next time is - checking in code without running the unit tests and the TestWeb tests. [23:40] It is really hard to learn to write unit tests. And it is not reasonable to ask newbies to write them. [23:40] But running them and see that nothing broke is easy. And the test web is even easier. [23:40] but they're so rarely not broken [23:40] (so it seems :( ) [23:41] They have been broken constantly the past two months [23:41] Currently 10 unit tests fail in perl 5.8 and 18 in 5.6. And in the test web half the test cases fail. [23:42] The main problem with this is when you try to be a good guy and run the tests before you checkin your new code you are met with a massive scrolling of bugs that you cannot see if YOUR changes did any additional harm [23:43] And then more and more use that as an excuse not to run them. [23:44] You also need to know that a successful run of unit tests gives errors in the output because some of the tests are checking that errors are caught (and some are just badly written). [23:44] But a successful test run says zero failures and zero errors at the end. [23:44] this also points to getting the tinderbox set back up and up-to-date with the current codebase [23:44] yes, we must first fix the bugs that make the tests fail [23:45] In 90% of the cases the code fixes are quite OK. It is the test cases that need to be updated. [23:45] which i'm willing to setup the tinderbox again... [23:46] And fixing an existing case is not hard. You just open the test case that fails - find the test that fails and alter the "expected result". [23:46] esp. with TWiki:Plugins.TWikiInstallerContrib which can do completely automated scripted installs [23:46] (in addition to the 'gui' ( see http://twiki.biohack.net/twiki/twiki-install ) ) [23:47] let me state a personal note: imho twiki was very stable up to cairo, we did not have to release emergency releases shortly after a production release, except to address security alerts. the problem with the old process was that it did not scale due to the core team bottleneck (the core team was the quality control system). we therefore changed the process to be more open, inviting more people. the hope was that unit tests would sol [23:47] step one is to make unit tests pass [23:47] step two is to increase the unit tests, e.g. get more coverage [23:48] step three is _not_ to rely on unit tests when making code changes, e.g. manually test all areas that are likey affected by your code changes [23:49] step four is to program more defensively, always looking out for potential bugs [23:49] i think that will help in improving the quality of twiki [23:49] Yes. The step 3 seems to be forgotten often. Example the thead/tfoot feature broke the EditTablePlugin. All you need to do is edit an edit-table and look. I would for sure have run such a simple test before committing code that affects tables. [23:51] When I committed the new iso date code I had tested for 3 hours all sorts of features that show dates, I ran the unit tests, and the test web tests before I checked it in. [23:51] There may be a bug lurking but at least I know I did not break the most basic features. [23:52] What I have on my test server is a real web from my Motion site and another web of a lot of test topics I have done for different purposes. And I play with the topics to see if something is broken. [23:53] I can recommend adding some real webs to the data directory of an SVN checkout. They do not get overwritten by pseudo-install. [23:53] Many plugins lack a testing topic [23:54] hm, a testing topic would be really nice [23:54] I also recommend having TWO SVN checkouts. One clean one for unit tests which fails if you add more users and topics in some webs. I have an SVN checkout I only use for unit tests. [23:54] good point kenneth on extra webs in svn checkout [23:54] the tableplugintest topic is very useful, I suggest to make one for edittable as well [23:54] since my experience, i never thought svn would be good for production use [23:55] you need to have all situations in one topic to test their use [23:55] or a XxxPluginExamples topic [23:56] as done in the chartplugin [23:56] would be difficult for some plugins i think [23:56] i did the same for the spreadsheetplugin, but not in a clean way so that it can be shared :-/ [23:57] time check: +115 min [23:57] kenneth, i think we are aware of the issues [23:57] time to execute on it [23:57] This is the small script I run to manually update my MAIN svn. [23:57] #!/bin/sh [23:57] cd /var/www/MAIN [23:57] rm `find . -type l -print` [23:57] svn update [23:57] perl pseudo-install.pl -link default [23:57] perl pseudo-install.pl -link BuildContrib [23:57] perl pseudo-install.pl -link TestFixturePlugin [23:57] cd .. [23:57] chown -R apache:apache MAIN [23:58] Note the perl pseudo-install.pl -link TestFixturePlugin. This is needed to run the TestWeb test pages. [23:58] The BuildContrib is needed to build plugins or TWiki. [23:59] useful info [23:59] Note that the pseudo-install adds links but does not remove them. So you can run it several times to add additional plugins to the test setup. [23:59] documented? Session Time: Tue May 29 00:00:00 2007 [00:00] I believe the default is now to link files [00:00] Yes. It is in the test case tutorial. And in other topics. But maybe I should add this script so people can find it. [00:00] otherwise write -copy [00:00] Oh. so the -link is not needed anymore? [00:00] i have to sign off now [00:00] please continue as needed [00:00] good meeting today [00:00] thanks all! [00:01] hope we will get more participants next time [00:01] Thanks Peter. I have no further. [00:01] ah, ok [00:01] so then lets close the meeting? [00:01] yes please [00:01] Lavr__, can we go through it another day? [00:01] fine [00:01] Sure. [00:01]