create new tag
, view all tags

Usability/Speed Improvement of "Topic not Found Page"

(was: Performance issue: Jump Box is slow in large webs)


Looking for similar topics in a large web (>10K topics) via Jump box is slow.

Description and Documentation

When you type the name of a non-existing topic in the Jump box you end up in the WebTopicViewTemplate topic that contains three sections for three use cases:

  1. List of similar topics (with one embedded SEARCH on topic names)
  2. Form to search for content (no embedded SEARCH)
  3. Form to create a new topic (with two SEARCHes on topic names)

Use case 1 is for users used to use the jump box to find similar topics. Use case 2 and 3 are not used that often.

The WebTopicViewTemplate topic is slow because of use case 3: The form to create a new topic lists all existing topics in order to select the parent topic. This is slow if the web contains 10K or more topics (slow to get the list, slow to download the list and slow to render the list.)

Example: Enter Bigweb.foobar in a big web that contains 50K topics:

Before/after change Download starts Download complete
Before: With form for use case 3 35 sec 70 sec
After: With link for use case 3 14 sec 15 sec

Impact and Available Solutions

WhatDoesItAffect: Rendering, Performance


Replace the form to create a new topic with a link to the topic creator topic. That requires an extra click to create a topic, a small price to pay for a rare use case. Replacing the form eliminates two embedded SEARCHes, which speeds up the "jump to similar topic" in large webs considerably.

-- Contributors: PeterThoeny


This is a small and simpe change. No-brainer for TWiki 4.1? See WhatIsIn04x01.

-- PeterThoeny - 18 Oct 2006

Yes. I agree this is a nobrainer. No compatibility issue with Twiki apps. No API change. No TML spec change. Should not affect normal tailorings.

Out would say it falls below the bar for what is required to go through WhatIsIn04x01. I would go ahead and implement it. It sounds like a good idea.

The jump box is probably more often used to jump to a topic you think you know what is called than to create new topics. You probably jump to existing 100 or 1000 times more often than creating. This is a good change in the right direction that helps corporate sites to grow and scale up.

-- KennethLavrsen - 18 Oct 2006

Actually, thinking this over, I would like to get feedback on the "how to". We have choices (from easy to more involved): Replace existing "create new topic form" with:

  1. link to create new topic
  2. simple form that has no parent selector and link to create new topic with parent selector
  3. same form elements, but delay filling in the select values of the parent list using AJAX technology


-- PeterThoeny - 18 Oct 2006

I vote for 1. But at the same time I plead for strong refactoring. This is my motivation:

Looking at the information the topic provides, it is too much oriented towards implementation details. For instance if I misspel the url in the browser bar, I have not used the "Jump" feature and I even do not have to know that that feature is.

The use cases actually are:

  1. Someone tried to go to a topic but failed
    • Either by mistyping, or because a topic has been renamed or deleted.
  2. Someone found a way to create a topic by entering a name in the url bar or in the Jump box.
    • This was the old way, before there was a Create topic page.

Now if the topic does not exist, we can provide clues to whatever a person was looking for.

  • Provide a spelling hint
  • Provide alternative spellings (and this could go further than just look for the characters in topic names - think about acronyms, synonyms, etc.)

The current page does some of this. But it also does too much. It is actually not that important how the user got there.
Just give the feedback:

  • "This topic does not exist."
    • Show currently entered name, possibly with go box, show spelling hint.
  • "Do you wish to create this topic?"
    • Link to Create new topic page.
  • "Or are you looking for something else?"
    • Show similar topics

The current links are unwieldy or need more info. What would "List all similar topic in this web" do? What is similar? Isn't this the list just above the link?

The current list is ok, but it needs a clear header. Then at the bottom provide the link "Expand search to all webs" (which would do a search as the current "Search for similar topics in all public webs").

Even the search box "If you intend to do a full text search" might go. I suspect not many people will start a search because it does not fit the use cases. And the lonesome user that does want to do a search already is familar with the search box that is always on the page.

  • And if we need to keep the search, at least we get rid of the line "Use the Search link on top of this page"

-- ArthurClemens - 18 Oct 2006

Actually the biggest and most powerful use case for those who know is:

  1. Someone wants to navigate to a topic in the current web or another web and remembers just part of the topic name (or is too lazy to type the full topic name in proper case)

With Bugs:Item3016 enhancement I addressed this use case. If you are aware of this feature you can navigate to almost any topic in a 100K wiki within seconds. Very useful. So this use case should be promoted first, hence it is first in the list (actually almost, it is the second bullet in the first section.)

To summarize, here are the prioritized use cases:

  1. Someone wants to navigate to a topic
    • I thought we were talking about the "not found" page. If someone succeeds, he directly goes to the topic. If he fails, we get usecase number 2. So this one does not exist. -- ArthurClemens - 19 Oct 2006
    • For me any many other this feature sure exists smile and I invite you and everybody else to try it out! There is no "not found" page in TWiki if you use this feature as a navigation aid. We need to publicize this feature since it is so useful/powerful in a large wiki. -- PTh
  2. Someone tried to go to a topic but failed
  3. Someone found a way to create a topic

-- PeterThoeny - 18 Oct 2006

Here is what is behind "List all similar topic in this web": The similar topics list is limited to 16 topics; this is done for speed and to allow room for the other use cases. The "List all similar topic in this web" shows the same search, but not limited to 16 topics. The proper way is to show a "List more similar topics in current web" conditionally, e.g. only if there are more than 16 entries. For performance I do not want to do an extra search just to test that. Any creative idea how that can be done?

  • It would be possible with javascript, to count the number of entries. -- ArthurClemens - 19 Oct 2006

-- PeterThoeny - 18 Oct 2006

From an IRC talk with Michael Daum.

[20:25] <Lavr> PeterThoeny I have a question regarding Bugs:Item3016
[20:25] <twikibot> http://develop.twiki.org/~twiki4/cgi-bin/view/Bugs/Item3016
[20:26] <Lavr> Look at this. http://merlin.lavrsen.dk/twiki/bin/view/Motion/WebHome?topic=BugReport
[20:26] <Lavr> Below the 16 proposed it says: This topic has moved to Motion.BugReport2005x02x28x170109.
[20:26] <Lavr> That must be a bug
[20:28] <Lavr> Or if it is not then I do not understand why it shows up there.
[20:28] <MichaelDaum> how are "similar" topics found?
[20:29] <MichaelDaum> may be too naively defined
[20:30] <Lavr> What is strange is that it is shown like it is. It finds similar OK. But what is that funny feature below? What does it try to tell me?
[20:30] * Flenser_ has quit IRC (Read error: 110 (Connection timed out))
[20:31] <MichaelDaum> in _any_ case this "topic not found" page is too resource hungry
[20:31] <MichaelDaum> while converting mediawiki data to twiki I DoSed my server by calling a non-existing page
[20:32] <Lavr> There is a proposal open from Peter to replace the create section at the bottom by a link to the topic creator. What really takes time is making the list of possible parents.
[20:33] <Lavr> Using the Jump feature to find a topic you know starts with BugReport is quite nice and such a search must be rather quick.
[20:34] <MichaelDaum> I prefer a simple text "Warning: this topic does not exist" end of story.
[20:34] <MichaelDaum> If I am looking for a fuzzy search then I would go to WebSearchAdvanced
[20:35] <MichaelDaum> or a special WebSearchFuzzy
[20:40] <Lavr> Well. The Jump field could jump to the advanced page. The plain URL could lead to the plain standard error.
[20:41] <MichaelDaum> for example
[20:41] <MichaelDaum> much better
[20:41] <MichaelDaum> I get lots of load on my servers from google bots crawling for old topics
[20:42] <MichaelDaum> took some time til I found the reason :(
[20:42] <Lavr> Yes. That does add a lot of load.
[20:43] <MichaelDaum> replaced the page with a plain error message and things were fine again 

Michael raises a real issue. The bots on public sites trigger a lot of load because of this nice feature.

How can we use a simple page when you arrive directly with a non existing URL. And show the advanced page when you use the Jump box.

-- KennethLavrsen - 19 Oct 2006

Can we list the decisions to be made?

  1. Remove the "Create topic" form and add a link to WebTopicCreator instead
  2. Remove the search box and add a link to WebSearchAdvanced
  3. Either make the list of suggested alternatives less performance heavy or replace it by a link.

-- ArthurClemens - 08 Nov 2006

I agree with the first two.

The suggested alternatives is a very useful feature to navigate a 50K topic wiki quickly, I feel pretty strongly to keep the list. It is not performance heavy since the search is limited to 16 topics.

-- PeterThoeny - 09 Nov 2006

I would like to implement this for 4.1. Any other things before voting on our release meeting next Monday?

-- ArthurClemens - 09 Nov 2006

I did a first cut on improved speed and usability, based on Arthur's feedback, and after some iterations. Bugs:Item3017, SVN 11956 and 11957:

  • Jump to topic form, on top.
  • List up to 16 similar topics, in reverse order.
  • Show link to "list more similar topics" (if any).
  • Show link to moved topic, in red (as before).
  • Show link to create new topic (removed form and two SEARCHes).

Possible further enhancement: Do a SEARCH for similar topics only if URL param topic is present. This will speed up the "topic not found" error even more, unless used via jump box. Note that the SEARCH for moved topic needs to be done in any case (for example if a user clicks on a URL in an e-mail and ends up in a topic not found message.)

Please give it a try, and lets discuss at release meeting (if anyone feels like.)

-- PeterThoeny - 12 Nov 2006

SVN 11958 is now suppressing the SEARCH for similar topics unless the Jump box is used.

-- PeterThoeny - 12 Nov 2006

Edit | Attach | Watch | Print version | History: r12 < r11 < r10 < r9 < r8 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r12 - 2006-11-12 - PeterThoeny
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.