Question

Following on from HowManyTopicsAreTooMany it appears that performance degrades from around 10k topics in a web.

In a web with several thousand users the Main web will soon grow to the above limit with just the users topics. We are seeing other webs approaching 10k files. Our file system can cope with between 20-64k files depending on the length of filenames.

I would be interested to learn how other large sites have configured their twiki sites in order to cope.

Sub webs would appear to be a help but the Main user web and the user topics is my first concern

Environment

-- PeterJones - 21 Jan 2008

Answer

If you answer a question - or have a question you asked answered by someone - please remember to edit the page and set the status to answered. The status is in a drop-down list below the edit box.

You may want to use the brute force approach:

• use more memory, that linux will use to cache files
• tweak your filesystem for optimal performance on big dirs, see: http://ygingras.net/b/2007/12/too-many-files%3A-reiser-fs-vs-hashed-paths

-- ColasNahaboo - 22 Jan 2008

in 4.0.3 onwards, you should be able to dispense with the UserTopics? and define a user mapper that can more efficiently convert between login and wikiname. (The JoomlaUsersContrib was faster than the TWikiUsers based one even a year ago) Also, DatabaseStore could well be the longer term answer to your issues?

As for real scalibility, I've been surprised over the years that there has been no discussion of real application server setups - where TWiki is running on several servers, behind caching proxies, with edits going to a master server.

-- SvenDowideit - 23 Jan 2008

Seems to be answered.

-- PeterThoeny - 02 Mar 2008