Suppress usage of "ls |" in Search.pm
Summary: The latest
TWikiAlphaRelease does not use piped commands for search anymore. Topic name search is now done natively in Perl instead of an external
ls *.txt | grep ...
command. TWiki.org's TWiki is updated as well.
--
PeterThoeny - 01 Nov 2003
The search is using extern commands for ls and grep. The problem is that running TWiki on Win32 with PERL "-wT" cause pipe fail.
- Is using such an external command for dir list needed here ?
[lib/TWiki/Search.pm] [Line 214] : $cmd = "$TWiki::lsCmd %FILES% | %GREP% %SWITCHES% -- $TWiki::cmdQuote%TOKEN%$TWiki::cmdQuote";
- My not to use readdir like in Store.pm ?
In fact, the problem is the pipe, I agree with the external
grep but TOPIC list could be get from readdir, like in other part of TWiki :
opendir( DIR, "$sDir" );
my @TopicList = grep /\.txt$/, readdir DIR;
closedir DIR;
Ok, no so easy to change Search.pm, but, if you agree with me, I could try to change it. Note that Impact would be restricted to topic list in current WEB.
--
PatrickNomblot - 17 Jun 2003
Getting rid of lsCmd is the right approach, it makes the code cleaner. Only Search.pm depends on it. If you produce a patch we will take it into the core code. It is a little complex since the grep switches need to be done conditionally for external grep and Perl internal grep.
This fix is needed for
TWikiMiniWebServer
--
PeterThoeny - 17 Jun 2003
I attached in this Topic the modified
Search.pm to suppress use of "ls" and "|".
I tried to do it to minimize the impact on grep in TOPIC (no change in fact) and did it so to keep the high level of code quality (I hope). Thanks to check if this code is good for you, I propose to put it in next TWiki release when ok and suppress ls.cmd in TWiki.cfg.
--
PatrickNomblot - 20 Jun 2003
I have just uploaded the last Search.pm : with this last version, TWiki can also search in
rcs files too. Simply say
scope="rcs" (rather than
scope="text") and grep will look also in rcs fileswhich makes the
MyEditedTopics working fine !
You will notice that this choice of
scope="rcs" keep Search.pm compatible !
: Note that Win32
grep.exe obtained from
http://unxutils.sourceforge.net will search into
*.txt* rather that what we tell it to do :
*.txt, so it allways grep in txt,v files, whichs is wrong in most cases !
I hope the TWiki team will appreciate my changes on Search.pm, I'have tried to made them in the respect of TWiki.
--
PatrickNomblot - 24 Jun 2003
I have not looked into details, but it looks like
@tmpList = grep /.*$token.*\.txt$/, readdir DIR;
does not allow "pattern$" or "^pattern" regex searches. Better to remove the extension (with map) before grep'ing the pattern.
On
$theScope eq "rcs"
, I would prefer a term that is not implementation specific.
RCS could be replaced by CVS or a versioned database backend. Not sure what term would be intuitive, "history", "rev", or?
--
PeterThoeny - 24 Jun 2003
- Thanks
- Works on Unix and saves a few forks. Plus hopefully resolves the issues with my Windows install, where everything works, except forking ls...
- Yes
-
^pattern
for names does work ... regardless of the regular expression checkbox!?
- No
-
pattern$
does not work. Never did, as mentioned in DynamicEditTopicTemplate. Try pattern.txt$
- Yes
-
history
instead of rcs
is a good idea, because, you know, erm ... CoolURIsDontChange
--
PeterKlausner - 24 Jun 2003
topic="rcs" was not a good idea ... sorry
! I have changed it to
topic="history" Thank you for the idea !
Search in TOPIC seems to have the same behaviour than original one. Anyway, I like the Peter's idea, the last Search.pm uses
map to remove
.txt from topic names before
grep. I think that it should be possible to have
pattern$ working know (but it still does not).
--
PatrickNomblot - 25 Jun 2003
I think it would be great to put this
little evolution of Search.pm in
CairoRelease together with
MyEditedTopics.
It runs very well on our system since 24 June now, whith no problems. Note I think that is also important to keep participation of people outside
CoreTeam
Many thank's !
--
PatrickNomblot - 02 Jul 2003
I've changed the status of this to
FeatureHack - I don't mean that it is one but we still don't have a
FeaturePatch category, without which your hard work is likely to get lost. I don't want you to get demotivated!
PleaseCreateNewCategories holds the discussion for the introduction of this process enhancer.
--
MartinCleaver - 16 Jul 2003
I am now working on this feature among
ExcludeWebTopicsFromSearch,
SearchTopicNameAndTopicText,
ArgumentListIsTooLongForSearch,
InlineSearchArgListTooLong.
--
PeterThoeny - 29 Oct 2003
This is now implemented, among with
SearchTopicNameAndTopicText,
ExcludeWebTopicsFromSearch,
ArgumentListIsTooLongForSearch,
InlineSearchArgListTooLong,
SearchWebHasTooManyParameters,
SiteMapIsSlow. A first step of an overhaul of the
lib/TWiki/Search.pm
module is also done. Therefore I did not apply
PatrickNomblot's patch.
--
PeterThoeny - 01 Nov 2003