Feedback on ReStructuredTextPlugin

Well, I'm new at this. Basically, this is a plugin that leverages some internally developed tools at Freescale (was: Motorola SPS) that translate ReStructured Text to HTML. The internal tool is soon to be released to the PD but the plugin is about to be released internally - so, I thought I would start off on the right foot and create a space here as well. More to come . . .

-- SteveRJones - 19 May 2005

Steve, I am trying to understand what you are trying to accomplish. If I get this right, reStructuredText is an alternative approach to TWiki to render simple text as a web page. You are then pulling this into TWiki. Would it not be easier to use TWiki markup in the first place?

What are the advantages of using reStructuredText over TML?

-- ThomasWeigert - 19 May 2005

ReStructured Text is one of many simple markup languages, apparently originating out of the python community and used by Zope and ZWiki.

Would it  not be easier to use TWiki markup in the first place?

I would agree with you given that the content originates within within TWiki. I was approached, however, with the goal to take quite a bit of software documentation already existing in ReST and making it fit into TWiki. The particular programmer had the idea of simply cutting/pasting the ReST text into the HTML edit field, then he thought he could simply write translated ReST source into the correct TWiki subdir and all would be ok. Not. Then he thought about writing a TML parser and translating from ReST to TML and back again . . . ack.

So, I gave it some thought and knowing that he had already made use of a ReST 2 HTML parser I thought this would be a simple instance of, say, the BeautifierPlugin. After all, the intent was to get the existing mounds of documents into the Twiki environment with a minimum of effort - hence the plugin.

What are the advantages of using reStructuredText over TML?

There are apparently advantages of ReST over TML in terms of nesting formatting, tables look like tables (a little too complex for me, though), etc. but I imagine that once the docs are in TWiki the maintainers will slowly migrate to TML where appropriate. Nevertheless, if the outside development community that works with our internal staff expects ReST formatted documents then we have them AND they fit inside TWiki -- best of both worlds.

Plus, I did a search on twiki.org and found a few references to "wouldn't it be nice if TWiki supported ReST" so I thought "a neat little opportunity."

Does this help?

-- SteveRJones - 19 May 2005

That's great Steve. Thanks for sharing this.

I can equally imagine that TML should be turned off for efficiency for documents formatted in RST. I am sure there is no way to accomplish this in Cairo (and probably not in Dakar).

-- MartinCleaver - 19 May 2005

Perhaps so, but the plugin calls the beforerender handler and returns html -- which I assume the TML renderer simply ignores and I would hope would therefore not add to the overhead. But then again, I'm not a core code developer so what do I know!?

Thanks!

-- SteveRJones - 19 May 2005

A possible alternative approach to "take quite a bit of software documentation already existing in ReST and making it fit into TWiki" would be to create a ReStructuredTextToTWikiAddOn along similar lines as GenericWikiToTWikiAddOn. May or may not meet all your needs but thought it worth throwing into discussion since it seems this would be much easier to implement than what you're discussing here.

-- LynnwoodBrown - 19 May 2005

Yep, thought about that too and threw that at the programmer. Turns out that the key to the issue is in the fact that alot of the documents are destined for consumption outside of our company and the standard format is ReST. So, this seemed to be the fastest way and in reality it took about 30 minutes to through together the plugin -- the main developer ahd already developed the backend translator.

Good thought - - maybe this will be next!!

-- SteveRJones - 20 May 2005

So much going on, I missed this Plugin until now. Thanks Steve for sharing it, I am sure some folks outside your organization will find it useful.

I changed the SHORTDESCRIPTION a bit (describe "what", not "how"). You can take that (or something similar) into the next release if you wish.

How about measuring and documenting the PluginBenchmarks?

ATTENTION: The TRIP = /usr/bin/trip setting is a big security risk since that can be used to execute anything on the server! Better to put that setting into the Plugin code, or a config file such as twiki/lib/TWiki/Plugins/ReStructuredTextPlugin.cfg. As an immediate measure, public installations should restrict the Plugin topic change to the TWikiAdminGroup. Google does not list any public installations of this Plugin so we should be OK.

-- PeterThoeny - 09 Jun 2005

Peter, we're looking at changing the external option to go against a separate cfg file.

On another note, we were caught offguard with teh fact that TWiki converts spaces to tabs, then back to spaces again. Once discovered we handled it, but can you tell me why TWiki does this?

Thanks!

-- SteveRJones - 10 Jun 2005

Legacy reasons. The original JosWiki had tabs, we kept that at the file level, but hid it from the users.

-- PeterThoeny - 10 Jun 2005

So, when a topic is stored it is stored with tab chars rather than spaces, correct? This seems to be counter to what we are seeing -- a plugin receives the %TEXT% data the tabs are in the data stream, indicating that spaces have already been converted. When exactly are tabs converted - we started using the commonTagsHandler callback because we thought that another part of the rendering process was munging the spaces.

-- SteveRJones - 10 Jun 2005

I believe TWiki only converts tabs to spaces at the end of the rendering process, e.g. data still contains tabs at the time commonTagsHandler is called.

-- PeterThoeny - 12 Jun 2005

Ah, then when is the best time to invoke the plugin? I ask because I need to make sure that nothing modifies the ReST format that is between the %RESTSTART%   %RESTEND% tags. trip has a directive to expand tabs to Xnumber spaces but I would rather not have to rely on this if possible. But, if it can't be helped . . .

BTW, working on the benchmarking.

-- SteveRJones - 12 Jun 2005

FYI, IIRC, the tab-to-spaces process has been eliminated in DakarRelease.

-- MartinCleaver - 13 Jun 2005

And StepByStepRenderingOrder shows that it is GetRenderedVersion that implements TML. Just override GetRenderedVersion with REST; conceptually you need another renderer.

-- MartinCleaver - 13 Jun 2005

Any news on when this will be available for download? I've been keenly anticipating the Perl ReST parser from Motorola for (IIRC) a year or more now

-- AdamSpiers - 20 Jul 2005

Aackk, I didn't know anyone was actually wanting this! We've been working out the bugs related to the store-a-tab/translate-to-spaces issue and this is fixed today. The rest was making sure that warning messages due to running in taint mode were managed and making sure it ran right under Perl 5.6.1. The rest is packaging and making sure the installation is done right. Oh, and Peter wanted me to benchmark it, which I haven't done yet. Probably in a couple of weeks? Oh, and this isn't from Motorola! We're Freescalers now!

-- SteveRJones - 21 Jul 2005

Beautiful - I'm away for a couple of weeks from Sunday anyway

I hadn't twigged on the Freescale breakaway. I originally had an email exchange with Mark Nodine (consulting my archives, that started in November 2003!) - is he with Freescale too now?

-- AdamSpiers - 21 Jul 2005

re: why ReST - because there are people who like it and are using it elsewwhere. And those same people may dislike TML.

The Company I work for has begun to use ReST for design specs. I might be abe to win more engineers over to using TWiki for ANYthing if they could use ReST there.

-- VickiBrown - 04 Oct 2005

but... where is the .zip file?

-- VickiBrown - 04 Oct 2005

Yes, where is the zip package? Please attach it to the ReStructuredTextPlugin topic.

-- PeterThoeny - 10 Feb 2007

Thanks SteveRJones for attaching the zip file!

-- PeterThoeny - 10 Apr 2007