Feedback on OpenOffice2TWikiAddOn

Add any ideas, bugfixes, criticism or whatever related to OpenOffice2TWikiAddOn the OpenOffice to TWiki converter add-on.

To start things off, I thought I would add some timing examples:

Note that size does not matter. Most of the size comes from images, which are extracted and saved quickly. Most of the processing time however, comes from doing the XSL transformations. There has to be a better way

-- WadeTurland - 06 Aug 2005

Fantastic idea Wade. I look forward to trying it.

> I waited for ages and eventually got a browser timeout. What happened?

I wonder whether you can fork, process in the background, and email the user when it has finished.

-- MartinCleaver - 06 Aug 2005

Thanks Wade for sharing, this is a very useful add-on for those who are using the OpenOffice family of products

Small feedback:

• Martin has a good idea on background process. This could be done with a lower priority so that the server does not suffer under the load.
• This is currently an add-on. The doc topic has many references to Plugin; replace those with add-on
• Since this is a server sice thingy you could consider converting this into a Plugin. The Plugin's init function could listen to a oo2twiki=convert (or the like; chose an unlikely name) URL parameter and fire the converter action if the correct parameter is supplied. This can be done in the skin with a https://twiki.org/cgi-bin/view/Plugins/OpenOffice2TWikiAddOnDev?oo2twiki=convert link.

-- PeterThoeny - 06 Aug 2005

Good idea, but how do you email TWikiGuest?

-- WadeTurland - 06 Aug 2005

You don't. Just mail registered people.

-- MartinCleaver - 06 Aug 2005

It would be great if this could be called from WysiwygPlugin at some point (just like inserting an image).

..And if this could process Word docs on the fly

-- JosMaccabiani - 06 Aug 2005

I think I found the better way. Initial tests with OpenOffice::OODoc perl modules look very impressive. The same doc that took 7:12 using XSLT takes a few seconds to convert to pod (perldoc) with the perl modules - and it runs in pure perl.

It means rewriting almost the entire addon, though.

WadeTurland cues the self-motivation tapes

-- WadeTurland - 19 Sep 2005

-- ChristianSurlykke - 11 Sep 2006

Sorry for the empty comment above. I'm definitely a newbie. Tried to install the addon. When I try bin/convert/Sandbox/WebHome I get:

Undefined subroutine &TWiki::setupRegexes called at /opt/twiki/lib/TWiki/Contrib/OO2TWiki.pm line 42. Compilation failed in require at /opt/twiki/bin/convert line 37. BEGIN failed--compilation aborted at /opt/twiki/bin/convert line 37.

Any hints would be much appreciated.

-- ChristianSurlykke - 11 Sep 2006

Regarding the PerlDoc option, did anyone look at the PerlDocPlugin? It has a conversion function...

-- BryceSchober - 11 Sep 2006

I have the exact same problem as Christian. One of the instructions is to add the variable $xsltprocCmd = /path/to/xsltproc; to your lib/TWiki.cfg. When I do this TWiki then trows a an Internal Server Error. If I add a line that looks like the following TWiki works:

$xsltprocCmd = "/usr/bin/xsltproc" . ' %CS{|-i}% %DET{|-l}% -H -- %TOKEN|U% %FILES|F%';

Is that how the variable is supposed to look? Regardless, I still get the undefined subroutine error.

-- GordonTerrell - 12 Sep 2006

I am getting the same error as well. I am running on TWiki 4.0.4.

-- RickMach - 21 Sep 2006

I added a SHORTDESCRIPTION to the "Add-On Info" section so that this add-on is represented properly in the AddOnPackage topic and query topics. Please feel free to take this into the next release.

-- PeterThoeny - 04 Nov 2006

This really doesn't work with TWiki 4.x does it? Even removing setupRexexes (the comment on the line says "why is this necessary" - and it isn't for 4.x) just causes another failure calling TWiki::UI::isMirror

Getting past that causes "Can't locate object method "readTopicRaw" via package "Sandbox" " but it refers to the Store.pm

I'm a bit confused. Shame it doesn't work anymore

-- ChrisHogan - 05 Jan 2007

In OpenOffice2TWikiAddOn, the "tested" form field states just the Sep 2004 version, so it needs to be updated for TWiki 4.0.

-- PeterThoeny - 06 Jan 2007

Changing the call from TWiki::Store::readTopicText to TWiki::Func::readTopicText gets things just a wee bit further:

Software error:

Can't use string ("

<!DOCTYPE html PUBLIC "-//W3C") as a HASH ref while "strict refs" in use at /home/user/public_html/twiki/lib/TWiki.pm line 2586.

I'm going to keep poking at it...I have an 800 page document I'd like to TWikify. (Of course, I may give up and manually process this one document using the style sheets...but then images are a problem, I think.)

-- JoeCooper - 20 Mar 2007

Well, i've updated this thing to twiki 4.x. I've just emailed the author asking for permission to change the add-on topic and upload my updated version (which still needs some polish and serious testing).

-- KoenMartens - 05 May 2007

Hmm, author seems to have misconfigured his email server (false rejects based on senderid). Anyway, since it has been 2005 that this add-on was last tended to i'll just upload the lot tomorrow.

-- KoenMartens - 05 May 2007

Yes, please do so. This plugin is marked as PleaseFeelFreeToModify.

-- PeterThoeny - 16 May 2007

Is it "uploaded"? Where can we find it? Thanks.

-- JayBee - 11 Jul 2007

There is still just the old version from 2005 attached to the OpenOffice2TWikiAddOn topic, and the addin is not developed in SVN. This add-on needs some TLC.

-- PeterThoeny - 14 Jul 2007

Sorry for the delay (lots of earning-bread-and-butter work came in between), i finally uploaded my updates. Note, they can be cleaned up a bit, but i really can't seem to find the time to really sit down for it.

Instructions as mentioned in the topic still apply..

-- KoenMartens - 08 Aug 2007

Hi, need help here. I get an empty page after uploading test.odt. My env is under Windows. I install those two file for XSLT libxml2-2.6.28.win32.zip libxslt-1.1.19.win32.zip from http://www.zlatkovic.com/pub/libxml/

can anyone give me some hint?

-- WenYuanChi - 14 Aug 2007

I have never tested this under windows. My best bet is that it has something to do with file paths. Did you check the logfiles (apache log and data/warn*.txt)?? Perhaps they shed some light on the problem.

If someone with twiki on windows could test this, that would be great. I have no windows machines.

-- KoenMartens - 21 Aug 2007

Wooh! I can't believe somebody has finally started using it

I did try testing under Windows but could never get it to work. I think it's something to do with the way perl handles child processes under Win32, and more specifically under Cygwin. The interface is the same as that used on GenPDFAddOn though, so maybe it will work one day.

I never tested under Dakar or later but I can take a look now to see where it's failing.

Those motivation tapes never work...

-- WadeTurland - 27 Aug 2007

Btw, i found out there was a problem with the encoding. Special characters (eg charachters with umlauts) would not get translated properly. To solve this, change in the xslt files the line:

<xsl:output method="html"/>

to:

<xsl:output method="html" encoding="iso-8859-15"/>

I'll upload this change some time, once Wade's motivational tapes arrive by mail.

-- KoenMartens - 28 Aug 2007

We are after a way of batch converting our MS Word documents into TWiki. After testing both OpenOffice2TWikiAddOn and MsWordToTWikiMLAddOn, the former seems slightly better in terms of conversion quality, but more importantly, seems to have better promise of growing into a batch process (images included, new topic created).

There's certainly no difficulty in converting batch of word documents into the Open Office format. What would it take to hack the convert script to handle a batch of documents or even directories?

See also the Import MS Word topic discussion.

-- EyalTeutsch - 20 Sep 2007

For some reason, in all documents I convert (including simple test documents), only a single image gets attached (all other images though linked, are not attached to the newly created topic).

I have attached a test document, image_conv_prob.odt, with three twiki graphic images - only the third gets (arbitrary) attached to the document.

Has anyone experienced similar problems?

-- EyalTeutsch - 11 Oct 2007

Twiki 4.1.2, Openoffice2TwikiAddon Every time I run Convert, the origionating page is deleted, and the contents replaced with a single link to the converted document. Nothing in the warn.txt Ideas?

-- BenSelinger - 12 Oct 2007

Great work KoenMartens! I thought the original work is frosen, I even started to work on an xslt for odt, but then I saw your contribution. Thanks.

Two small issues:

1. installation: I got Archive::Zip is missing. Maybe it should be added to installation instructions.
2. Equations: I'd like to (start and) add support for importing equation, using MathModePlugin. I guess I'd need to extract code from odf to Latex xslt converter. Any ideas?

One big issue:

It would be great to have the full editing cycle with OpenOffice, i.e. browsing TWiki done with your favored browser, and editing a topic is done with your favored ODF editor.

For this, however, a TWiki2odf has to be implemented. Any ideas where to start here?

-- YuvalAviel - 17 Oct 2007

Even odt2wiki doesn't deal with equations

But I found this useful converter: http://www.hj-gym.dk/~hj/writer2latex

-- YuvalAviel - 18 Oct 2007

Experienced the same problem reported by Eyal above.

Switched on debug for the Add-On and looked at the errors in the various logs.

It turns out that the extraction of the images was failing for the second and subsequent images because the filehandle used was read-only.

I wasn't sure why the temp file would work with the first image and then fail with subsequent ones, so I worked around the problem by creating a new temp file for each image.

This fixed the problem and the Add-On now works really well for us.

I will attach my version of the OO2TWiki.pm file so others can use it.

In terms of fixing this in the actual distribution, do I just re-package the original tar-ball and update the attachment on the OpenOffice2TWikiAddOn page?

-- DuncanKinnear - 16 Nov 2007

Note also (for anyone re-creating the distribution tar-ball) that the OpenOffice2TWikiAddOn.txt in the distribution is out-of-date. This should be updated from the topic here on the TWiki web site.

-- DuncanKinnear - 16 Nov 2007

The ModificationPolicy of this add-on is PleaseFeelFreeToModify. That is, you can repackage and upload a new version. Thank you Duncan for contributing to the TWikiCommunity!

-- PeterThoeny - 20 Nov 2007

I'm having the same problem as BenSelinger using TWiki 4.0.2. Additionaly, when I click on the upload button in the convert dialogue, I get partial raw html output of the dialogue page in the browser window. The convert process finishes after a while, however, the originating page becomes deleted as described above.

-- JoachimBlum - 27 Nov 2007

Ben and Joachim,

The problem you are experiencing is a bug with the "Create a link" option.

Avoid ticking the box that says "Create a Link ..." and the problem will not occur.

I have actually tracked down the problem in the code itself, but I'm just heading home for the weekend. I'll post an update 'pm' file on Monday.

-- DuncanKinnear - 07 Dec 2007

OK, I've uploaded new versions of the addon tar.gz and zip packages containing the fixes I have done.

The major changes are:

• The temporary files used to extract the various parts of the source (.zip) file are given more specific file extensions (.xml, .img, etc.). The output text is given the .tml extension. This aids the debugging of the Add-On.
• Separate temporary files are created for extracting each image file (.img extensions). This fixed the bug with only one image being extracted.
• Fixed creation of the link in the parent topic (problem was the naming of the variable that the original parent topic text was extracted into)

Please let me know if you experience any problems with the releases.

-- DuncanKinnear - 10 Dec 2007

Hi Duncan,

it works now. Thanks for the patch.

-- JoachimBlum - 11 Dec 2007

Thx Duncan - image conversion problem indeed fixed

-- EyalTeutsch - 25 Dec 2007

Special Characters (German "Umlaute" like ä,ö, etc.) are not imported correctly. I. e. after Importing "Björn" I get "Björn", in Edit Mode as well as in View Mode.

Shure, it smells like an character set problem, so here's some additional information:

• changing the browsers (firefox and IE) view setting from "western (iso-8859-15) to "unicode utf8" solves the problem displaying (but only for the one page actually displayed)
• TWiki Configuration
  • UserInterfaceInternationalisation ist set (only German activated)
  • All Perl modules for unicode handling were installed after the error was detected, but didn't change anything
  • {Site}{Locale} ist set to "de_DE.utf8" (other setting does not work)
  • {Site}{CharSet} is set to "iso-8859-15"
  • "locale -a" on the server shows "de_DE.utf8" among others
• If I open the imported file on the file system, the umlaute are displayed correct within the linux terminal, but wrong in TWiki. If I edit a new text in TWiki with Umlauts and save it, these charaters are display correct in twiki, but wrong within a unix terminal.

Do you have any hints or ideas? BTW: Besides this problem the import tool is absolute great!!!

-- ThomasStieglitz - 18 Jan 2008

In the OO2TWiki.pm script, around line 333, there is a set of simple substitution commands that convert various multi-byte characters into the XML/HTML equivalent.

As the original author stated in the comments above these:

  # replace some common Unicode chars
  # SMELL - this should be done in xslt, but how?

This has been on my todo list for this AddOn, but I haven't got to it yet.

I have found out that there is a way to do this in XSL using the <xsl:character-map> construct (see this page at w3.org), but I just haven't had a chance to test it.

I also think there must be a better way to translate international characters such as yours without having to define every possible one in a character-map. But maybe there isn't.

You could always start by adding substitutions to the script like the others and then translate these into xsl character-map's.

It is a good idea to switch on debugging for the AddOn (in your TWikiPreferences topic) so you can examine the 'xml' file that is extracted from the 'odt' file. It is usually placed in '/tmp'. The 'xml' file is the original document and the 'tml' file is the resulting TWiki Markup topic text.

Let us know how you get on.

-- DuncanKinnear - 20 Jan 2008

Thomas,

I just re-read the other comments above and noticed Koen's comment on the 28th Aug last year about adding the encoding="iso-8859-15" to the <xsl:output method="html"> line in the odt.xslt file (it's in the pub/TWiki/OpenOffice2TWikiAddOn sub-directory).

I just tried this suggestion here and it appears to fix the umlauts.

-- DuncanKinnear - 21 Jan 2008

Adding the encoding line to odt.xslt fixed the problem. Thank you!

-- ThomasStieglitz - 21 Jan 2008

Does anybody mind if I check in this package to SVN and do some fixes, i.e. implementing an own oo2twiki template instead of using the attachnew, as well as fixing some TWiki API violations?

-- MichaelDaum - 05 May 2008

I am getting the following error when I try the Sandbox link in the usage page:

TWiki detected an internal error - please check your TWiki logs and webserver logs for more information.

Can't locate object method "login" via package "HansSchwing" (perhaps you forgot to load "HansSchwing"?)

I have checked the logs and it says there is an error at .../libTWiki/Contrib/OO2TWiki.pm line 65

Any ideas?

-- HansSchwing - 03 Jun 2008

Emailed the original author whether he is still maintaining this package.

-- MichaelDaum - 12 Jun 2008

Okay got a reply. Wade abandoned this package. I'll try to clean it up so that it becomes an option again.

-- MichaelDaum - 12 Jun 2008

Thanks.

-- HansSchwing - 17 Jun 2008

To get This AddOn to work with 04x02 i had to apply the followin patch:

diff -Naur OpenOffice2TWikiAddOn/lib/TWiki/Contrib/OO2TWiki.pm OpenOffice2TWikiAddOn.sjosi/lib/TWiki/Contrib/OO2TWiki.pm
--- OpenOffice2TWikiAddOn/lib/TWiki/Contrib/OO2TWiki.pm   2007-12-10 22:36:22.000000000 +0100
+++ OpenOffice2TWikiAddOn.sjosi/lib/TWiki/Contrib/OO2TWiki.pm   2008-07-24 15:19:03.000000000 +0200
@@ -62,7 +62,7 @@
 
     my $filePath = $query->param( 'filepath' ) || "";
 
-    my $userName = $session->{user}->login();
+    my $userName = $session->{user};
 
     # hack to make TWiki::Func::* work
     $TWiki::Plugins::SESSION = $session;

-- SimonJosi - 24 Jul 2008

To the add-on author: MichaelLissner added a "Usage Through OpenOffice" section to the add-on topic, and I did a few fixes as well. Please feel free to take this into the next release.

-- PeterThoeny - 18 Nov 2008