Tags:
create new tag
, view all tags

Archive of MathModePluginDev Discussions

The MathModePlugin allows you to insert LaTeX formatted math into TWiki documents, and renders images of the expressions using latex2html Here we describe limitations of the plugin, how to fix them, discuss new features people would like to see, and how to implement them.

Note! I have refactored this page to reflect the new version of the plugin. Comments that were addressed got removed, commentary that is still valid remains. View the pre-refactored page here (opens a new window).

Additions in the Second Version

  • The syntax has changed (for the better). The delimiter now uses the percent sign boundaries that are the convention in plugins. Two new tags have been added that render an image in big mode on its own line, centered.
  • Preview mode, deleting images, and all that: If the expression has already been rendered into an image, the saved image is used. If no image exists for the expression, it is rendered and saved (this happens during all ways of looking at a page: view mode, preview mode, viewing an earlier revision). If the plugin finds an image that isn't being used, it is deleted, but only if you are view ing the page when this happens. That is, preview mode will render images it needs, but it won't delete images that you remove from the page. This way, when the author is previewing some deletions, the images won't get deleted, so someone else viewing the page concurrently won't have to re-render the images. Only when a page gets viewed does it try to delete images. This can cause the author to re-render some images he has added, but this is a lesser evil than forcing the innocent viewer to wait for re-renders. You can still have problems when looking at old versions, in that images used in the latest revision of the page will be deleted, but there is no obvious solution for that. The plugin determines whether it's in view mode (and therefore allowed to delete images) by looking at the name of the script that called it. If the script starts with view then it decides it's in view mode.

Problems without Solutions

  • The plugin is heavily dependent on latex2html. Several issues have surfaced due to different versions of this tool. For example, some versions require a preamble to the LaTeX file that the plugin generates. Since doing this doesn't break anything, it has been added. Some versions generate ugly left and bottom borders on all images. Some versions pad own-line equations with left-hand whitespace in a bad attempt to center the equation. If you notice these issues, try upgrading your installation of latex2html

Deficiencies That Must Be Resolved

  • Finding errors in LaTeX is rather hard: We've got a page which uses rather a lot of LaTeX, and one mistake can destroy half the page images (latex2html stops with an error, and the images don't get made). Is there a way to improve the error reporting ? For instance, other plugins come back and tell you that there's an error. (By the way, it's a fantastic plugin !)

-- DavidBowler - 28 Aug 2003

Limitations That It Would Be Nice to Remove

  • The plugin currently relies on the twiki_l2h.init file, attached to the MathModePlugin page, to feed options to latex2html. It would be nice if some or all of the options that effect generation of images were available as TWiki variables. For example, by manipulating TWiki variables, you should be able to choose gif or png images, extra scaling of the math, etc.

What the Plugin Should NOT Do

  • I think that the plugin should NOT try to implement any more of LaTeX than math mode. A TWiki page and a LaTeX document are very different for a reason: They are tools to solve different problems, and should stay that way. The inline display of math in a TWiki page is a useful, manageable addition. If you find you want more LaTeX functionality in TWiki, I would argue that you should rethink what it is you want to do. What's your opinion?

  • I agree. People who use LaTeX a lot (including me) can always continue to do so in a twiki. E.g., I do so when I'm pasting in bits of what I'm writing elsewhere. Most non-mathematical LaTeX is easily readable whether it's rendered or not. -- JasonGrossman - 08 Jan 2004

  • I don't fully agree. The picture environment of LaTeX could also be implemented by this plugin. It is very common to use it to draw mathematical diagrams. - LouisGranboulan - 30 Apr 2004

What Does the Plugin Do Currently?

Here we describe what the current version of the code does.

The goal: For each unique math expression in a page, generate an image of the expression. After the image is generated, subsequent views of the page should not require a re-render.

The Perl program latex2html is used to convert LaTeX formatted math into inline images ( png or gif ).

  1. A regular expression searches each line for math expressions.
  2. Text between the math delimiters is saved to a hash table. The MD5 hash code for each string of math is calculated. This is done so we can uniquely identify each string by a fixed-length string, which we will use for the filename of the image representing the formatted expression. Since we are using a good hash function, any difference in the math string will result in a different hash code.
  3. In the document, the math string is replaced with a link to an attached image whose filename is the hash code, with an appropriate extension (e.g. .png). If the image is supposed to be on its own line, it is centered using a div tag.
  4. When all math strings have been collected, we find the attachment directory for the page.
  5. We look in the directory for images with the characteristic filename (the hash code is always the same length, and composed of a subset of possible characters). If the image corresponds to one of the math strings, then we don't need to re-render that one. If the image doesn't correspond to a math string, then it is probably an old image that we no longer use, so it is deleted, but only if the plugin was called from the view script!
  6. If we still have new math strings to render, they are output to a LaTeX file. The latex2html program will call the images img1.png, img2.png, etc., so we remember the association of the hash code with the order we placed the math strings into the file.
  7. We run latex2html on the file, using the init file that specifies variables that control the generation of the images.
  8. We rename the files according to the hash code scheme.
  9. We delete all the extra files, the directory that latex2html created, and the file we created.
  10. We reset the hash table of math strings, so that subsequent passes during the current render will not try to re-render images.

-- GraemeLufkin - 05 Apr 2002

Very good. I developed a similar extension for an old version of TWiki, and it was planned to move it to the new version, but it is no more needed smile

There are a few points I would like to discuss.

  1. latex2html already performs the hash comparisons, so in my old version it was sufficient to collect all mathematical expressions to a file, and pass it to latex2html, without erasing the created directory. Indeed, it is sufficient to insert a \begin{rawhtml} at beginning, replace each starting dollar with a \end{rawhtml} and each closing dollar with a \start{rawhtml}, and finally insert a \end{rawhtml} at the end of the file and pass the whole thing to latex2html, which takes care of checking image versions.
  2. There are problems with previews that do not lead to a save, since during preview you delete the old images (if they are eliminated by the author). I was considering using the following approach:
    • when previewing, create a "preview" directory with unique name in the pub directory of the topic. Copy the contents of the old "preview" dir (indicated by a META tag) into the new dir.
    • remove the PRE and VERBATIM parts, as done in RicherSyntaxPlugin, replacing them with an indexed tag
    • run latex2html on the file with the replacements described above: $math expr$ => \end{rawhtml}mathexpr\start{rawhtml}
    • insert back PRE and VERBATIM stuff. This should be faster than processing the whole thing a line at a time.
    • build a preview version of the page processing the file generated by latex2html.
    • when saving, insert the name of the preview dir in a META tag, and delete the old preview dir (this can be done also at a later time by a cron job).
    • since this preview dir is actually a cache for the page, one can also store there a preprocessed version of the page, elaborating all "static" tags. One can also store the page contents in this dir and avoid resending them back as an hidden field in the preview phase, it is sufficient to send the name of the preview directory.
    • this cache would permit the recover of an interrupted editing (as I would like to have when writing this comment...)
    • I planned to atomize as much as possible the math expression, in order to save regeneration time (using the expert mode of latex2html). I think that your approach of using a digest to identify the images would be very useful if one modifies latex2html so to use a "central" database of images of mathematical expressions. In this way the mathematical expressions would be shared by all pages, thus reducing the need of regenerating them (if you assume that the same mathematical espression is used by more than one page). Moreover, this solves a problem in latex2html: since it number images as 1,2,3..., if a new image is inserted it is not reloaded automatically by the browser, since it thinks that it is already in its cache.

-- FrancoBagnoli - 05 Apr 2002

It would be useful to have a generic service for storing data produced by plugins, like for example to store the math images. This brings up PluginDataStore.

-- PeterThoeny - 05 Apr 2002

Regarding when to try and delete old images: I agree with all the commentary above, but couldn't see an easy way to really solve the problem. This is most likely because I'm dumb. So I took a more reserved approach, and decided that the only time the plugin should delete files is when a page gets viewed (that is, called by the view script.) This addresses some, but certainly not all, of the issues addressed here.

-- GraemeLufkin - 05 Jun 2002

I've attached a couple of patches (MathModePlugin_DJ1.patch). They clean up the debug messages (adding indentifiers), and add a new preferences variable LATEX2HTMLEXE, which allows one to specify the path to latex2html. I didn't like the idea of adding the directory to the default path (for security reasons) and latex2html knows where to find all of its bits on its own.

-- DiabJerius - 08 Aug 2002

I've tested MathModePlugin, and it's simply beatiful smile . But I noted that is does not work with others skins but the deafult one. I've attached ?skin=plain at the end of URL, and the LaTeX part was not substituted by the math image. Where we can fix this?

-- AntonioTerceiro - 16 Aug 2002

I don't see this error on my installation, and I can't offhand see where in the code it might matter. Maybe post a link to the broken page and we can figure it out.

-- GraemeLufkin - 16 Aug 2002

I think that was a little confusion, I've tested again and it worked. I'will try to reproduce the error, and if I get I'll give more details.

-- AntonioTerceiro - 18 Aug 2002

Not a bug with MathModePlugin but this email mentions that some browsers don't understand the method used to encode transparency into PNG images so you may see a grey background. If you can't live with this, change your browser smile .

-- MichaelGindonis - 30 Sep 2002

I'm afraid I have to disagree with you there. Check out this test page from my personal twiki, it demonstrates two very similar images that latex2html generates with and without the transparency. The math strings differ by only one character, yet one gets generated with broken transparency, and the other with working. And it's NOT the browser! I've used several browsers that I know do transparent PNGs correctly. You can download the images, and view them with whatever program you like, and still the transparency is broken in one and not the other. Anyway, just my two cents.

-- GraemeLufkin - 01 Oct 2002

Just installed MathModePlugin and I like it. But there is a small problem. Please look at this example and notice the error in the rendering. It occures when the same equation is present twice in a document but each time with different number of spacing.

-- MichaelMazilu - 04 Oct 2002

Competition Time! I've created LaTeXToMathMLPlugin that directly translates embedded LaTeX into MathML in place. Browsers implementing the 10-year-old MathML standard should be able to render the page directly. This gets around the embedded image solution used in this plug-in.

I've removed the previous discussion about this, the deed is done... smile

-- SimonClift - 22 Apr 2003

I get an error in error.log when I run the example on the MathModePlugin topic. I get:

404 not found for:

  • pub/TWiki/MathModePlugin/bf17d2b6259bb6065011f86e0bfa5ee0.png

Any clues as to how to track this? I do have latex2html installed. Thanks.

-- MartinCleaver - 08 Jun 2003

I had a similar error. In my case the ownership of /twiki/pub/ and subdirectories was not set to nobody or www-data -- MiguelF - 09 Jun 2003

checked .zip file into CVSplugins:MathModePlugin

-- WillNorris - 24 Nov 2004

I can't get this plug-in to work. I have latex2html installed, and playing well with another package (Webworks web quiz system). I'm new to twiki, but thought I had the permission thing down... I've used the error directive (Apache 2.0 workaround) and the error message I get is Fatal (syswait): exec " ./images.tex" failed: Permission denied at /usr/local/bin/latex2html line 3785

Cannot read logfile './images.log': No such file or directory Any specific clues? -Mike Gallis

-- MichaelGallis - 26 Apr 2005

Im having a similar problem. Here is the error log by Apache 2. Any ideas ? I checked the permissions. They were indeed not right, but after being corrected it still breaks. Is this a issue of a new version of TWiki ?

[Wed Apr 27 15:52:33 2005] [error] [client 127.0.0.1] [Wed Apr 27 15:52:33 2005] view: print() on closed filehandle MATHOUT at /usr/share/perl5/TWiki/Plugins/MathModePlugin.pm line 246., referer: http://***/cgi-bin/twiki/view/TWiki/WebHome

[Wed Apr 27 15:52:33 2005] [error] [client 127.0.0.1] [Wed Apr 27 15:52:33 2005] view: print() on closed filehandle MATHOUT at /usr/share/perl5/TWiki/Plugins/MathModePlugin.pm line 252., referer: http://***/cgi-bin/twiki/view/TWiki/WebHome

[Wed Apr 27 15:52:33 2005] [error] [client 127.0.0.1] [Wed Apr 27 15:52:33 2005] view: print() on closed filehandle MATHOUT at /usr/share/perl5/TWiki/Plugins/MathModePlugin.pm line 256., referer: http://***/cgi-bin/twiki/view/TWiki/WebHome

[Wed Apr 27 15:52:33 2005] [error] [client 127.0.0.1] Cannot read ./twiki_math.tex , referer: http://***/cgi-bin/twiki/view/TWiki/WebHome

[Wed Apr 27 15:52:33 2005] [error] [client 127.0.0.1] File does not exist: /var/www/twiki/pub/TWiki/MathModePlugin/bf17d2b6259bb6065011f86e0bfa5ee0.png, referer: http://***/cgi-bin/twiki/view/TWiki/MathModePlugin

-- IgorNikolic - 27 Apr 2005

Maybe the temporary files should better be generated in /tmp/?

-- ChristopherHuhn - 08 May 2005

Note: this plugin seems to run on DEVELOP too.

The first 'problem' I've had was with TWikis internal PATH setting. I had to add the path to the directory containing the latex2html command to TWikis cgiPath in my LocalSite.cfg resp. specify the full path in MathModePlugin.pm in order to make it run.

The second 'problem' I still have is that I get

texexpand V2002 (Revision 1.11)
This is dvips(k) 5.95a Copyright 2005 Radical Eye Software (www.radicaleye.com)
' TeX output 2005.04.29:1030' -> /d/temp/l2h1968/image (-> /d/temp/l2h1968/image001) <tex.pro><texps.pro><special.pro><color.pro>
<cmr10.pfb><cmr5.pfb><cmmi7.pfb><cmmi10.pfb><cmsy7.pfb><cmex10.pfb>[1]
entries in my apaches error_log. Does anybody know how to turn this off? Thanks!

My third 'problem' is that I get black borders at the left and the bottom side of the generated .png. How do I have to configure latex2html to solve this? (it's ugly!) -- Oh I see, this is a known bug (compare Problems without Solutions) with some versions of latex2html. I'm using version 1.67, guess this is too old, isn't it?

-- FranzJosefSilli - 29 Apr 2005

I having problems when I render "If %$x>y$% is true". The greater-than gets embedded in the image's alt attribute, and TWiki thinks that it's the end of the img tag. It then translates the real end of the img tag to &gt;. The tag never gets terminated and the "is true" doesn't get rendered at all on the page.

I fixed it by changing

   #remove any quotes in the string, so the alt tag doesn't break
   my $escaped = $_[0];
   $escaped =~ s/\"/&quot;/gso;
to
    #remove any quotes in the string, so the alt tag doesn't break
    my $escaped = $_[0];
    $escaped =~ s/\"/&quot;/gso;
    #remove any > symbols in the string, so the alt tag doesn't 
    #fool TWiki into ignoring the rest of the img element.
    $escaped =~ s/>/&gt;/gso;
in the handleMath subroutine. Warning: I'm not a Perl wizard.

-- MichaelMcDougall - 29 Apr 2005

The current version assumes that the pub/WEB directory has already been created, and fails when that is not the case (for example, the web has no attachments yet). I recommend the fix that was added to the DirectedGraphPlugin -- use mkpath to create the directory. See Steffen's comment from April 5 2005 for details.

-- MichaelMcDougall - 07 Jun 2005

After working through most of the latex2html related installation pain, I still can't manage to get the plugin running.

apache error_log:

texexpand V2002-2-1 (Revision 1.12)
[Sat Jun 11 23:13:38 2005] [error] [client 145.3.33.134] File does not exist: /wikiroot/twiki/pub/TWiki/MathModePlugin/bf17d2b6259bb6065011f86e0bfa5ee0.gif

debug.txt:

11 Jun 2005 - 23:13 - TWiki::Plugins::MathModePlugin::initPlugin( TWiki.MathModePlugin ) is OK
11 Jun 2005 - 23:13 - MathModePlugin::endRenderingHandler( TWiki.MathModePlugin )
11 Jun 2005 - 23:13 Path to attachment dir: /wikiroot/twiki/pub/TWiki/MathModePlugin
11 Jun 2005 - 23:13 Directory already exists.
11 Jun 2005 - 23:13 Renaming twiki_math/img1.gif to bf17d2b6259bb6065011f86e0bfa5ee0.gif
11 Jun 2005 - 23:13 Math strings reset, done.
11 Jun 2005 - 23:13 - MathModePlugin::endRenderingHandler( TWiki.MathModePlugin )
11 Jun 2005 - 23:13 - MathModePlugin::endRenderingHandler( TWiki.MathModePlugin )

This doesn't give me much info to work with... does anyone have any idea what might get this working? Thanks!

-- JosMaccabiani - 11 Jun 2005

I guess it's my latex2html installation. Could someone send me his prefs.pm file? I guess my problem is in there, but I can't find it...

Thanks!

-- JosMaccabiani - 12 Jun 2005

Jos: I've run into similar problems. If latex2html fails the plugin won't notice and will print out debug strings as if everything was fine. There's about zero error checking in the current version of the plugin. I recommend going into "MathModePlugin.pm" and redirecting latex2html's output to a file (it currently goes to /dev/null) and seeing what it says.

-- MichaelMcDougall - Jun 20 2005

Michael, thank you. I've changed /dev/null to /path/to/twiki/l2h.txt but the file never gets created.

Actually, what seems to happen is that a symbolic link is created:

lrwxrwxrwx  1 nobody nobody  9 Jun 15 18:03 bf17d2b6259bb6065011f86e0bfa5ee0.png -> image.png

while the apache error log says:

[Sat Jun 25 22:42:20 2005] [error] [client 145.3.33.130] File does not exist: /wikiroot/twiki/pub/TWiki/MathModePlugin/bf17d2b6259bb6065011f86e0bfa5ee0.png

this error is confirmed by the fact that image.png cannot be found by myself either i.e. it does not exist.

I have the feeling that latex2html is not supposed to put symlinks in the proper location but the files themselves, otherwise the images would have to be re-rendered all the time, no?

I you have any other hints for me it would be highly appreciated.

-- JosMaccabiani - 25 Jun 2005

On my system the plugin goes through the following steps

  • construct latex file twiki_math.tex
  • run latex2html on twiki_math.tex
  • rename twiki_math/image1.png to HASH_VALUE.png (HASH_VALUE is the md5 sum of the latex formula)
  • repeat the previous step for images 2, 3, ..
  • remove the twiki_math directory
  • remove the twiki_math.tex file

The plugin pretty much does all of that even if one of the early steps fails. The debug log can therefore be kind of deceiving when it claims to be renaming files that never got created.

In my system, the image files are renamed, not linked. (Though maybe perl implements "rename" as a link in some cases--I'm not a Perl expert.)

Some things that helped me solve problems in my system:

  • Like I mentioned above, capture the output of latex2html. Get both the error stream and the standard out stream--the plugin doesn't do that by default. Direct that information to the debug log, some other file, or even get it rendered on the page containing the latex formulas.
  • comment out the line in "MathModePlugin.pm" that deletes twiki_math.tex, and then try to run latex2html on it from the command line.
  • comment out the line line "MathModePlugin.pm" that deletes the twiki_math directory. Examine it to see if anything went wrong.
  • Try switching the image format from png to gif. On one of my systems the libraries for PNG weren't available and latex2html failed.

-- MichaelMcDougall - 28 Jun 2005

Michael, thanks for the reply. The twiki_math.tex is generated. When I manually run latex2html on it the first output line reads:

Warning: This system does not support generation of images

So i guess that's a hint. Otherwise, the output does not show any signs of errors (apart from the lack of an image file actually being generated).

Argh, this is getting frustrated for a Linux n00b like myself...

-- JosMaccabiani - 02 Jul 2005

It sounds like your latex2html can't find the libraries to generate images (or couldn't find them when it was configured and compiled). Check out the latex2html help and try to get latex2html running manually with images being generated. Once that works, go back to getting it working in TWiki.

-- MichaelMcDougall - 07 Jul 2005

I recently came across the same rendering problem listed above by "MichaelMazilu - 04 Oct 2002". This problem originates in how latex2html handles white-space within the math to be rendered, producing a mismatch in the hash table between this plugin and latex2html.

After reading this discussion page more closely, it is my impression that this problem, and a number other difficulties listed above, could be easily corrected by abandoning the dependency on latex2html and using a plain latex-to-ps and a ps2png-type converter. Specifically, latex2html does alot of work generating the node links between equations, sections, figures, and the like. This plugin is specifically geared toward rendering latex mark-up. So, much of the latex2html functionality appears to be discarded from the outset. I'm curious if other users of this plugin feel the same way.

My thought is that one could use the ImageMagick or GraphicsMagick 'convert' function to generate png images from latex (via a temporary ps file). 'convert' includes the ability to tightly crop the image and set the background transparent.

I will likely end up changing have modified this module to elliminate latex2html for my needs and to get around the problems I have encountered. If others are interested, I'm happy to share the changes.

-- ScottHoge - 20 Jul 2005

I have one minor patch to add. In my TWiki installation, I'm running both this plugin and the WysiwygPlugin. At some point within the Kupu editor cycle, some Latex commands get translated to HTML entities. Adding the following code to the plugin will translate them back, so that equation editing can be done within Kupu.

sub handleMath
{
# This function takes a string of math, computes its hash code, and returns a
# link to what will be the image representing this math.
### my ( $math_string ) = @_;   # do not uncomment, use $_[0], $_[1] instead

        my $math_string = $_[0];
        # remove latex-common HTML entities from within math env
        $math_string =~ s/&amp;/&/og;
        $math_string =~ s/&lt;/\</og;
        $math_string =~ s/&gt;/\>/og;

        #compute the MD5 hash of this string
        my $hash_code = md5_hex( $math_string );
        #store the string in a hash table, indexed by the MD5 hash
        $hashed_math_strings{$hash_code} = $math_string;

        ...

-- ScottHoge - 25 Jul 2005

Refactored Mikaels question to the support topic MathModePluginRenderingInTable.

-- FranzJosefSilli - 08 Aug 2005

See related LatexModePlugin

-- PeterThoeny - 22 Aug 2005

I miss one simple feature: LaTeX math mode is not complete without the AMS TeX extensions. Since this is only meant for maths anyway, I suppose it can't hurt to add \usepackage{amsmath} to the preamble of the generated LaTeX?

-- MartinGiese - 05 Sep 2005

why was this set to ObseletePluginPackage? the existence of LatexModePlugin doesn't make this one obselete.

-- WillNorris - 29 Sep 2005

Oh, sorry, my fault. I stupidly thought the LatexModePlugin includes a mode that's backward compatible to the MathModePlugin. Well IMHO it should. wink

-- FranzJosefSilli - 30 Sep 2005

Franz, the syntax in LatexModePlugin is backwards compatible to ease migration, but the rendering machinery is very different. I can't speak to obsolescence, however.

-- ScottHoge - 02 Oct 2005

I got this feedback via e-mail:

For this plugin, I have a request. The current plugin uses a grey background with black writing. Some of my users are complaining that this is hard to read. I would like the ability to change the background colour or set the colour to transparent.

-- PeterThoeny - 13 Oct 2005

Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r2 - 2006-08-05 - PeterThoeny
 
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2017 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.