Tags:
create new tag
, view all tags

SID-02340: CreateTopicAndAttachInOneStep

Status: Answered Answered TWiki version: 6.0.2 Perl version:
Category: CategoryAttachments Server OS: Last update: 50 minutes ago

I tried to implement the suggestion of TroyGoodson on CreateTopicAndAttachInOneStep on the Main web (Main), and calling the Main.CreateTopicAndAttach from MyWeb, with urlparam originweb="MyWeb". In that case I can't get this thing working. The temp topic is created an renamed, but the attachment is not attached. Even when I delay the call of the rename action. In my logs I can't find any attempt of Twiki to attach a doc to the temp topic. However, ONLY the first time I run this procedure with a new created Main.CreateTopicAndAttach, I find a TempTopic with an attachmand AND a renamed TempTopic with a NewTopicName. Anyone any suggestion why this isn't working?

-- Emiel Van Riel - 2017-12-18

Discussion and Answer

I admit that I don't understand exactly what you want to do when you write you are calling the Main.CreateTopicAndAttach from MyWeb. I also have to admit that I've no idea where originweb should be applied. In any case, there are two variations of the theme which I got working with small changes in my test installation, maybe one of those works for you:

  1. You have a topic Main.CreateTopicAndAttach, but you want to create topics with attachments in Myweb:
    • The form in the page has a hidden parameter: <input type="hidden" name="newweb" value="Sandbox">. Convert that to type="text" and you can chose the target web of the renaming step.
  2. You want to %INCLUDE{}% Main.CreateTopicAndAttach in a topic in MyWeb and operate it from there:
    • In that case, due to the way TWiki handles variables in included topics, you need to repeat the stanza
      * Set temptopic = FormTargetTemp in the including topic.
    • If you want the new topics also to appear in MyWeb, you can change the value attribute in the same input field from "Sandbox" to "Support".
About debugging: Are you looking into TWiki's log or into the web server's? TWiki's log might not show an entry for a failed operation - if "rename" is processed before "upload" is finished, then the latter will fail because the topic it wants to attach to no longer exists. The web server's log should still show the POST requests in the expected sequence, but if they run in different threads/processes there's no way to tell which one is finished first. TWiki does have a locking procedure, but this just grants locks based on your browser session - if the same session fires three requests without waiting for a reply then I guess this will not get caught. I am safe from this because I run the test installation single threaded...

-- Harald Jörg - 2017-12-18

My purpose is to use the Main.CreateTopicAndAttach topic in my Twiki's Main Web as a 'template' topic for all my webs to create a new topic and attach a document in one step. But the new topic should be in the web I'm working in at that moment, e.g. MyWeb, and not in Main Web. So I made a link in MyWeb: [[Main.CreateTopicAndAttach?originweb=!Support][Create new topic]]. In I replaced 'Sandbox' by %URLPARAM{"originweb"}% . So far so good. Twiki creates MyWeb.FormTargetTemp (which I meant with 'temp topic') and renames it to MyWeb.NewName as given in the input field. But a selected document is not attached to new topic, nor to MyWeb.FormTargetTemp. Only the very first time I saw two topics: MyWeb.FormTargetTemp with an attachment and MyWeb.NewName, without any attachment. So I tried to delay the start of rename by setting the delay time in the script. But this didn't help. So I think there is a problem with the sessions, as you describe, but I have no idea how to solve this.

-- Emiel Van Riel - 2017-12-18

When I submit the forms step-by-step it works fine. But when using the JavaScript, no matter how long I make the delay between the actions, two topics are created: the 'renamed' topic MyWeb.NewName WITHOUT attachment and the 'temporary topic' MyWeb.FormTargetTemp WITH the attachment. It seems to me that Twiki 'doesn't know' that the attachment is attached to MyWeb.FormTargetTemp, when it starts renaming this topic to MyWeb.NewName. Or the process of uploading the attachment takes to long (but in my tests I gave the system time enough to upload a small attachment). Or the system isn't able to finish the upload or give feedback to the system that the upload has finished. I don't know. Is there any difference in how 'upload' is processed from 'save' and 'rename'?

-- Emiel Van Riel - 2017-12-27

Both 'upload' and 'rename' are quite different from 'save' because they involve directory operations on the system.

  • 'save' just writes to the topic file - and done (if I say 'write' here, I always mean 'checkin into TWiki's version control system', usually RCS).
  • The 'upload' script needs to create a directory in the 'pub' area, then write the data provided with the request to the attachment file, and only after this is finished, it updates the topic with the metadata about the attachment (visible in the "attachment table"). Creating a directory can be slow if your file system is rather full or fragmented or on a NFS share or....
  • 'rename' needs to create the attachment directory for the new topic name, too (if only by renaming the directory from the old topic). It also checks whether there are links to the old topic which need to be redirected to the new topic by reading and rendering each and every topic in the current web, or in all webs. So, depending on the size of your TWiki, 'rename' can take really long to complete.

From your observations, the sequence seems to be like this:

  1. 'rename' checks whether there are attachments before 'upload' has created the attachments directory - and therefore moves the old topic without its attachments.
  2. 'upload' creates the attachment directory to the old topic, and writes the old topic (which was just before removed by 'rename') with the attachment meta data.

For a test, you could try to reduce Apache's number of threads to 1: Set MaxRequestWorkers 1 (Apache 2.4) or MaxClients 1 (Apache 2.2) in the server configuration. If the JavaScript solution works in that configuration, some parallelism is probably the culprit. I'm not so sure about that: I just tried the stuff on a plain Apache web server with mod_cgid plus plain TWiki 6.0.2 - and it just works with the default setting (150 parallel threads) and both the original form and your variation containing originweb. So something in your environment behaves differently.

About the JavaScript solution in general: It is quite obvious that it isn't too robust - just consider two people trying to create a new topic with attachment at the same time. Both 'save' operations would be synchronized by TWiki, both 'upload' operations might work - depending on which 'rename' operation is processed first, because only one rename can succeed. The renamed topic of the winner might have two attachments, but the topic content of the other guy, while the loser has no chance to find out where his attachment and text went because all he gets is the information that the rename operation failed.

-- Harald Jörg - 2017-12-27

...I should add that I'm interested in that stuff because in my last company we'd really liked to have such a feature. We had a "document library" where relevant PDF or MS-Office documents could be uploaded and archived - together with meta data and comments. The traditional TWiki work flow (first create the topic, then attach the PDF or MS-Office document) quite often made me (and others) to just forget to attach the document. So, in my opinion, this would make a good feature request - only that I didn't have the time to implement it. This might change in 2018.

-- Harald Jörg - 2017-12-27

Harald, that is exactly why I am trying to make this work. For 'first users' the Twiki work flow may be indistinct or confusing; this could be made more user friendly.

-- Emiel Van Riel - 2017-12-28

The problem of two people creating topic at the same time can easily be solved: Add * Set temptopic = TempTopic%SERVERTIME{"$epoch"}% to Main.CreateTopicAndAttach, so each user gets its own TempTopic...... Or 'call' Main.CreateTopicAndAttach with an URLparameter =temptopic = "TempTopic%SERVERTIME{"$epoch"}%" and use the URLparameter in the forms.

-- Emiel Van Riel - 2017-12-28

After setting MaxRequestWorkers 1 everything worked fine. But the processing took a while. So probably the time intervals I tried were still too short. I succeeded in implementing an EventListener in the JavaScript, to 'listen' whether the topic creation and attachment upload processes are finished. Since I have no experience with JavaScript, I think this code can be made ' cleaner', but for now it works. This is what it looks like now:

<!--
    * Set READONLYSKINMODE = 1
   * Set temptopic = TempTopic%SERVERTIME{"$epoch"}%
   * Set originweb = %URLPARAM{"web" default="Main"}%
   * Set origintopic = %URLPARAM{"origintopic" default="TempTopics"}%
-->

<SCRIPT language="JavaScript">

function handleResponseReceived() {
 
  var uploadFrame = document.getElementsByName('dummyattach')[0];
  document.formattach.target = 'dummyattach';
  document.formattach.submit();

  if (uploadFrame.addEventListener) {
    uploadFrame.addEventListener('load', handleResponseReceived2, false);
  }
  else {
    uploadFrame.attachEvent('onload', handleResponseReceived2);
  }
}

function handleResponseReceived2() {

  var uploadFrame = document.getElementsByName('dummyrename')[0];

  document.formrename.submit();
}


function submitform()
{
  var uploadFrame = document.getElementsByName('dummycreate')[0];
  document.formcreate.target = 'dummycreate';
  document.formcreate.submit();

  if (uploadFrame.addEventListener) {
    uploadFrame.addEventListener('load', handleResponseReceived, false);
  }
  else {
    uploadFrame.attachEvent('onload', handleResponseReceived);
  }
}
</SCRIPT>

-- Emiel Van Riel - 2017-12-29

Now I want to complete my 'project' by adding a selection box for the parent topic and implementing the Dropzone. After that I will put the result in te Sandbox Web.

-- Emiel Van Riel - 2017-12-29

Hello Emiel,

thanks for sharing, and for running the single-threaded test! Adding an event handler sure helps to mitigate the issue. A minor nitpicking suggestion: If we are concerned about two people creating a topic at the same time, then %SERVERTIME% will likely be the same for both. I'd rather augment the topic name with %WIKINAME%, reducing the conflicts to the same user id creating two topics at the same time which should, if at all, only happen for the guest user(s). There's also %SESSIONID% which is really unique per session.

-- Harald Jörg - 2017-12-29

See NewTopicCreateAndAttachInOne for the result (without the Dropzone).

-- Emiel Van Riel - 2018-01-02

      Change status to:
ALERT! If you answer a question - or someone answered one of your questions - please remember to edit the page and set the status to answered. The status selector is below the edit box.
SupportForm
Status Answered
Title CreateTopicAndAttachInOneStep
SupportCategory CategoryAttachments
TWiki version 6.0.2
Server OS

Web server Apache 2.4
Perl version

Browser & version

Edit | Attach | Watch | Print version | History: r9 < r8 < r7 < r6 < r5 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r9 - 2018-01-02 - EmielVanRiel
 
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.