create new tag
, view all tags

Uploading Large Files by FTP.

FileAttachment says:
  • ALERT! It's not recommended to upload files greater than a few hundred K through a browser. Large files can be extremely slow-loading, and often time out. Use an FTP site for large file uploads.

I routinely have either large files or a large number of files (usually zipped together into a single large file) of scientific data that I'd like to be able to upload to a TWiki. I know that I could just use plain FTP access to copy the files into the TWiki file system, but I would prefer to:

  1. Not trust myself or others with access to the underlying filesystem so that they can mess it up.
  2. Be able to upload the files without having to do anything especially complicated.
  3. Make it easy to upload files to a TWiki page where details about the files can be recorded without having to navigate twice (i.e. once for editing the page and a second time for uploading the file).

A quick google turned up http://www.perlservices.net/en/programs/psupload/index.shtml, showing that it should be possible to do.

I guess I am going to have to learn perl a bit before I can do this myself - anyone else interested in taking up the challenge?

-- BenWatts - 14 Jul 2006

Did you see the WebDAVPlugin?

-- PeterThoeny - 15 Jul 2006

I did see the WebDAVPlugin, but I am not sure that it is what I'm looking for. webDAV still appears to use HTTP for file transfers and so is likely to have the same issues with large file transfers. I also don't need the versioning features so much since I will be dealing with data files that are uploaded once and never changed (though it would protect against somebody messing things up accidentally). But I guess I should install the WebDAVPlugin anyway and see how it goes. I am also considering another way to solve the problem that would involve:

  • Force TWiki to populate the pub directory with subdirectories for every TWiki page (it currently only creates a directory if a file is attached) so that users can just navigate without having to create directories themselves (thus solving inevitable spelling errors).
  • Install ftp server and give ftp access to the pub directory (jailed so that it is only the pub directory).
  • Work the permissions so that users cannot overwrite anything that the TWiki does.
Is there a simple way to get TWiki to fully populate pub with subdirectories? (It probably requires a tweak of the perl scripts for creating a new topic)

-- BenWatts - 17 Jul 2006

It would be possible to change TWiki to create a pub folder at the time of topic creation, but that would result in upgrade issues. If you need that functionality you could create a shell script that creates a pub dir for all topics that do not have one yet; you can run it from a cron job as the cgi user once a day.

If your files do not need to be version controlled you could setup a separate FTP server and expose the ftp root as http (with Apache config, possibly over NFS). In TWiki you can create a FTPROOT setting in the TWikiPreferences. That way, users can refer to the files located in FTP as %FTPROOT%/path/to/file.txt.

-- PeterThoeny - 18 Jul 2006

Edit | Attach | Watch | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r4 - 2006-07-18 - PeterThoeny
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.