Feature Proposal: Allow the ATTACHURL variable to reference other topics

Motivation

Sometimes I want to reference attachments on other topics.

Description and Documentation

This will extend the syntax of the ATTACHURL and ATTACHURLPATH variables to take optional arguments. The default behavior is the same.

ATTACHURL -- full URL for attachments in the current topic

• Syntax: %ATTACHURL%
• Expands to: https://twiki.org/p/pub/Codev/AttachUrlForOtherTopics
  • Example: If you attach a file you can refer to it as %ATTACHURL%/image.gif
• Syntax: %ATTACHURL{...}%
  • Parameters:
    web - specify the web
    topic - specify the topic (or web.topic)
    Examples:
    %ATTACHURL{web="Main" topic="WebHome"}%/foo.png
%ATTACHURL{topic="WebHome"}%/foo.png
• %ATTACHURL{web="Fred" topic="FlintStone"}% is identical to %PUBURL%/Fred/FlintStone
• Related: ATTACHURLPATH, PUBURL, PUBURLPATH, SCRIPTURL, SCRIPTURLPATH, FileAttachments

Examples

%ATTACHURL{web="Main" topic="WebHome"}%/foo.png
%ATTACHURL{topic="WebHome"}%/foo.png

Impact and Available Solutions

Implementation

Diff versus a stock 4.1.2 installation. Hopefully the cut and paste job works okay.

*** lib/TWiki.pm   Sat Mar  3 06:45:57 2007
--- lib/TWiki.pm   Wed May  2 14:55:19 2007
***************
*** 3349,3359 ****
--- 3349,3365 ----
  
  sub _ATTACHURLPATH {
      my ( $this, $params, $topic, $web ) = @_;
+     $web = $params->{web} if (exists($params->{web}));
+     $topic = $params->{topic} if (exists($params->{topic}));
+ 
      return $this->getPubUrl(0, $web, $topic);
  }
  
  sub _ATTACHURL {
      my ( $this, $params, $topic, $web ) = @_;
+     $web = $params->{web} if (exists($params->{web}));
+     $topic = $params->{topic} if (exists($params->{topic}));
+ 
      return $this->getPubUrl(1, $web, $topic);
  }
  

-- Contributors: GordonTetlow

Discussion

-- GordonTetlow - 02 May 2007

Just noticed this very old feature request: ExtensionToAttachUrlDirective. While arguments that using %PUBURL% is the standard way, it certainly is less obvious than this extension to %ATTACHURL%. Having a single way to reference all attachments is better than having one way for the local topic, and another way for attachments on other topics.

-- GordonTetlow - 02 May 2007

I've considered doing this several times. The reason I didn't is that the whole approach to url generation is flawed. Let me explain.

%SCRIPTURL% originally generated a fragment of URL path. Thus %SCRIPTURL%/view%SCRIPTSUFFIX%/Web/Topic?blah=blah was required to generate a script URL. I changed this so that you can write %SCRIPTURL{"view"}%/Web/Topic?blah=blah. The problems with both of these approaches are:

1. they require TWiki activity in both phases of TWiki rendering - variable expansion and TML rendering
2. they require an intimate knowledge of URL construction from the user

Originally I wanted to move the web, topic and URL parameters into the %SCRIPTURL%, thus: =%SCRIPTURL{"view" topic="Web.Topic" blah="blah"}%. The huge advantage of this is that it is resolveable entirely during variable expansion; it doesn't require TWiki rendering to recognise a URL. However resistance to any change was so great I felt I had to compromise.

Given that single-pass URI assembly is a desirable goal, and rather than mucking with the existing mechanisms, how about a slightly different approach:

• %LINK{script="view" topic="Web.Topic" blah="blah"}%
• %LINK{topic="Web.Topic" attachment="blah.png"}%

This would also give scope for extensions to generate esoteric and powerful URLs in a more friendly and portable way

• %LINK{script="rename" topic="Web.Topic" attachment="blah.png" newweb="Trash" newtopic="Rubbish"}%

I used the name LINK rather than URL or URI because it feels less likely to scare the innocent (SCRIPTURL and PUBURL are very scary to most people).

-- CrawfordCurrie - 03 May 2007

I ran into a similar problem in a TWiki application where links to attachments were returned in a format string of a %SEARCH% - %ATTACHURL% was expanded in the topic of the search result instead of in the context of the topic found. My current workaround is to write things which look very similar to Gordon's initial example:

%PUBURL%/Main/WebHome/foo.png
%SCRIPTURL{viewfile}%/Main/WebHome?filename=foo.png

Obviously the %SCRIPTURL% example is an excellent proof of Crawfords second observation: I need to know how viewfile works. But this is the only way to do it if access control is required (e.g. for documents with non-disclosure agreement).

On the other hand, I don't think you can avoid TML rendering with an enhanced %SCRIPTURL% processing. The TML renderer has some "context intelligence":

• Bare URLs are replaced with <a href="...">...</a> elements.
• URLs embedded as attributes of HTML elements are not replaced. If a topic contains explicitly <a href="...">...</a>, it is left "as is".

When expanding the %SCRIPTURL% variable, you should not need to know which of these cases apply. This sort of decisions should be left to the TML renderer.

And finally, as another incompatible idea, I'd favour Gordon's approach to separate web and topic parameters instead of the current Web.Topic notation (if only to avoid some of the ambiguities with hierarchical webs).

-- HaraldJoerg - 03 May 2007

Crawford, your reasoning is laudable, but seems to try to do too much. The scope of my change is very small and doesn't change the rendering path of the ATTACHURL at all, it just let's me redirect it to another topic.

Harald, I chose separate web and topic parameters because most variables seems to take them as separate parameters already (SEARCH and REVINFO spring to mind). It also simplifies the code. =)

-- GordonTetlow - 03 May 2007

I like Gordon's proposed syntax, which is referenced as an example of what TWiki does not do already, in GenPDFAddOn. I would intuitively expect that you could do %ATTACHURL{"Web.TopicName"}% as well, that might be nice if it doesn't interfere with some other notion of its default argument--it would be easier to accept that as user input, for example.

-- DylanPartridge - 03 May 2007

I don't think you understand my point. While the proposed syntax is fine as far as it goes, it is another example of an incremental change approach that has left TWiki with an awful lot of cruft. At the moment ATTACHURL is just a simple TWiki variable, so the cruft quotient is low. Adding a new function is a lot more of a burden, so it has to be done properly, and designed taking into account all the impacts. Until we have that design, I don't want to rush into adding yet more features that have limiting potential.

(As an example of how cruft breaks the future, consider rest. Calls to this script expect the path info to specify the implementing module and the function; not the web and the topic. As a result, rest URIs are not interchangeable with other TWiki URIs. You have to know not only how to construct a URI, but also the intimate details of the rest implementation)

Harald makes an excellent point about passing authen info on to pub access URIs. However the same is true of any twiki URI, and is a limitation of the current implementation of pub. If pub URIs followed the same pattern as script URIs there would be no issue; a TWiki that used a protected pub would generate a script URI for %LINK, while an open pub (or one protected by an apache module) would expand it to a plain file URI.

I for one would prefer to hide raw URIs from the user. Aside from making the interface friendlier, it significantly improves the portability of topics, improves security, and allows better implementations of the pub store.

PS please focus on the design and user interface; the implementation of %LINK is trivial and should not be considered here.

PPS separating web and topic is pretty pointless, as topic="Web.Topic" still has to be supported for compatibility. I'd love to be able to deprecate web completely (after all, the paradigm isa file system, and you don't have to type ls dir=/etc file=hosts). But that's another discussion.

-- CrawfordCurrie - 04 May 2007

You don't want to hide URIs from the user, you want them to correspond with the stable, logical document structure. This might happen to map to storage or it might not.

As for interface, something like %LINK% sounds cool but it's not exclusive of extending %ATTACHURL%. The way Gordon has it is how someone would expect it to work, given previous experience with variables that reference other topics like %INCLUDE%.

-- DylanPartridge - 04 May 2007

You don't want to hide URIs from the user, you want them to correspond with the stable, logical document structure. This might happen to map to storage or it might not. - that's a very good point, and it has made me realise that I'm the one getting hung up on implementation detail. The reality is that stable, logical document structure suggests that the current viewfile URI is up the creek, because it isn't logical.

The current viewfile requires ?filename=file.png instead of another path component to represent the leaf attachment. A small point, perhaps, but a key one, because this construction is what prevents us from equating an unauthenticated pub URL such as with an authenticated on like https://twiki.org/cgi-bin/viewfile/Codev/AttachUrlForOtherTopics?filename=file.png. A small amount of recoding in viewfile would make it handle , which is far more consistent. And in that case I would drop all objections to the proposed %ATTACHURL extension.

-- CrawfordCurrie - 05 May 2007

Sounds like there is a bit of consensus. What do we need to do to move this to a committable state and get it into the release?

-- GordonTetlow - 07 May 2007

I find Gordon's proposal a good one, it is consistent with other variables, such as INCLUDE, METASEARCH etc. I would even go further and enhance it to optionally include also the filename. Same for ATTACHURLPATH.

Examples, all the same:

%ATTACHURL{web="Main" topic="WebHome"}%/foo.png
%ATTACHURL{topic="WebHome"}%/foo.png
%ATTACHURL{topic="Main.WebHome"}%/foo.png
%ATTACHURL{topic="WebHome" file="foo.png"}%

Outside of the scope of this feature, we could also add a rev="" parameter to point to attachents of a specific topic revision.

Outside of the scope of this feature, we can also enhance SCRIPTURL and SCRIPTURLPATH in a similar way.

I do not favor yet another LINK variable, it would duplicates functionality of existing variables.

-- PeterThoeny - 08 May 2007

I provided text for VarATTACHURL above. Some conditions:

1. Implemented as documented above
2. ATTACHURLPATH is also extended in the same way
3. viewfile is extended as I described, to use the path_info
4. %PUBURL(PATH)? documentation is rewritten to note the fact that %PUBURL%/Fred/FlintStone is identical to %ATTACHURL{topic="Fred.FlintStone"}%
5. Unit tests are extended to test the functionality (copy Fn_SCRIPTURL.pm to create Fn_ATTACHURL.pm)

-- CrawfordCurrie - 08 May 2007

Crawford, I do not understand how viewfile is related to ATTACHURL(PATH)?

What about the file="foo.png" parameter?

-- PeterThoeny - 08 May 2007

viewfile is related because it is the only way to control access to attachments using TWiki permissions. At the moment, if you want to link to an attachment, but switch between controlled and uncontrolled, you can't do it, because the URL format is different. This bites especially deeply when you find you need to apply access controls to a site where %ATTACHURL etc have already been used widely.

I silently ignored the file parameter because it does exactly what the discussion above says is not wanted viz. hides the construction of URLs. You can't have it both ways.

-- CrawfordCurrie - 08 May 2007

On "viewfile is extended as I described, to use the path_info", what I meant was that a viewfile enhancement should be treated as a separate enhancement. On viewfile, the current syntax should be preserved for compatibility. Not sure if filename in path_info is compatible with subwebs.

-- PeterThoeny - 08 May 2007

If filename in path_info is not compatible with subwebs, then %ATTACHURL%/file.png is a logical impossibility, as it assumes the filename in path_info. As I said, extended, not redesigned; the file parameter to viewfile should be deprecated only; usage will gradually disappear. The implementation is trivial; if viewfile is invoked with no file parameter, then it assumes the last component in the path is the attachment name.

I brought viewfile into the equation because it is the single biggest remaining gotcha in URL construction, and it needs to be fixed. For anyone is working in this area, that needs to be top priority, with %ATTACHURL enhancements very much lower down the scale. I'm trying to get everyone to understand the big picture, and not just this one corner.

-- CrawfordCurrie - 09 May 2007

I agree that usage if the file parameter will gradually disappear, as far as explicit links are concerned. But please don't deprecate. It is still useful for <form...> elements which pass the file name as input. Having both paths and parameters is in common use in view and other scripts, where the topic parameter, if given, overrides the path.

Use case: I have a topic with a gazillion of "trivial" (auto)attachments where the traditional attachment table would bloat the topic incredibly. With a crafted attachtables.dropdown.tmpl you see just one line with the dropdown menue, submitting the form gives the desired download.

-- HaraldJoerg - 09 May 2007

Good point.

-- CrawfordCurrie - 09 May 2007