[Home]FileUpload

MeatballWiki | RecentChanges | Random Page | Indices | Categories
Possibility to submit binary files (images, pdf, whatever) to a wiki.

Attachment

Some wikis view these files as attachment to the wiki. A link is provided to upload a file, the file is stored in a directory dedicated to uploaded files.

examples: UseMod 1.0, TwikiClone, ElispArea on the EmacsWiki

Problems of this solution:

  1. The directory must accessible by the browser directly (this makes initial installation a bit harder)
  2. Alternatively, the images are served by the script, in which case the default expiry time of 10s used by some wikis would have to be increased significantly (1h?) for this to be useful.
  3. All the RecentChanges stuff needs to be duplicated, unless ordinary pages contain references to the files, in which case the solution is very similar to the one where encoded files are stored in files, the only difference being that the files take less disc space, and decoding is less resource intensive.
  4. File type checking -- if the script serves the pages, the file must be saved together with MIME type. Perhaps MIME types can be associated with a regular expression. Then we can save the file to disc, and run `file' on it and check the output against the regular expression.

Files as a Wiki page

Files are stored in an encoded form (MIME-encoded, uuencoded) in the normal page database, together with their MIME-type. Storing them on ordinary pages (mimencoded) would make reverting easy, and it would automatically produce entries on RecentChanges, do all the username/host/IP magic, and allow for edit summaries. It would also allow diffing, which makes no sense whatsoever, and viewing as mimencoded raw text, which also makes no sense.

To submit a new file, one just have to create a Wiki Page as usual and then choose the option to upload a file instead of editing a text as usual.

examples: TinyWiki, OddMuse

Linking to the attach files

Linking directly to the file in a Wiki Page is done by using:

Wikis usually have the ability to directly inline the files, ie insert the image in the page if the file is an image

Caching concerns

Binary files are bound to be bigger than the usual text submission. If the files are stored and served as ordinary files, your web server might already have a decent default setting that will handle caching for you.

If you keep images on the wiki, however, your WikiEngine will have to provide this information. If you allow people to upload new files, you want them to see the new files immediately, so having expiry times larger than a few seconds will confuse your users.

The solution is to react to the If-Modified-Since header that browsers send back to you on their next try, and immediately return a '304 Not modified' if appropriate. (This may require emitting an appropriate Cache-Control header on the outgoing side.) You've still got an incoming GET to deal with, but this at least saves the bandwidth of sending the whole (unchanged) file over again while letting updates go through immediately.

Here's a quick demo (in PHP) that puts out some headers that seem to work: 

        <?
        header("Content-type: image/png");
        header("Cache-Control: private, must-revalidate, max-age=0");
        header("Expires: Mon, 15 Jan 2001 00:00:00 GMT");
        header("Last-Modified: " . gmdate( "D, j M Y H:i:s", time() ) . " GMT");
        if(isset($_SERVER["HTTP_IF_MODIFIED_SINCE"])) {
          readfile("check.png");
        } else {
          readfile("nocheck.png"); // Cleared cache or first visit?
        }
        ?>

It's the value you send for "Last-Modified" that the user agent (ua) will come back with in an 'Is-Modified-Since'. To ensure that it asks every time (and will thus update when the file is changed), put 'must-revalidate' and a 'max-age' in the Cache-Control header. The 'private' will cause proxy servers to avoid caching the file while browsers still should; 'public' may be okay in your case, if every visitor should receive the same output. --BrionVibber

Without the max-age=10 info, the browsers may not revalidate the image (and thus not refresh). Look at the access log on the server. Does the browser do a separate request for the image? If not, the browser has decided that the cached copy is still fresh.

Debugging Caching Problems

These caching things are fiendishly difficult to debug, unfortunately. If you're using Firebird, I recommend installing the "[Live HTTP headers]" extension. It's a little easier than running a packet sniffer over your connections, but it won't help for tracking down issues specific to other browsers. --BrionVibber

Soft Security

SoftSecurity for files does not have to be the same as SoftSecurity for normal wiki pages. Here are few considerations specific to file uploads.

file size
A limit to the size of the attach file is a strong requirement to prevent DenialOfService attacks. The big temporary files not only suck up bandwidth, they will also fill up temporary disc space, possibly crashing your system.
file type
A limit to the kind of files people can upload can be used to prevent people from uploading virus-infected binaries. Usually the MIME type is guessed from the file-name on the server or supplied by the client while uploading. Both methods can be spoofed. A possible solution to this would be the more sophisticated file type detection by the command line tool file.
reverting changes
If someone upload porn images to a wiki, there must obviously be a way to remove it (unless your run a porn wiki).
easy of access
Since an uploaded file wastes so much more system resources, making this more difficult to do than editing text on a wiki seems reasonable. If newbies experiment with page editing, they are not wasting a megabyte or more of discspace.

Implementations

The WikiWay can work for images too. On WikiPedia we support uploads of images; people can of course upload porn or whatever, and other people can delete it. All uploads are logged and appear in both a log page and RecentChanges. When an upload is replaced, the old versions are kept available in an archive, with a single click to revert in case of accidental name collisions or petty vandalism replacements. As a speed bump, the upload capability is only available and advertised to users who've logged in. -- BrionVibber

UseModWiki 1.0 has a minimal file upload feature available only to "editors" (with a special edit password) and admins by default. (There is a setting to allow everyone to upload for private wikis.) -- CliffordAdams

OddMuse stores files on normal pages, MIME-encoded, together with their MIME-type. You can upload new files using the edit action, and download the files using the download action. The download action will display images nicely, for example, because the file is served with the correct MIME-type. Furthermore, editing such a page will automatically replace the textarea with a filefield. When saving, no diffs are saved. When browsing the page, the page content will be replaced with an img link inlining the image (via the download action), or if the MIME-type is not of type 'image/*', the page content will be replaced with an ordinary link to download the file (also via the download action). When editing a page, you can switch it from text-editing to file-uploading mode and back. -- AlexSchroeder

About linking to attach files

These are some

In the index file use a flag to indicate the type of page (text or embedding a file, possibly with its mime type) when rendering a link check this flag to see the type of the page and render the link accordingly ie :

I don't like the [image:animage] [link:text] because:

only to create or edit the attachment, you need to put another link to attach the file where you want, and keep this useless link to edit the page.

I like to use the same freelink link because:

for a free link.

adding a flag would make it easy to provide lists of files on the wiki

The benefits are:

Against this:

instead of the ? use two links (create | upload) to be able to go directly to the form you want. Though this may be to much.

Not related to upload I wonder if using a link like (create) would be better than the mysterious ? link --PierreGaston

Requirements

As far as I can tell, there are several use cases:

Therefore I think if we are talking about normal attachments, using normal link syntax is enough. This will take us to the source page, and there we can download the attachment.

For images, however, the requirements are slightly different. Therefore I think it is valid to add a new text formatting rule. After all, if the same formatting rule does different things depending on what kind of document is saved at the other end of the link, then that is difficult to understand as well.

However, a link that does different things depending on what kind of document is saved at the other end might prevent some common mistakes for newbies.

I don't like to change the pageidx file, because regenerating it now just requires us to examine the page directories , without actually examining the contents of the files. And since deleting it regenerates the pageidx, we can delete it often just to make sure that it gets regenerated correctly. It is resilient against failures. The new solution would be either expensive or brittle.

Sow's ears

A simple Alternative [to FileReplacement]: Anybody May Upload

Another similar idea from the EmacsWiki: It should be possible to upload programs to the wiki, and it should be easy: No ftp, no passwords. At the same time, it should be possible to undo the works

of vandals. A wiki page would have been great. However, a simpler solution was chosen because the wiki is there for the content, and a new file archive had to be created quickly.

The upload of files happens via a little Perl script.

It will save the file you upload immediately. It will also create a backup file if there is no backup file or if the existing backup file is older than two weeks. Furthermore, it will maintain a list of recent uploads where it uses your wiki username or your host name.

It will also compress any file that is larger than 20k, and if the file is bigger than 100k, only the compressed version is linked from the index and changelog pages. If the file size is between 20k and 100k, both the plain text and the compressed version are linked.

There is a new InterMap link such that Lisp:color-theme.el.gz will link to the correct file.

Therefore:

This is even more "Wiki" as it follows WikiWiki's one-backup EditCopy model, minus RecentChanges.

Discussion

MeatballWiki | RecentChanges | Random Page | Indices | Categories
Edit text of this page | View other revisions
Search: