Eyefood

2. Substructure

(Part one)

I’ve been meaning to write more about my little e-Portfolio tool for some time now, but it’s almost an unnecessary endevour. There are that many similar systems and sites across the whole of the Web 2.0 spectrum these days, that it almost makes no sense to outline what this one does — seeing as its functions are replicated in several other places.

Many people, for example, liken it (alarmingly so) to Flickr … and it’s very true that I’ve been very-much ‘inspred’ by Flickr in the way that a lot of the features of the site work. In a way, Flickr is almost the perfect model for an Art-School-Portfolio site like ours, and I’ve been very aware of the similarities all the way through coding it.

(Having said that, there are some features that are replicated in Flickr, like being able to make Group sets for example)

I have a very pressing need at some point to make an “Introduction to the Site” presentation — not for public consumption, but for the students who will be arriving in September. But in the meantime, I wanted to explain a little about what drives the site, and then follow it up with some screenshots.

…

I’m lucky, in a sense, that I have a very small and carefully-defined target audience for what I’ve made. In essence, it’s never going to extend beyond an active set of users of about five hundred or so. True, ex-students are an increasing part of what goes on on it, but even the most active ones tend to die down a bit in their engagement, and it’s mainly the Undergraduates and Postgraduates that are the focus of activity.

Because of that — and the lesser emphasis of scalability that seem to plague a lot of larger sites these days (*coughtwittercough*) — I’m able to use a very simple file-system for processing the work that gets posted.

Essentially, the main uploading part of the site places whatever was sent to it by a user into a common ‘queue’, to be processed by a series of little scripts — scripts that take advantage of some open source software: ImageMagick and ffmpeg (mainly, but there are others)

It’s like a system of ‘Watch Folders’ if you like — every minute or so, cron scripts check to see what’s in certain places on the server, and fire up if they find things. They move the files to other places, process them into web-ready versions of the original upload, and then move them all over the place (and update the database) when done.

It’s quite an elegant little system, if I do say so myself. There’s even a nod towards being able to expand the file storage if needs be in the future — with the site choosing a specific external server to store the files on.

It also means that it’s relatively simple to add different media-types to the site, just by finding appropriate open source (or, perhaps, not) tools that will squish formats from one to another, and process whatever the students might have on their hard drives into something more web-friendly. I have (for example) an experimental installation of wv — a Word-to-HTML convertor (which isn;t working actually, and I think I need to install AbiWord instead) — that used to take word files and automagically present them as web pages (after being run through tidy to get rid of Word’s hideous formatting)

…

One of the original features that I haven’t yet perfected (and so is still disabled) is the post-processing of moving-image work into Quicktime 7’s h.264 format. What used to happen (when the whole thing ran on my home servers) was that after upload, a quick-and-dirty MPEG4 would be spat out (after about 5-10 minutes), just to acknowledge that something had been uploaded. In the background though, the original file was moved to a second queue, and later that night (always at 3am — when I in theory wouldn’t be working on the machine that did the encoding) it was recompressed into a much better copy.

As I said a few weeks ago though, playing around with Compressor has given me the idea that it might be much easier to set up a render-farm using that, rather than relying on the (rather kludgy) QuickTime scripts that I had before … and it would also give me the options to easily convert the files into many more formats (iPhone anyone?)

…

Anyway, bleh. Rather than talking about each and every feature over the next few weeks, I’m just going to upload some screenshots and describe some of the functions…

Tags: cagd, development, quicktime, website