We’ve had a lot of conferences recently, and two of them yielded videos! We’re hosting them on YouTube nowadays, so they tend to come in 10-minute chapters, hopefully that won’t interfere with the message too much.

First was BarCamp Antwerp, with Mathias Baert presenting in Flemish/Dutch an introduction to the Mine and Pymine:

Second was Alec Muffett at DJUGL London – this time in English, with a review of the project, goals, and pymine development up to February 2010:

We’ll add more videos and content when they arise!

Well, it’s been ages. Too long, really, and I apologise.

A lot has been going on with The Mine Project, notably Pymine development – the core codebase is being refactored and (I hate to put it like this) is almost back to the state of functionality where it was in December — however it’s now half the size, considerably more capable, and in a position to be grown in the future without being tragic.

The main changes were to the Django database model; by the nature of Django this was the first code that I wrote for pymine, and thus was the most complex and arcane in some senses; it is now considerably simplified, although it will help to send mail to the maillist if you’re looking to delve into it.

Last weekend we had out first code-camp, in Chelsea; Mathias Baert came from Brussels for an overnight stay, and was hacking JavaScript all weekend; Ketan worked with Mathias, Adriana and me all saturday afternoon and evening, and after considerable thrashing-out of some open questions, the eventual UI will be in a much better state for users.

The development UI, OTOH, is still a bit rough. Mathias and I are trying-out a new JavaScript tagging widget which will make life better.

Upcoming news:

• TheMineProject will be presented at DJUGL – the London Django User Group, on Thursday February 25th
• Mathias will be presenting on TheMineProject at BarCamp Antwerp on March 6th

More news as time warrants; I will try to update more frequently, I promise.

Just posted at my blog:

Stripmining The User: DataPortability, The “Pragmatic” Web, And A Bad Philosophy

A little too ranty to post here, I feel, but pertinent.

My friend Jim wrote, regarding Google’s Chrome OS:

It’s an interesting idea – a distro where the only app is the browser – but it’s not really anything new; it’s just another iteration of the thin client idea. Of course, it might be the right iteration this time.

It also rather goes against a lot of the ideas people like Alec and Adriana are coming up with – ideas of owning one’s own data are rather scuppered by every application running on some anonymous server somewhere. Maybe, after a while, users of Chrome OS will start to buy UI-less home servers to run their apps on and store their data.

Maybe we need a UI-less distribution of Linux, running Apache (or whatever) and a whole bunch of open source webapps – word processing, spreadsheets – and, of course, a Mine! server.

…and I responded in the comments, but alas his homebrew blog software ate my formatting:

ideas of owning one’s own data are rather scuppered by every application running on some anonymous server somewhere

Unless it’s yours

I’m not worried by more and more browsers; I am watching my colleagues realise that all the blogs they have on blogs.sun.com are rather a single point of failure and that their continued publication is at the whim of Larry Ellison. Suddenly people are really really interested in the means of exporting and transporting their blog to another platform, which would not be an issue was the software under their control. I’ve had three people say they now understand why I’ve always been blogging elsewhere…

As a counterpoint, go browse “Opera Unite” which is really interesting; presented as a way for you to take back ownership your data, it is in fact a truly remarkable way for Opera to host (read: “cache and serve”) your data on the Web in a way that gives them a degree of control; as an Eminence Grise for the “empowered user” it would present an interesting attack on Google’s stranglehold of free services and cloud data.

All theMineProject seeks to do is go a step further and expunge the middleman.

a distro where the only app is the browser

Anyone here remember HotJava?

I like the idea of more browsers. The browser is the new end-user platform – anyone who’s not realised this yet has not been paying attention.

The important thing is to maintain diversity this time around, and not let a single vendor take 90% of the installed base.

Back in April, when Adriana and I visited the Bay area, we did a lunchtime talk about the Mine! project at Google. Here is the second part; apologies for the delay in posting it, there were more slides to sync audio against, in my half.

Hi All,

If you want to know more about The Mine Project and are on the west coast of the USA, you may be interested in the first West-Coast Mine Developer Meeting:

West Coast Mine Developer Meeting
Sunday 19th April
5:30pm for a 6:00pm start
Mountain View CA

Location details are currently secret – numbers and space are limited – but if you want to know more please mail alec.muffett@gmail.com and I’ll get back to you ASAP.

If it proves very popular, we may shift the location, hence not wanting to commit to it in ASCII.

Thanks!

UPDATE: ps: we’re planning on doing dinner somewhere on Castro St shortly afterwards.

Hi All,

There will be a meeting of the Mine Developer community in London, at a secret location, at 7pm on Tuesday the 10th of March.

What I would like to do is show off the current state of the Mine code:

• what it currently can and cannot do
• where you can access a running mine
• how you can run your own
• how you can get involved in enhancing it, plus adding tools and applications
• what the software roadmap is

…plus get attendees’ input regarding “gaps” – stuff that needs to be done or written in order to enable a community of developers to come aboard.

If you are interested in attending, please mail alec dot muffett at gmail dot com before sunday; pizza will be on the menu.

I like the ReST interface, but I am not fundamentalist about it.

Most of the objects in the Mine implementation are “Things” – ie: a series of key=value pairs which are bundled together under a numeric identifier, which have a Class and associated methods that operate on them.

For reasons of security and sheer implementability, “Things” may not be updated under the ReST CRUD scheme; to recap ReST you are allowed four “CRUD” operations against a URL which represents an object:

• Create (a new object, in a directory, say)
• Read (an existing object)
• Update (an existing object)
• Delete (an existing object)

So for example:

• CREATE (key=value …) /api/object.xml
  -> new object populated with keys/values, id = 42
• READ /api/object/42.xml
  -> a bunch of key=value pairs in XML
• READ /api/object/42.json
  -> a bunch of key=value pairs in JSON
• DELETE /api/object/42.xml
  -> deletes object 42, status returned in XML

…but the one that gives problems is UPDATE since the presumption is that you will atomically replace an old Thing with a new thing, much as:

mv new.jpg current.jpg

…might work in a filesystem; you are swapping one bunch of bytes for another one; but Things are complex structures with some bits you can poke, and other bits you cannot. Merely splatting them with replacement data would be painful.

Amongst the other problems with this is that “Update” is usually mapped to the HTTP-PUT method, which is badly implemented in web browsers and actually I think is below the tipping point – ie: it’s so badly implemented that ReSTful people work around it rather than get it fixed. Standard Perl::CGI for instance, does not support it.

The way I have gotten around this is slight but elegant piece of doublethink; I have implemented a ReST interface atop each Thing object, to access its key=value pairs:

• READ /api/object/42/key/keyName.xml
  -> value associated with keyName for object 42, in XML
• READ /api/object/42/key/keyName.json
  -> value associated with keyName for object 42, in JSON
• DELETE /api/object/42/key/keyName.xml
  -> unset keyName in object 42, status in XML

…and then realising that object 42 now must exist for this trick to work at all, and that there is no point in having CREATE “choose” a key name for you – predefined variable names do not fit the ReST-CREATE model – then the key/CREATE and key/UPDATE operations are otherwise functionally identical (ie: they poke the values of keys) and therefore the latter can be dropped. Also key/CREATE is cooler, since it can poke multiple keys at the same time.

Further, that means key/CREATE is functionally identical to Thing/UPDATE so that can be dropped, too.

So all that is necessary is to mentally “rebrand” the key/CREATE operation as the Thing/UPDATE operation, and a whole pile of UPDATE operations go out of the window.

So now there is only 1 ReST-UPDATE operation that remains in the Mine – down from 11 – and that is the “update of auxilliary data”, for example replacing one JPEG that is associated with object 42, with a different JPEG.

To me this is a straight swap of one blog of data for another one, and so should remain as a ReST-UPDATE; but anywhere else that I have a ReST interface onto complex objects, I shall in future consider very carefully before implementing a UPDATE method.

Or some of the reasoning, at least…

The code is still pre-alpha – there is a lot of functionality going to be added over the next few days – but last night we got the first cut of the Protomine developer kit going.

So if:

• you are interested in The Mine! Project
• you have OSX (current development platform, Ubuntu and Solaris next)
• you know what Subversion, perl, and /usr/bin/make do

…then join the Developer list and help!

Alec