Ye Olde Clue

Random musings on random stuff.

guest, Login, All Changes, Home

Synopsis Box

When you move your mouse over the headlines on the boxes, this synopsis box will update.

If you were 7 again...

Posted: Michael, 27 Jun 2010 at 18:28
Comments: 0

If you were 7 again, what would you expect to find in a book on beginning programming? I have some thoughts on this, and going to do this, but I'm curious to the thoughts of others.

Python Magazine is dead ?

Posted: Michael, 20 Mar 2010 at 21:08
Comments: 9

For the past several months the python magazine hasn't sent any new issues out. Indeed, since late last year they ripped out their website and had a banner saying "We're busy building a new python magazine", with a link laughably suggesting that there is more information available. This is after last year them getting several months behind with the magazine. They also said " Don't worry—your subscription and back issues are safe and will be available when the new site launches. ". That's fine, in theory. However consider:

  • Whilst they may let the grass grow under their feet they haven't bothered telling their subscribers. Paid subscribers.
  • They haven't bothered updating their website telling their customers what they're doing.
  • Indeed, they appear, from a subscriber point of view, to have simply cut and run.
I can't actually think what excuse they can come up with that justifies not actually bothering to contact subscribers for well over 1/2 a year, but I'm sure they have one. On the flip side, they don't have any contact address on their front page, nor on their content free "what we're doing" page. Beyond this, last year they decided, of their own volition to charge my credit card to renew my subscription. Now I was going to renew anyway - it's been a great magazine in the past, but charging my card without upfront consent struck me as rather dodgy.
Since they've now reneged on their half of the sale contract and not delivered, and actually have a good reason to need to get in contact with them, I can't. This means I'm left with 2 choices:
  • Either put out a public notice in the hope that it's something that someone there will read, and actually get back in contact to let me know how to contact them
  • Contact Visa and say that they're a rogue trader, and that they should be banned from making any further transactions against my card (especially given the last one was done without my explicit consent.
Neither is particularly attractive, and hopefully someone knows how to get in contact with them because they sure aren't advertising any contact details right now.
Kamaelia components from decorated generators. Pythonic concurrency?

Posted: Michael, 4 Oct 2009 at 22:52
Comments: 12

A few months ago, there was a thread on the then google group python-concurrency about some standard forms for showing how some libraries deal with concurrent problems. The specific example chosen looked like this:

#!/bin/sh
tail -f /var/log/system.log |grep pants
Pete Fein also posted an example of this using generators, based on David Beazley's talk on python generators being used as (limited) coroutines:
    import time
    import re
Restarting Python Northwest. 24th Sept

Posted: Michael, 14 Sep 2009 at 23:36
Comments: 3

Traffic Server to be Open Source?!

Posted: Michael, 7 Jul 2009 at 12:02
Comments: 0

If this happens this will be awesome. Traffic Server is some really nice code. It's a large codebase, but it's really cool, and it *scales*. (I used to work at Inktomi, so have been inside the code as well). For those that don't know what it is, it's a very high performance web caching proxy, with a plugin architecture, allowing for the addition of other protocols. It used to support HTTP (& obvious friends), NNTP, RTSP, RTP, WMV, etc.

Europython Videos Transcoding

Posted: Michael, 7 Jul 2009 at 01:02
Comments: 1

Since I've had a few questions about this, a short status update. At Europython last week I was recording all the talks I was attending. Including the lightning talks this means I have video from 55 talks. The video files from the camera are too large for blip.tv, so I'm transcoding them down to a smaller size, before uploading them. Since these 55 talks are spread over nearly 80 files, that naturally takes time.

Autoloading in python

Posted: Michael, 21 Jun 2009 at 16:14
Comments: 10

Before I started using python, I'd used perl for several years, and one thing which I'd liked about perl was its autoload facility. Now in python the closest equivalent that I've seen is __getattr__ for classes, but not __getattr__ for a module. This seemed like a real shame since there are times when autoload can be incredibly useful.
If it seems chaotic, consider the Unix PATH variable. Any time you type a name, the shell looks in lots of locations and runs the first one it finds. That's effectively the same sort of idea as autoloading. Yes, you can do some really nasty magic if you want, but then you can do that with the shell to, and generally people get along find.
Anyway, vaguely curious about it I decided to do some digging around, and came across this post by Leif K Brookes, which suggests this:

You could wrap it in an object, but that's a bit of a hack.
Europython

Posted: Michael, 10 Jun 2009 at 17:35
Comments: 0

 I've mentioned this in a couple of places, but mentioning on my blog seems appropriate too.

New Tool for helping debugging Kamaelia Systems

Posted: Michael, 8 Jun 2009 at 18:17
Comments: 2

I've added a PythonInterpreter component to Kamaelia in Kamaelia.Experimental.PythonInterpreter. The primary use for this is intended to assist with debugging, but there are other possible uses.

Use in a system

This allows a variety of things, from a basic command line console:
StandaloneInterpreter().run()
Which you can run in the background on any console. For example you could do
this:
    ServerCore(protocol=myprotocol, port=1234).activate()
    StandaloneInterpreter().run()
Alternatively, you can use an embeddable component that speaks to inbox/outbox rather than stdin/stdout. Crudely you can do something like this:
    Pipeline(
        ConsoleReader(),
        InterpreterTransformer(),
        ConsoleEchoer(),
    ).run()
But you can also put it inside a pygame application, reading & writing from a
Textbox/TextDisplayer:
    Pipeline(
        Textbox(size = (800, 300), position = (100,380)),
        InterpreterTransformer(),
        TextDisplayer(size = (800, 300), position = (100,40)),
    ).run()
Going from generator coroutines to Kamaelia Components

Posted: Michael, 17 May 2009 at 02:09
Comments: 0

Earlier this evening an announcement by Pete Fein regarding the formation of a python-concurrency mailing list (aka Python Concurrency Special Interest Group) bounced past my inbox in the unladen-swallow mailing list. Naturally, given my work on Axon (a concurrency library) and Kamaelia (a bunch of components that use it), it jumped out at me as interesting. (5 minute overview for those that don't know what Kamaelia is...)

Bar Camp Leeds UK

Posted: Michael, 5 May 2009 at 21:56
Comments: 0

Bar Camp Leeds (UK) is running for its 3rd year running. It'd be great to see other Python people there if you're around. Details:


Schedulers matter

Posted: Michael, 4 Mar 2009 at 10:26
Comments: 2

Improving the scheduler. It's been something we've avoided for quite a while in Kamaelia, but Simon Wittber's recent post benchmarking Fibra vs Kamaelia vs stackless is interesting. His key showing that "Stackless is 7x faster than Fibra, and Fibra is 10x faster than Kamaelia" are cool for him :-) ... and naturally led me to think "why". Looking at the code, it struck me that he's doing something more interesting with the scheduler given these code forms:

scheduler.install(self.messageLoop())
# self.MessageLoop is a regular python generator
...
            yield self.incrementCounter()
            yield kickTo.messageQueue.push(self)
If you delve inside the fibra scheduler (which doesn't appear to be here unexpectedly) you see the following core:
    def tick(self):
        """Iterates the scheduler, running all tasks and calling all
        handlers.
        """
        active = False
        for handler in self.handlers:
            active = handler.is_waiting() or active
        active = (len(self.tasks) > 0) or active
        tasks = []
        while True:
            try:
                task, send_value = self.tasks.pop(0)
            except IndexError, e:
                break
            try:
                if isinstance(send_value, Exception):
                    r = task.throw(send_value)
                else:
                    r = task.send(send_value)
            except StopIteration, e:
                r = e
            if r is None:
                tasks.append((task, None))
            else:
                try:
                    handler = self.type_handlers[type(r)]
                except KeyError:
                    raise RuntimeError("Don't know what to do with yielded type: %s" % type(r))
                handler.handle(r, task)
        self.tasks[:] = tasks
        return active
The core of this appears to be "when I'm done, do this later" next. If you think that's familiar, it should be - its very similar (at least conceptually) to what twisted does with deferreds. It's not identical, but similar. So what does this mean for the hacksack demo? Well, if we look at self.tasks after each run, by changing:
    def run(self):
        while self.tick():
            pass
to:
    def run(self):
        while self.tick():
            print "TASKS", [x[0] for x in self.tasks]
It becomes apparent what's happening (though it's fairly obvious from above):
TASKS [<generator object at 0xb7b04fcc>]
TASKS []
TASKS [<generator object at 0xb780f94c>]
TASKS []
TASKS [<generator object at 0xb7a1b04c>]
TASKS []
TASKS [<generator object at 0xb776fcac>]
TASKS []
TASKS [<generator object at 0xb79327ac>]
TASKS []
TASKS [<generator object at 0xb7b0ff2c>]
TASKS []
Fundamentally, the reason it's quicker for two reasons:
  • It always knows which generator is ready to run next.
  • It also defaults to pausing the generator, unless it specifically asks to be run. ie the tasks default to being passive.
  • The sender knows who the receiver is. This allows the sender to schedule the receiver explicitly.
These two points matter because Axon's scheduler (from jesse's post, hopefully people are aware that Axon is the relevant part of kamaelia here):
  • Is effectively a round robin scheduler - essentially for simplicity. The key question we wanted to ask was "does a component model based around inboxes/outboxes make it easier to make easier to write/reuse concurrent software", rather than "how can we build a faster, more responsive scheduler". As a result the round robin scheduler was always a compromise. There's a little bit of intelligence in there, but not much.
  • Kamaelia's components default to active, rather than passive. That is they're expected to run continuously unless explicitly paused. This design decision impacts the scheduler as a whole.
  • The sender does not know who the receiver is. This requires something different to happen in the scheduler. On the upside, it means that code can be tested easier in isolation, or reused in situations it wasn't originally expected to be used within.
This leads me to the rather obvious solution here - can we rebuild a better Axon scheduler by reusing fibra and throw away our scheduler? If so that would be really neat - throwing away our scheduler for something faster and more intelligent would be rather fun :) If we can't then we've been pondering improving it - that is making it more intelligent - for a while. Fibra's benchmarks suggest that doing so would be well worth it. The question this raises though is whether doing this would help us in the general case. At present I'm unclear on that, but until you try, you don't know.
The 5 minute (version), not the full half hour...

Posted: Michael, 26 Feb 2009 at 23:15
Comments: 5

As you may know Kamaelia is a project I'm rather familar with (ok understatement), but one of the key ideas behind it is about how to make concurrency usable for everyday problems (from greylisting, database modelling assistance, though to learning to read & write). No one concurrency method suits everyone - as Jesse's recent series of posts shows (if you've not read them they're well worth it :), but Kamaelia's approach fits my head, and hopefully yours too. There may be a few areas where nicer syntactic sugar may be appropriate and so on, and feedback is always welcome - preferably to the project's google group/mailing list.

OK, the point of this post is that recently at O'Reilly Ignite UK North I gave a 5 minute talk which was essentially "what we've learnt about practical concurrency", and by necessity of time it's a whistle stop tour. The slides are on slideshare, and the video is on blip.tv - along with the rest of the talks - and by the magic of the internet they're below as well.

Embracing Concurrency (slides)

Embracing concurrency for fun utility and simpler code

Embracing Concurrency (video)

Actor Model vs Denial of Service

Posted: Michael, 17 Nov 2008 at 03:39
Comments: 24

Kamaelia implements something similar to, but not the same as the Actor Model. Since I'd not heard of the actor model for the majority of time whilst working on Kamaelia (kamaelia is more like occam, unix pipes, hardware, etc), I've been reading up on it. One thing that I've come across about it that suprises me is this:

Communications among actors occur asynchronously: that is, the sendingactor does not wait until the message is received before proceedingwith computation.
(this summary courtesy of the Wikipedia page on the Actor Model, but the literature supports this view as well)
The difference between Kamaelia and the Actor Model is in a few places, but possibly the most fundamental is this:
  • With the actor model (as I understand it)
    • You have a mailbox where you receive messages. ("recv from inbox")
    • communications defaults to unidirectional, though you can build bi-directional
    • When you send a message, you know the recipient and send it direct to the recipient. ("send to bob")
    • Message sending is asynchronous
    • ie sender knows receiver, receiver does not (necessarily) know sender
      • This potentially introduces coupling in ways that makes co-ordination languages harder to build
  • With Kamaelia's model:
    • You receive messages from multiple named inboxes (ala receiving data on stdin, or receiving signals)
    • communications defaults to unidirectional, though you can build bi-directional
    • You send messages to named outboxes (ala sending to stdout, stderr)
    • A higher level co-ordination language (effectively) joins the dots between outboxes to inboxes (generally)
    • Message sending defaults to asynchronous, but you can define a "pipewidth" or "max number of messages in transit" to allow synchrony, where needed (such as a producer that produces data faster than the consumer can consume it)
    • ie sender does NOT know receiver, receiver does not (necessarily) know sender
      • Much like a CPU doesn't know whether it's plugged into a motherboard or a testing harness for example.
      • This defaults to needing a co-ordination language - but this encourages greater reusability of components, through a dataflow model
    • I say "kamaelia's" model here, but this is pretty similar to hardware, unix pipes, etc.
Now the thing I can't tell by looking at literature is what the general case is for most actor systems in practice:
  • Do they do the literal, and solely asynchronous thing? (ie always return immediately without error when you send a message)
  • Or do they introduce the ability to add in synchrony where necessary? (block or return an exception when a maximum inbox size is reached)
The reason I ask is because if you don't have the ability to define pipewidths, maximum inbox sizes, maximum outbox sizes or something then you can easily cause a denial of service attack in that scenario by having producers overload consumers. Consider a frame grabber feeding a slow, experimental video codec. In that scenario, it becomes rather important to be able to have a form of blocking (or EAGAIN) behaviour available. Indeed, this is such a common issue, that it's why unix pipes & filehandles will buffer a small amount of data, but block if you send too much data (or exhibit EAGAIN behaviour :). Similarly this is what's behind the socket.listen(argument) call in a server - to allow a certain number of connections to queue up, before refusing connections...
Simple Chat Server in Kamaelia

Posted: Michael, 13 Nov 2008 at 11:00
Comments: 4

On the #kamaelia IRC channel , the issue was raised that a number of the examples on the Kamaelia website are just that, small examples, perhaps too small to be useful beyond illustrating the basic idea. (I'm not sure this is universally true, but I accept its definitely the place in more places than we'd like!). Now, one thing we've got planned for the december release is to include a lot more examples, and as a result I'm interested in hearing what sort of examples people would find useful. For example, the example mentioned on the channel is "what does a simple chat server look like" ? (and hence why I'm interested in what examples people would find interesting/useful)

A post per day in November

Posted: Michael, 2 Nov 2008 at 14:38
Comments: 0

Interesting new meme, I think I'll jump on this bandwagon. For my first post in this vein, I ought to retroactively do November 1st's post. So, in that spirit of catching up, I'll post the following short presentation I wrote to answer a query posted on my blog a few days ago. As it indicates, it's a walkthrough about what happens inside a particular, simplilfied layered protocol written in Kamaelia. It's probably worth noting that every "box" in the diagrams is a separate component, meaning it runs concurrently with other components.

Kamaelia Protocol Walkthrough
View SlideShare presentation or Upload your own. (tags: concurrency python)

The reason for it being that form is because it is significantly clearer than text.
The world is big enough

Posted: Michael, 30 Oct 2008 at 14:35
Comments: 1

In "Actors, concurrency and Kamaelia" Jesse Noller writes:

"I believe there is room within Python as a language, and CPython as an implementation of that language - for a virtual buffet of helpful libraries for concurrency and distributed systems."
He's absolutely right - if we only have hammers, every solution involves rnails. The more that we build these things, the quicker we'll end up with a healthy eco-system trying out various approaches. I would hope that would lead to something similar to WSGI - something enabling various different concurrency ecosystems to live side by side happily. Naturally I think the miniaxon approach of component+boxes is close, but then I would -- I'm biassed. No single metaphor matches every head, but it'd be really nice if the various approaches could interoperate nicely. The more we all try different approaches, the more likely we are to hit something that "works" for the majority of python developers.

I'd write more, but fluey at the moment.

Jesse's comment did remind me of this though:

"The best way to have a good idea is to have a lot of ideas." Linus Pauling.
It also reminds me of my personal view on things like this - the world would be very boring if we all agreed and nobody ever did anything new or different.
Bar Camp Liverpool

Posted: Michael, 26 Oct 2008 at 23:42
Comments: 0

Bar Camp Liverpool has been announced, for those interested. For those that have never been to an unconference, they're fundamentally as interesting as
the people who attend, and Pythonistas are an interesting and varied lot, hence posting this here. Key info:

If there's interest, and if it seems appropriate, could hold a Kamaelia sprint that weekend ?

Planning Kamaelia 0.7.0 release

Posted: Michael, 24 Oct 2008 at 00:24
Comments: 0

No sooner do you put a new release out the door and you're reviewing things and planning the next. Since there seems to be a general trend towards numbering releases based on date rather than on (equally arbitrary) release numbers, we've decided to follow that trend. However, we've also planned to fill in the gaps to figure out how to manage shorter release cycles. Also, I don't want to artificially jump from version <1 to version 8, since it would very much say the wrong things.

When we hit version 1.0 I really want certain things in the system which aren't fully there yet, though most of the necessary spikes are in place). As a result, rather than using the versioning of YY.MM, we're going to move to using Y.Y.MM by the end of the year. That implies we hit version 1.0 in early 2010, which is actually only about 14/15 months off, so that seems reasonable. (hence the reason for switching to date based releases)

This means our next release will be 0.7.0, followed by 0.8.12. From then on we'll be moving to 6 weekly release candidate cycles, with a decision made at the beginning of each cycle whether that release candidate will be targetted as a full release, but there will definitely be release candidates. Given the delay between 0.5.0 and 0.6.0, jumping to a 6 weekly cycle seems challenging, so that's part of the reason for aiming for 2 releases by the end of the year (to get into the habit :).

As a result we're currently planning what's going into 0.7.0 now, with a planned feature merge freeze weekend 22nd November, and planned release date of weekend ending 30th Nov. That is intended to be followed up in late december by 0.8.12.

I've started a discussion on the mailing list about these, but the current planned focus for 0.7.0 follows.

  • SuSE packaging for Axon & Kamaelia updated from 0.5.0 to K0.6.0 & A1.6.0
    • And future packaging changes automated, ideally. (enabling 0.7.0 easily)
  • Better graphline shutdown as discussed on the list.
  • Tyson's extended version of the file appender,
  • Merged of Chong's Google Summer of code project - 3D visualisation work (since this is something I actually need sooner rather than later)
  • Packaging up of the whiteboard (pdf) as a standalone app
  • Packaging up of the batch file processor (image/video transcoder) as a standalone app (or better packaging than this)
  • A mirror of the Ben's Kamaelia AWS code into Kamaelia.Apps.AWS, if it's at a stage where it's ready. (aside from reviews etc)
  • Merge of Jason's google summer of code on on extensions/improvements to the HTTP server code, including a better example of the seaside like functionality. (I think Kamaelia Publish itself should probably wait until 0.8.12 or 0.9.X) Probably after it gets a clearer name.
  • Perhaps initial work on integration of the STM code into the CAT. (The what? The STM and CAT tools were discussed at pycon uk) Though I suspect this will get started now, and merged in 0.8.12
  • 2.6 cleanups (probably based on hinting from 2to3), and work started on a 3.0 autoconversion/build system.
  • Other stuff I'd like to see includes : work started on rationalising Kamaelia.File, Full review and merge of UDP_ng code in place of current UDP code, basic "connected" UDP server support (perhaps) (ie such that it can be used as a drop in replacement for TCPServer in ServerCore)

Any suggestions and/or improvements or offers of help welcome :) Whether all this is doable in the time frame is a little speculative at present, but as the project's reaching maturity, it seems appropriate to take a more predictable approach towards releases, even if a release focus is "just" bug fixes, or documentation :)

New Release: Kamaelia 0.6.0

Posted: Michael, 22 Oct 2008 at 12:12
Comments: 0

With great pleasure I'd like to announce the release of Kamaelia 0.6.0 and Axon 1.6.0

Release overview

For the short of time:
Overview: library/framework for concurrency using message passing components as the concurrency metaphor. Consists of a kernel (Axon) and collection of components (Kamaelia). Support for generator, thread & process based components. Targetted towards supporting maintenance, so /accessible/ to novices, but general purpose. Uses concurrency to make life simpler, not harder. Designed as a practical toolkit.
Download:
http://www.kamaelia.org/GetKamaelia
http://www.kamaelia.org/release/Kamaelia-0.6.0.tar.gz
sudo easy_install Kamaelia  
Change Summary:
Major update, multicore, STM, easy concurrency, creation of Kamaelia.Apps namespace for reuse of applications, significant amounts of new functionality, major documentation improvements (including full offline reference docs), support for using Kamaelia components cleanly in non-Kamaelia apps. (ie a clean linear -> concurrent interface (Handle))
Release notes: http://www.kamaelia.org/ReleaseNotes060
Deltas: Kamaelia 0.5.0 -> 0.6.0, Axon 1.5 -> Axon 1.6.0
Last full release: October 2006
Kamaelia @ Linux Expo Live, London, Olympia 23rd Oct

Posted: Michael, 17 Oct 2008 at 00:48
Comments: 0

Just a note for anyone in the london area - if you're interested in kamaelia or what we've learned about making concurrency easy (or much easier) to work with and useful for simplifying common tasks or want hear a talk about it, or just meet up, grab food and chat about kamaelia or python or related stuff, I'll be giving a talk at Linux Expo Live London next week at Olympia in London. Date: 23rd October, time 2:30pm.

Kamaelia AWS (amazon web services)

Posted: Michael, 15 Oct 2008 at 09:57
Comments: 0

Many thanks to Ben Ford for the heads up and release of their code using kamaelia that accesses amazon web services.  His blurb from the email:

Ben Ford wrote:
I've set up a googlecode project with access to the code I've written
to interact with Amazon Web Services here:  http://code.google.com/p/kamaelia-aws/.
I've set up source control here: http://freehg.org/u/ben/kamaelia-aws/.

It's my first time using kamaelia so I'd love to hear any feedback on
the code. Feel free to use it, and let me know what you think.
And from the two links:

Concurrent access to AWS

This project came about when we needed to integrate SQS and S3 into our web application.

It'sreleased here to share with the community in the hope that it's usefuland for anyone familiar with kamaelia to pick apart, or use for demopurposes.

What does it give you?

We have several components at varying levels of abstraction.

SQS

Components to handle passing json messages into and out of SQS

S3

Components to upload to S3, download from S3 and delete from S3.

At higher levels we have a few Graphline based components that integrate various different components with a logging module

Assuming no major issues, planning on that going in 0.7.0, due in mid november, mirror to trunk probably this weekend :-)
Interesting project

Posted: Michael, 8 Oct 2008 at 09:20
Comments: 3

Interesting project: http://yieldprolog.sourceforge.net/ noting here since it's somewhere I'll come back to, and it strikes me that others would find it interesting too.

Walk through of how to build a simple protocol using Kamaelia

Posted: Michael, 7 Oct 2008 at 02:23
Comments: 2

A question was raised on the Kamaelia list asking for a demo on how to nest protocols using Kamaelia. In order to show some clarity, and avoid a protocol mismatch I decided to build a protocol which is a composite of the following 3 things:

  • Something clumping data into messages
  • JSON data being forwarded in serialised form as those messages
  • The serialised messages encrypted (naively) using DES encryption.
Rather than repost all the steps leading to that here, you can find the post that does just that here:
You'd probably want to take the resulting code a little further than I've taken it if you were to use it in production, since (for example) DataChunker is designed to be used with larger & larger numbers of messages. It would be relatively simple to transform this to a pub/sub system/router for sending recieving Javascript fragments, but wanted to leave the example looking relatively simple for now. I may come back to that as a pub/sub demo.
Any sufficiently advanced...

Posted: Michael, 1 October 2008 at 12:14
Comments: 0

Any sufficiently advanced...

Not impressed with changes to shell service on sourceforge...

Posted: Michael, 19 Sep 2008 at 17:08
Comments: 2

So, sourceforge are changing their services. Can't really complain since it is free, but it really is pain. They've switched off shell access, and broken the web servers (our home page there now shows this:

Traceback (most recent call last):
  File "/home/groups/k/ka/kamaelia/cgi-bin/Wiki/wiki", line 24, in ?
    import pprint
  File "/usr/lib64/python2.4/pprint.py", line 39, in ?
    from cStringIO import StringIO as _StringIO
ImportError: /usr/lib64/python2.4/lib-dynload/cStringIO.so: failed to map segment from shared object: Cannot allocate memory
And there's no way (that I can see) to change the website to point at the main website now. We were building/readying a replacement server, but hadn't quite got there, so weren't putting redirects in place yet. Ho hum. I've updated the link on http://sourceforge.net/projects/kamaelia to point at the new server, but given we've moved SVN elsewhere, mailing lists elsewhere and the webserver elsewhere I think this final breakage by SF marks the end of the road for use with sourceforge.
Shocking...

Posted: Michael, 31 Aug 2008 at 00:37
Comments: 7

And people wonder why I have no desire to visit the US any more : http://www.salon.com/opinion/greenwald/2008/08/30/police_raids/index.html

Hope for the copyright system?

Posted: Michael, 20 Aug 2008 at 21:55
Comments: 0

"Open source licensing has become a widely used method of creative collaboration that serves to advance the arts and sciences in a manner and at a pace that few could have imagined just a few decades ago" -- The U.S. Court of Appeal for the Federal Circuit

Unit test crib sheet

Posted: Michael, 7 Aug 2008 at 12:13
Comments: 0

Mainly for me, but hopefully of use to others too:

#!/usr/bin/python
#
# running this as "test_CribSheet.py -v "
# - gives you some cribsheet docs on what's going on and runs all the tests
#
# running this as "./test_CribSheet.py -v LikeCycleOfATest"
# - allows you to just run one of the suites.
#
# This doesn't replace documentation, and there's probably some hidden
# assumptions here, but it's quite useful.
#

import unittest
import os

class DemoDocStringsImpact(unittest.TestCase):
    # Note that the next test doesn't have a doc string. Look at the results in -v
Kamaelia Talks at Pycon UK, September 12-14th Birmingham

Posted: Michael, 28 Jul 2008 at 01:15
Comments: 0

My talk suggestions have been accepted by the committee [*] and so I'll be giving the following two talks at Pycon UK:

George Bernard Shaw was wrong

Posted: Michael, 27 Jul 2008 at 14:00
Comments: 1

... or rather incomplete. This quote is often used as a good rationale for sharing ideas, and as a nub of an idea it's good, but incomplete:

If you have an apple and I have an apple and we exchange apples then you and I will still each have one apple. But if you have an idea and I have an idea and we exchange these ideas, then each of us will have two ideas.
It's nice and on a most basic of levels it's true. However it's utterly incomplete as anyone who's worked on anything based on sharing ideas, be it brainstorming, collaborative working, open source or anything else. It's actually more a combinatorial explosion of ideas you get and in fact with just two "completely atomic" ideas you never have just 2 ideas, you always have at least 3 - A, B, meld(AB). In fact this sequence should bring flashbacks to many people to their maths:
Concurrent software is not the problem - Intel talking about 1000s of cores

Posted: Michael, 5 Jul 2008 at 12:47
Comments: 5

Intel have recently been talking about the fact that we'll probably have to deal with not hundreds but thousands of cores on a machine at some point in the near future (5-10 year time frame based on their 80 core prototype). Now many people seem to be worried about how to use even a dual core machine, so naturally many people are probably going wa-HUH? However I suspect the Erlang crowd have a similar reaction to us - which is "cool".

Why? Like the erlang group, in Kamaelia, the thing we've focussed on is making concurrency easy to work with, primarily by aiming for making concurrent software maintenance easier (for the average developer). In practical terms this has meant putting friendly metaphors (hopefully) on top of well established principles of message passing systems, as well as adding support for other forms of constrained shared data. (STM is a bit like version control for variables).

We've done this by using various application domains as the starting point, such as DVB, networking and use of audio/video etc, and used Python as the language of choice to do so (Though we probably could've shouted about our application uses more/better, though we've getting better I think :-). However the approaches apply to more or less any non-functional language - so there's a proof of concept of our miniaxon core in C++, Ruby, & Java as well. (C++ & ruby ones deliberately simple/naive coding style :)

This does mean that now when we approach a problem - such as the desire to build a tool that assists a child learning to read and write - we end up with a piece of code that internally exhibits high levels of concurrency. For example, even the simple Speak And Write application is made of 37 components which at present all right in the same process, but could be easily be made to use 37 processes... (prepending all Pipelines & Graphlines with the word "Process")

Despite this, we don't normally think in terms of number of components or concurrent things, largely because you don't normally think of the number of functions you use in a piece of code - we just focus on the functionality we want from the system. I'm sure once upon a time though people did, but I don't know anyone who counts the number of functions or methods they have. The diagram below for example is the high level functionality of the system:


Unlike many diagrams though, this has a 1 to 1 correspondance with the code: (skipping some details below)

bgcolour = (255,255,180)
Backplane("SPEECH").activate()
Interesting post on requirements for project websites

Posted: Michael, 4 Jul 2008 at 09:06
Comments: 0

I quite like this list of requirements for project websitesby Brian Jones. We've been planning a rebuild of the Kamaelia website to focus on applications written using kamaelia, how to modify them, make your apps using it, and how to join in (hoping people are interested in doing so), and whilst that's a wider scope than some of the things he's suggesting, but it's a good checklist. (After all, each of the applications themselves should actually have a project page with that sort of information).

Semantics changing on whitespace?

Posted: Michael, 3 Jul 2008 at 10:41
Comments: 4

In python it's clear that the semantics of a piece of code change depending on whitespace - for example:

if True:
    print "Yay"
    print "Woo"
frag. 1
vs
if True:
    print "Yay"
print "Woo"
frag. 2
However, generally speaking this does actually mean what people intended it to mean. The common exception is where, you might want to write this:
class Foo(object):
    def main(self):
         while True:
              print "Woo"
sys.stderr.write("DEBUG - UM, in loop\n")
              print "Yay"
frag. 3

Whereas of course python views the sys.stderr.write line as the end of the while, def, & class blocks. Often people do the above (in non-python languages) because they want to make it easier to find where they've inserted debug code, and lament the lack of it python. As an aside, you can of course do the above in python, if you add an extra line in:
class Foo(object):
    def main(self):
         while True:
              print "Woo"
              \
sys.stderr.write("DEBUG - UM, in loop\n")
              print "Yay"
frag. 4
Is it just me...

Posted: Michael, 2 Jul 2008 at 17:20
Comments: 0

... or is this a neat way to open a gaping security hole on your server? I can only presume that the backend of said beast only allows "arbitrary" connections to known hosts and ports, or it's a spammer's (and similar) delight.

Drawing Kamaelia Systems?

Posted: Michael, 28 Jun 2008 at 00:29
Comments: 4

Just been fiddling around with the idea of using the handwriting/gesture recognition code to allow the creation of kamaelia systems using an entirely pen driven approach. At first blush this does actually look like it could be a realistic proposition. The code that enables this in terms of proof of concept is *incredibly* hacky at present, but does seem, on a very basic level, to work. Using our packaging tools, I've packaged this up as an experimental release here:

If you're curious about it and have python & pygame already installed, installation boils down to this:

~/tmp> tar zxvf Kamaelia-Scribble-N-Link-0.1.0.tar.gz
~/tmp> cd Kamaelia-Scribble-N-Link-0.1.0/
~/tmp> sudo python setup.py install
Right at this instant all that application does is this:
  • It will recognise strokes drawn that look like an "o" or a "u" and assume you want to "draw"/add a new unconfigured component
  • Drawing a joined up "x" (ie curvy one like a "backward c going upwards, then forward c downwards") will delete the last component added
  • Drawing a stroke from top left to bottom right will "switch on" the "makelink" mode.
    • The link will start from the next link clicked - eg an outbox
    • The link will terminate with the following link clicked - eg an inbox
And that's pretty much it. No configuration of nodes at this time. The sort of output it will happily create looks like this:


However, it is an interesting start/proof of concept - it certainly is beginning to look like we will be able to literally draw software at some point in the future... Any feedback welcome. :-)
Twitter: Why?

Posted: Michael, 25 Jun 2008 at 16:33
Comments: 6

Why do you twitter?

Hack for Mashed: Kamaelia Speak And Learn

Posted: Michael, 22 Jun 2008 at 06:10
Comments: 0

This hack is a "toy" for a small child to assist them to learn to readand write. They are asked to write specific words - both textually andalso using speech synthesis. What they write is then read out to them.This version is by no means "complete", but it works. It currentlyallows a child to learn to write the word cat :-)

Where to Get it:

First initial version here:

Installing:

~/incoming/> tar zxvf Kamaelia\ Speak\ N\ Learn-1.0.0.tar.gz

Where's the Source?

Running:

~> SpeakNLearn.py

Dependencies:

  • You need to have espeak installed
  • You need to have aplay installed
  • You need to have pygame installed
Michael, June 2008
Mashed Hack: IRC Speaker Bot

Posted: Michael, 21 Jun 2008 at 22:52
Comments: 0

Since I wanted to "sit" on  some IRC channels here at Mashed and rather than flit channels or waste screen estate I decided to do a short hack to listen to the channel. As a hack for mashed, it does the following things:

  • It's IRC bot
  • It joins 3 channels - #kamtest, #kamaelia & #mashed
  • It listens to the channel

Anything that anyone says is then spoken (via espeak) by the bot. ie speech synth.
Kamaelia Mashed Release Candidate

Posted: Michael, 21 Jun 2008 at 20:12
Comments: 0

As per my previous post, the Kamaelia 0.6.0 release candidate #6 is available and can be downloaded from http://edit.kamaelia.org/release/

Kamaelia at BBC MASHED

Posted: Michael, 20 Jun 2008 at 19:53
Comments: 1

Going to BBC Mashed? Want to go? (a few are tickets left) Interested in one of these : {Kamaelia, Python, Concurrency, Multicore, DVB, Building network systems, Pygame Based Systems, Audio/video based} ? If so, please come along to the talk! If there's a clash, don't worry, I'm starting a page called GetMashed which will contain at least some starting points which I'll cover. (It's not the same as seeing it for yourself though :)

Powering a home server using solar power?

Posted: Michael, 31 May 2008 at 16:07
Comments: 4

I'm considering looking at how to do this, since it looks doable, but wonder if anyone has any experience of actually doing this, even partially. It's a very similar problem, it seems, to running a TV using solar power when camping, so it's not that strange a scenario. I've come to the conclusion that one way of doing this is as follows:

  • Use a cheap/old/low power laptop
  • Buy a car battery - or perhaps a 12V 100Ah battery
  • Use a 60W solar panel (or similar) to charge the 100Ah battery
  • Use a car cigarette light connector based power adapter to connect laptop to battery
The issue there is the 60W solar panel is expensive right now, and I'm wondering if there's a way of reducing this figure down lower - if anyone has experience of doing this. I'm also not sure it's sufficient - I suspect it has to be higher rated than that.

I'd be interested in hearing from anyone who's tried doing this.

How will you be celebrating the 60th Anniversary of the Computer?

Posted: Michael, 30 May 2008 at 18:31
Comments: 0

The Small-Scale Experimental Machine, known as SSEM, orthe "Baby", was designed and built atThe University of Manchester,and made its first successful run of a program on June 21st 1948.It was the first machine that had all the components now classically regardedas characteristic of the basic computer. Most importantly it was the firstcomputer that could store not only data but any (short!) user programin electronic memory and process it at electronic speed. -- http://www.computer50.org/

We'replanning a new release of Kamaelia for that day, which also co-incideswith BBC Mashed (aka BBC hackday 2). The new release should include, among other things, our multicore support, basic software transactionalmemory, generator, thread, process based components, a variety ofexample applications, a revamped website (yes, it needs a lick of paint) as well as a large number of new components fromimproved DVB support, better pygame tools, etc.

How are you planning on celebrating the 60th anniversary ?

:-)

Where to put our Multicore support in the Kamaelia tree?

Posted: Michael, 28 Apr 2008 at 21:11
Comments: 3

As discussed before, Kamaelia now has multicore support in subversion in the form of ProcessPipeline and ProcessGraphlines, meaning taking any Kamaelia system and making it multicore is a matter of simply pre-pending "Process" to the pipeline set up. That's pretty cool - especially if you're aware of what our normal cookbook code looks like. The question I've really got is where to merge this on the mainline. Given how useful this would be, it strikes me that like ThreadedComponent & LikeFile and Introspection, this ought to live in Axon. However, like Introspection, it probably also ought to be used from inside the Kamaelia Namespace.

Interesting thing new faces....

Posted: Michael, 27 Apr 2008 at 23:29
Comments: 0

New faces bring new ideas. As part of Google Summer of Code this year I asked upfront our students why they didn't want to use the mailing lists the project uses on sourceforge (since it was clear they didn't) - preferring private mails and IRC (as a project we tend to prefer IRC generally nyway). The response we got was unaminous around the fact they didn't want to subscribe due to SF lists being highly spammy, and requested we move over to google groups - so they could also gain better web access and RSS feeds. It was the one answer I hadn't thought of or expected :-)

Recent requests/Q and A

Posted: Michael, 27 Apr 2008 at 23:22
Comments: 0

People have used the comment or new post feature of my blog recently to leave some useful comments. Rather than have them as top level posts though, I'll drop them as replies to this post. In summary though:

  • I had a complaint/bug report from Steve (no surname) about the dates in my RSS feed (which should now be fixed), about double escaping of entities (which is also hopefully fixes). Steve also made a comment saying that he wished he didn't have to enter HTML manually. This strikes me as odd, because I'm typing this using Dojo's editor widget, so I guess that failed disgracefully on his machine. I've seen that on a few machines, but Dojo's generally got the most browser friendly wysiwyg editor, so I'm not sure what's up there.
  • I had a request whether Kamaelia's latest release is available for windows as a special bundle. The answer to that really is "it's python, so it should work, but we tend to develop under linux & Mac OS X so you may find some niggles". I certainly haven't created an installer for windows. All that said, we are overdue doing a new release, and /trunk is far and away recommended. Even so, that needs updating as well with recent developments... :-)
  • Finally, another anonymous poster made the following comment:
    • Came across this today and thought you might be interested in it Michael:
Python Bindings for AR Toolkit

Posted: Michael, 28 Mar 2008 at 16:56
Comments: 0

Not tested this, but this looks really interesting/promising. The AR Toolkit is really cool, and this appears to be linux & mac os x bindings for it. (Means that using this people should be able to access a webcam easily from both of those OSs using python...)


Writing successful applications - common questions and answers regarding summer of code

Posted: Michael, 28 Mar 2008 at 13:09
Comments: 1

We're getting some commons questions regarding GSOC, which boil down to "how do I write a successful application". I've taken the common starting point and generalised my comments, questions and suggestions in the hope of helping those in the crunch. This will also go on the Kamaelia website linked from the GSOC pages. As such this is written on the assumption the reader is someone interested in putting in a GSOC application.

Summer of Code

Posted: Michael, 27 Mar 2008 at 10:38
Comments: 0

Only a few more days to get your application in, and less if you want good feedback. The ideas we're seeing look really good and interesting, but we'd love to hear from more talented developers, and see your really cool ideas. Our ideas page continues to expand with more ideas and the mixture we're seeing is really interesting. If you haven't put in your application yet, why not? :) After all, this year we're after ideas that are completely different from previous years, since the intent is to allow students to produce something really cool they've always wanted to build, such that it grows the component set in interesting ways, as well as the kamaelia cookbook which given all components (from DVB through Open GL through networking) can be used with one another is kinda fun. (as well as useful because the whole thing is naturally multicore friendly) 

Kamaelia and Google Summer of Code

Posted: Michael, 21 Mar 2008 at 14:23
Comments: 0

Kamaelia Multicore Pipeline. The future is here.

Posted: Michael, 16 Mar 2008 at 00:16
Comments: 1

As the next step forward, I now have multiprocess support for basic pipeline. Essentially I have the equivalent of "self.link" implemented for multiprocesses. Kinda like the Postman in the MiniAxon tutorial. It's a little messy right now, but works, and does NOT require any changes to existing Kamaelia components. It builds on LikeFile from last years summer of code efforts and pprocess. The core code currently looks like this:

# setup code snipped, and specifically ProcessWrapComponent snipped
Sociable Software (Presentation)

Posted: Michael, 3 Mar 2008 at 23:57
Comments: 0

Well, I travelled all the way up to barcamp this weekend, and it was well worth it. Whilst I was there I gave a talk called "Sociable Software", as a general over arching description of how Kamaelia deals with concurrency. I'm going to write this up as a proper blog entry as well shortly, but in the meantime I thought adding the slides here would be a good idea.

What is wrong with free software zealots?

Posted: Michael, 29 Feb 2008 at 12:12
Comments: 0

Now they think they have the right to dictate to me what software I get to choose to run? cf in italics
On Thursday 28 February 2008 15:58:08 Dave Crossland wrote:
>  Even if I choose to use a proprietary program on a open source operating
>  system. Sorry, I'm not wrong,
25 years

Posted: Michael, 25 Feb 2008 at 00:23
Comments: 0

Well, this year will mark the 25th year that I've been writing code, so at some point later in the year I'll probably have some sort of party/drinks relating to that, since it means something (not sure what :-) ) to me. I have to decide a date where that makes sense, maybe some of the following will make sense. This post is a bit self indulgent as a result.

Javascript turtle

Posted: Michael, 25 Jan 2008 at 09:05
Comments: 0

A turtle/worm written using javascript: http://languagegame.org/tmp/jstile/exp/worm.html

Register Article on cross platform iPlayer "issues"

Posted: Michael, 15 Jan 2008 at 00:42
Comments: 3

I tend to understate certain things sometimes. From The Register article: MP accuses BBC chief of illegally championing Microsoft - John Pugh - MP for Southport:

As part of a Public Accounts Committee session on BBC procurement last Tuesday, the director-general was grilled by Liberal Democrat John Pugh MP on the decision to release the download version of the iPlayer for Windows and Internet Explorer only.
...
Pugh followed up the meeting with a letter to Thompson on Wednesday. He wrote: "By guaranteeing full functionality to the products of one software vendor [the BBC] is as a public body handing a commercial advantage to that company - effectively illegal state aid!
Parsing XML in Kamaelia using an expat parser

Posted: Michael, 13 Jan 2008 at 15:05
Comments: 0

Generally we've used sax for parsing XML, but it's useful to show how to parse XML using a parser like expat that works by calling back into your code. The trick, as usual with anything long running, is to put the thing that does the long running call into a thread and have it emit messages when a callback is called. The following is a minimal example:

import time
import Axon
import xml.parsers.expat
from Kamaelia.Chassis.Pipeline import Pipeline
from Kamaelia.Util.Console import ConsoleEchoer
Debugging Kamaelia Systems

Posted: Michael, 13 Jan 2008 at 14:56
Comments: 0

One thing that is particularly useful when debugging Kamaelia systems - especially servers - is the ability to dump out what's running. Fortunately Kamaelia provides a mechanism to introspect the scheduler, so you can do this as follows:

import time
import Axon
from Kamaelia.Chassis.Pipeline import Pipeline
from Kamaelia.Util.Console import ConsoleEchoer
Wish list for component systems in python

Posted: Michael, 13 Jan 2008 at 01:32
Comments: 11

Interesting wishlist for kamaelia (and others, but I'm interested from a kamaelia perspective) here: http://darkness.codefu.org/wordpress/2007/12/26/295
I'm not sure I buy all the criticisms, and feel they're more a wish-list in terms of improvements specific points mentioned about kamaelia as (IMO) potential wishlist items:

  • "the implementationneeds to be updated to support the new features of generators inPython 2.5...." - that I don't buy particularly. The new features of generators are : you can now do try..finally (which was a wart, and is just a matter of using that in new code) Another is that you can send values directly into a generator. If you do that, you effectively make your generator synchronous (or at best isochronous) with the caller. I'm not convinced by a long shot that's a good thing and it feels like a mistake in the language. However by chaining with a pub/sub generator you may be able to gain somethng a bit erlang-ish (to do the matching). I'm not convinced though. The other major thing is that you can send exceptions in to kill a generator. Now that is useful, but is something we'll only do when we decide to kill off completely the kamaelia on python series 60 - which is something I'm not sure I want to do yet. The other point is "how do you model that correctly in other languages" - which is unclear.
  • "....as the current syntax strikes me as rather ugly." - is the other half of the sentence. That can be true at times, though I think the use of WaitComplete and the new STM code makes this somewhat nicer - cf the greylisting server.
  • "In fact,it looks like Kamaelia needs a recent release, period: the last one Isaw was from 2006." - ouch! I hadn't realised it had been quite as long as that. That's actually quite embarassing! It was on my todo list, but I'll take that on board.
  • "Kamaelia also fails to offer multi-process operation" - yep. This is something I don't make any bones about - we wanted to make sure that the current API was useful and used it in a variety of scenarios before moving this forward. The recent experiments I've made here suggest this should be relatively simple to achieve. (multiwindow pygame, first multiprocess test)
  • The idea of a networked pipe seems desirable as well given this: "itneeds a way to generalize its “wires” to support communication with,e.g., remote components. You could actually combine the marshallingcomponent with the framing component and a TCP client/servercomponents and make this work" - this is something I've played with in /Sketches - but that's nowhere near the level I'd want to use as yet - partly because we hadn't at that point really got a feel for how this could be useful. (That's changing) However, this isn't something I've had a real chance to move forward at all.
  • The code should be able to migrate from microthread to thread to process as desired. This comes back to the idea that ThreadedComponent should be a chassis (ala FirstProcessBasedComponent) rather than a component you instantiate. (that would allow the threaded component to accept any component into running in its thread). This is something I realised early on could be a real problem when it comes to the issue of shared data. However the recent STM code designed to replace the dictionary in the CAT could be a real boost there. Definitely worth reconsidering. This opens up the general idea of proxy components further.
  • Components in multi-language- this seems at odds with the request to use more features of python generators. My guess is actually the core would be a core set of constructs,and then a predefined inter-language comms style. I've been leaning towards JSON more and more lately, and this would certainly hit the multi-language compatibility & ease of working with issues.
Beyond that however there's also an implication of a desire for lower level configuration as well - along the lines of Shards as experimented with by Tara last summer. All of this is doable, but with limited time, its debatable how much kamaelia can satisfy all these things, especially when I'm the only person working on it, with little feedback. As a result I'll focus on the areas I'm getting feedback on :-)
Axon.STM 1.0.1 - New Release

Posted: Michael, 24 Dec 2007 at 11:59
Comments: 0

Just posted this to comp.lang.python - but posting here as well :-) I've received some great feedback since the initial beta release of the minimalistic STM code I discussed and released 2 weeks ago. I've incorporated the feedback, and created a couple of examples based on the canonical dining philosophers example. (One based on normal python threads, one based on Kamaelia)

Dining Philosophers In Kamaelia

Posted: Michael, 23 Dec 2007 at 23:36
Comments: 0

After playing around with the STM code, and spotting the dining philosopher's code in the MASCOT example/manual that I found, I've now implemented the traditional dining philosophers example using Kamaelia. This is largely about acquiring and releasing resources - something we largely do in a really simplistic way right now, and have explicitly excluded threaded code from directly accessing.

Software Transactional Memory updated - Code Review Sought!

Posted: Michael, 20 Dec 2007 at 12:49
Comments: 0

I've updated the STM code in subversion at present here:

https://kamaelia.svn.sourceforge.net/svnroot/kamaelia/branches/private_MPS_Scratch/Bindings/STM/Axon/STM.py
I would really appreciate any code review from anyone who's capable of doing so - even if you've never done a code review before! Things I think need particular attention is the detail around locking. Specifically the following 4 methods are intended to be used by a user of the code:
class Store(object):
    def usevar(self, key):
    def set(self, key, value):
    def using(self, *keys):
    def set_values(self, D):
The above 4 functions entirely manage the locking. The following functions are internal and all assume that the store is locked appropriately before an update attempt:
    def __get(self, key):
    def __make(self, key):
    def __do_update(self, key, value):
    def __can_update(self,key, value):
Anyhow, why is this useful? Well, it turns out this should make resource management a lot simpler. If I'm right, the following code should be sufficient for the Dining Philosopher's problem:
take forks... (pseudo python, ignoring various details)
not_have_forks = True
while not_have_forks:
   try:
      S = <get the store from somewhere>
      forks = S.using("fork.1", "fork.2")
      if forks["1"].value == None and forks["2"].value == None:
          forks["1"].value = id(self)
          forks["2"].value = id(self)
          forks.commit()
          not_have_forks = False
   except ConcurrentUpdate:
       time.sleep(random()) # random backoff - doesn't guarantee success
   except BusyRetry:
       time.sleep(random()) # random backoff - doesn't guarantee success
eat()
# Put forks back - should always succeed. Should. :-)
try:
    forks["1"].value = None
    forks["2"].value = None
    forks.commit()
except ConcurrentUpdate:
    pass
except BusyRetry:
    pass
... which I think is pretty neat :-)
Concurrency: The difference between functional call context & object context

Posted: Michael, 17 Dec 2007 at 22:25
Comments: 0

A common idiom in functional programming at least with the functional languages I've used and systems I've needed to build - is a call context. That is any sufficiently large functional piece of code is at some level modelling interacting with a model. So in pseudo-functional-language you (conceptually) end up doing something like:

fun move_x context distance:
   let
       x,y,z = Parts(context)
   in
       return Context(x+distance,y,z)
   done
This context then gets passed around, and updated. This means you can end up with lots of functions that look like this:
fun move_x contet ...
fun move_y context ...
fun move_z context ...
The web is stateless, please

Posted: Michael, 12 Dec 2007 at 00:50
Comments: 0

Arrgh. If I hit "delete" on a form, I'd like the thing I'm deleting to be encoded in the form rather than associated with an opaque sessionid which "helpfully" remembers what I last looked at. The web is meant to be stateless after all. kthxbye.

Simple assistive short term memory

Posted: Michael, 10 Dec 2007 at 22:54
Comments: 2

OK, here's a challenge. Build a tool to assist with short term memory loss. The idea is this:

  • You go to the top of the stairs to do something
  • You get there, and wonder what you were there for
It'd be neat if you could do this:
  • Wear a simple looking pedant, which was a short term memory assistive device
  • Simply mutter to yourself what you're doing
  • When you forget what you're doing you press/tap a button & hear a repeat of the last muttering
    • Press it again in rapid succession, and you get the one before, and so on. (giving you context "what was I doing before...")
  • Aesthetically, it would be nice if the device looked like a pedant or similar.
  • The playback/recording could be a simple earpiece/microphone (ala mobile phone handsfree).
  • It should be sufficiently low power to run happily for as much of a day or more as possible.

image (c) 2007 daniboi1977 (flickr)
(original license cc-nc-nd)
Technologically this sounds like it should be really quite simple & trivial to make today.
  • Now here's the thing - you could do this today with a notebook & pencil hanging from a necklace - except that you replace muttering with scribbling, and pressing a button with looking at the notebook. Or perhaps hanging a dictaphone round your neck and remembering to hit "record" every time you say something.
  • However, not everyone who might find such a thing useful (ie most of us when we get older) would be comfortable with the notebook on a necklace or dictapone on a necklace approach, but may find a short-term memory pendant assist socially acceptable.
So, the challenge is - could you make this today (almost certainly), and given that, how would you do so? This isn't entirely a thought experiment - people with alzheimers may find such a beast useful, and simply having to press a button to hear what you were doing and double tapping to hear what you were doing is probably something learnable. (maybe, dunno in that context really)
Interesting: Transparent messages instead of modal "monolog" boxes

Posted: Michael, 10 Dec 2007 at 01:37
Comments: 0

Wins bonus points for looking straight out of a film :-)
http://humanized.com/weblog/2006/09/11/monolog_boxes_and_transparent_messages/
Their point is somewhat less shallow than this, but it does remind of film interfaces... :-)


Widefinder in Kamaelia?

Posted: Michael, 9 Dec 2007 at 17:55
Comments: 0

Tim Bray's widefinder project is an interesting little problem. How do you find the top 10 most popular accesses to your website in a way that gains performance when you use more CPUs? Having done log analysis in the past, there's always a tradeoff Looking at the original Ruby source

counts = {}
counts.default = 0
Minimalistic Software Transactional Memory

Posted: Michael, 8 Dec 2007 at 23:03
Comments: 2

I've just posted this message on comp.lang.python looking for comments, but I'll post it here in case there's any feedback this way :-)

Fascinating new definition of Free Software

Posted: Michael, 5 Dec 2007 at 19:37
Comments: 2

On the backstage mailing list, out of the interminable rantings and ravings about whether DRM is a good or bad idea, whether people should call Linux GNU/Linux of "that piece of software I use" or just Tux, or indeed on a dozen other things, a (not to hot) flame war about BSD licensing vs GPL licensing came about. Now I've seen this argument played back and forth dozens of times (at least) over the past 10 - 15 years and it just get repetitive with the same arguments thrashed back and forth on both sides.

Multiple Windows in Pygame, using true concurrency in python via Kamaelia

Posted: Michael, 27 Nov 2007 at 02:11
Comments: 1

First of all, the proof that this works in Kamaelia, right now:


OK, this is using the new code for running multiple Kamaelia type systems in multiple processes relatively simply. Specifically, the "interesting" code looks like this:
The first Truly Concurrent Kamaelia Component

Posted: Michael, 25 Nov 2007 at 22:20
Comments: 0

OK, this is using a massively simplified version of the primitives needed for concurrency in Kamaelia, but the following is the first component that will happily run completely in parallel with the rest of the system.

class FirstProcessBasedComponent(SimplestProcessComponent):
    def main(self):
        while 1:
            yield 1
            time.sleep(0.3)
            if self.dataReady():
                print time.time(),"main : RECEIVE FROM CHANNEL", self.recv()
            else:
                print time.time(),"main: CHANNEL NOT READY"

As you can see this is pretty much identical to the traditional Kamaelia model. Indeed, change the baseclass & you get a single threaded component, though you'd probably want to change the time.sleep behaviour.
Kamaelia based (Extended) Entity Relationship Modelling Tool

Posted: Michael, 25 Nov 2007 at 01:52
Comments: 12

This weekend's hack - a tool to make my life easier - a tool to make creation, modelling and playing with extended entity relationship diagrams simpler (Of the kind found in Elmasri & Navathe). The tool is currently sitting in my /Sketches/MPS area named somewhat imaginatively ERTopologyTool.py and I think it's really quite funky. I'm using it to make it easier to communicate ideas I've got on the participate project (which is taking most of my dayjob time at the moment), and it can take a textual description of the schema that looks like this:

entity missionagent
entity person(missionagent)
entity team(missionagent)
Benchmarking: Kamaelia vs Stackless

Posted: Michael, 21 Nov 2007 at 23:48
Comments: 0

Interesting post by rhonabwy on comparing Kamaelia to Stackless. The benchmark figures there are pretty grim from my perspective, but a useful starting point:

10 concurrent objects, 1000 loops
python timeit.py -s "import hackysack" "hackysack.runit(10,1000)"

Pitivi

Posted: Michael, 21 Nov 2007 at 22:07
Comments: 0

So, I've been recommended to use "pitivi" a few times, so I figures, why not give it a go. Well, I'm running the latest version of open suse, just released, what can go wrong eh? So I download it. And then I start downloading the dependencies it asks me to install along the way. Then it finally starts. Then I get told I need to install another dependency (gnonlin). OK, grab that. And after all that then what? It then says that the version of gstreamer I've got is too old. I'm sorry, but at this point I'm giving up.

Using Python to push files using bluetooth

Posted: Michael, 19 Nov 2007 at 09:32
Comments: 0

Just jotting this down really. You need Lightblue installed, which also requires pybluez (hence bluez) and also openobex installed. OK, no further ado:

>>> from lightblue import *
>>>
>>> finddevices()
[('00:11:9F:C3:7E:A7', u'Nokia 6630', 5243404)]
>>> services = findservices("00:11:9F:C3:7E:A7")
>>> services
[('00:11:9F:C3:7E:A7', None, u'SDP Server'),
 ('00:11:9F:C3:7E:A7', 1, u'Hands-Free Audio Gateway'),
 ('00:11:9F:C3:7E:A7', 2, u'Headset Audio Gateway'),
 ('00:11:9F:C3:7E:A7', 10, u'OBEX File Transfer'),
 ('00:11:9F:C3:7E:A7', 11, u'SyncMLClient'),
 ('00:11:9F:C3:7E:A7', 12, u'Nokia OBEX PC Suite Services'),
 ('00:11:9F:C3:7E:A7', 9, u'OBEX Object Push'),
 ('00:11:9F:C3:7E:A7', 3, u'Dial-Up Networking'),
 ('00:11:9F:C3:7E:A7', 15, u'Imaging')]
>>>
>>> D = [ (x,y,z) for x,y,z in services if "obex" in z.lower() and "push" in z.lower()]
>>> D
[('00:11:9F:C3:7E:A7', 9, u'OBEX Object Push')]
>>> (device, channel, description) = D[0]
>>> obex.sendfile(device, channel, "/media/usbdisk/Kamaelia/cat-trans.png")
[btobexclient_connect] Connecting transport...
[btobexclient_connect] Connecting OBEX session...
[btobexclient_done] Request 0x00 successful
[btobexclient_put] Sending file 'cat-trans.png' (116671 bytes)...
[btobexclient_done] Request 0x02 successful
[btobexclient_disconnect] Disconnecting...
[btobexclient_done] Request 0x01 successful
[btobexserver_cleanup] entry.
>>>

The Selfish Programmer

Posted: Michael, 1 Oct 2007 at 01:18
Comments: 0

I have the distinct pleasure of chairing the EBU Open Source workshop later today, having been invited to do so and give the opening welcome/talk. For a while I've been wracking my brains about "how do you open a workshop on open source where there's going to be a lot said on a lot of interesting topics without prejudging them or jumping the gun?". I decided therefore to do the only thing you can - to talk about what it means to me, why I develop open source, and what speaks to me. After all, many people distrust the concept of "something for nothing" (even though that's not really what open source is!).

Recently published RFCs

Posted: Michael, 25 Sep 2007 at 12:38
Comments: 1

RFC 4861 Neighbor Discovery for IP version 6 (IPv6)
This document specifies the Neighbor Discovery protocol for IP Version 6.  IPv6 nodes on the same link use Neighbor Discovery to discover each other's presence, to determine each other's link-layer addresses, to find routers, and to maintain reachability information about the paths to active neighbors.
http://www.rfc-editor.org/rfc/rfc4861.txt Draft Standard Protocol.

Greylisting using Kamaelia

Posted: Michael, 19 Sep 2007 at 22:49
Comments: 12

I've written a greylisting server using Kamaelia, and its turned my mail back to something usable. I've been running this server for 52 hours now & it's processed over 5000 mails. 94% of those have been rejected as spam, leaving a handful of spams coming through from mailing lists. It's a spectacular change for me.

Ye Old Cue: Visualizando multiples posts

Posted: guest, 13 Sep 2007 at 09:08
Comments: 2

Posted here ---

Ruby Based Kamaelia Core (miniaxon.rb)

Posted: Michael, 10 Sep 2007 at 23:07
Comments: 0

I finally got around to learning sufficient ruby to write a Mini Axon in Ruby - which means there is a basic Kamaelia core in Ruby available now. This means the following code is valid Ruby code and the components work in exactly the same way as Python based Kamaelia components:

class Producer < Component
   @@name = "Producer"
   def initialize(message)
      super
      @message = message
   end
   def main
      loop do
         yield 1
         send @message, "outbox"
         showboxes if $debug
      end
   end
end
Erlang vs Stackless

Posted: Michael, 2 Aug 2007 at 00:16
Comments: 0

Not sure how valid the following benchmark is, but it's an interesting datapoint.

More People on Facebook than live in Iraq

Posted: Michael, 31 Jul 2007 at 11:08
Comments: 0

Interesting factoid - there are over 30 million people on Facebook. That means if facebook was a nation, it would be the 39th (possibly 38th) largest country in the world according to the list of countries by population page of Wikipedia. This puts it in the spot above Iraq and below Uganda.

Greylisting for non-techies

Posted: Michael, 31 Jul 2007 at 00:03
Comments: 1

Greylisting is like magical glass for mystical flies. Real email is delivered by real flies, whereas spam email is delivered by fake mystical flies. The difference is that fake mystical flies don't bang their head against the glass repeatedly when they hit a window to try and get through, they just bounce of and don't try again. However real flies do bang their head against the window repeatedly until they get through.

Design Thinking Links

Posted: Michael, 20 Jul 2007 at 10:38
Comments: 0

A bunch of links nabbed from Michael Tiemann's OSI blog

I like this, because the second from last link focusses on design books. 3 of which I already own. Though it would be good to read the other 7 probably.

It also matches with the way I tend to manage development in Kamaelia's SVN - it's set up to encourage a high diversity of ideas, large amounts of checkins, and rigourously stable and clean code in releases. Design is crucial in making something new. Taking that design and moving it to engineering is just as vital however, and there are some very important steps to bear in mind how that happens. (not least that the skill set can be extremely different and many people need to learn at least one of those sets of skills)

Python, Nokia Mobiles, Easy control from linux; Sync of profile/blog picture with facebook

Posted: Michael, 19 Jul 2007 at 01:41
Comments: 1

Quick and simple access to the BT console (based on notes here)

# sdptool add-channel=27 SP
# rfcomm listen /dev/rfcomm0 27
Waiting for connection on channel 27
Connection from 00:11:22:33:44:55 to /dev/rfcomm0
Press CTRL-C for hangup
connect using btconsole.py on phone
on different console on linux
minicom -s -m (set device to /dev/rfcomm0 )
Also, facebook image syncing:
curl 2>/dev/null -O -A "Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.8.1) Gecko/2006102
3 SUSE/2.0-30 Firefox/2.0" http://www.facebook.com/p/Michael_Sparks/747770380
F=`grep profileimage 747770380|sed -e "s/^.*http:/http:/"|sed -e "s/\.jpg.*/\.jpg/"`
G=`grep profileimage 747770380|sed -e "s/^.*http:/http:/"|sed -e "s/\.jpg.*/\.jpg/"|sed
 "s#.*/##"`
curl 2>/dev/null -O $F
mv $G michael.jpg
cp michael.jpg /usr/local/httpd/sites/com.yeoldeclue/docs/michael.jpg

Configuring Exim to block email to all except specified addresses

Posted: Michael, 20 May 2007 at 17:41
Comments: 0

Ever needed to only allow emails from specific people through to specific addresses using exim? If you have then hopefully this post is of use to you. I'm writing it up here because it's proving useful to me right now.

Come to PyCon UK - September 8th&9th !

Posted: Michael, 20 Apr 2007 at 23:26
Comments: 0

The PyCon UK Society has announced a UK Python Conference. This is an affordable community conference taking place on 8th/9th September. The conference is fantastic value, especially if you take advantage of the extra early bird booking offer. Both new and experienced Python programmers will benefit from the varied programme.

Why am I posting about this? I'd personally like to invite UK pythonistas to come, share their knowledge with others, learn new things and hang out. It's a community conference, which means it has the following characteristics:

  • You can help make it amazing, by participating & speaking, by helping, by attending!
  • It is cheap
  • It will be fun, and accessible. We (I'm helping organise this :) ) really want the conference to be accessible to all, from those who have no idea of what python is, let alone coded in it, through to those who are working on their upteenth bytecode hack/compiler.
Seriously though aside from this, I'm really posting about this because despite being a language pragmatist - ie I'll use any language that gets the job done - I'm largely finding that I'll use python for almost everything these days. Community conferences are a real opportunity to dive in and help and learn and share. Like python itself, the conference is also platform agnostic, so you're welcome if you use Windows, Mac OS X, Solaris, FreeBSD, or even the same OS as me - Linux :) .
One Laptop Per Child Project Looking For Pygame Devs

Posted: Michael, 7 Apr 2007 at 09:48
Comments: 0

From the pygame mailing list:

From: Noah Kantrowitz [ check pygame archives for email (don't want to cause spammage) ]
Subject: [pygame] Calling all game developers

Calling all game developers! The One Laptop Per Child project needs
talented game developers to work on software for the XO laptops. Thanks
to a few awesome developers, PyGame is now up and running under Sugar
(the OLPC graphical environment) and should be included in the build
system shortly. What we need now is games geared towards children in
developing areas. Information about PyGame on the XO can be found at
http://mailman.laptop.org/pipermail/games/2007-April/000036.html. I
would ask anyone interested in either building new games or porting
existing ones to join the OLPC games list
(http://mailman.laptop.org/mailman/listinfo/games) and discuss your
ideas there. This is a chance to have a major impact on the lives of
millions of children, as well as work on a unique platform. If you have
any questions please don't hesitate to email me, or ask on the games list.

--Noah Kantrowitz
OLPC Evangelist

If you know and enjoy pygame and are looking for something good and fun, it'd be well worth getting in touch.
Dilbert

Posted: Michael, 22 Mar 2007 at 10:25
Comments: 0

Tongue in cheek, but I quite like (based on occam's razor):

"Given a variety of options, the dumbest one is probably right" - applies to "which decision did management make"
(not to be taken seriously :)
I have a cunning plan

Posted: Michael, 9 Mar 2007 at 19:31
Comments: 4

I had a great idea today, and I've registered the domain for it just now. I'm probably going to knock up something quite quickly (I hope - time permitting, having a social life is restricting part time hacking somewhat :), but it's something that's probably useful for work, but also scratches a few itches as well. In short though, it's the beginnings of a possible answer to the question: How do you do a bigger piece of high quality film or TV in an open source way. That said, I don't want to restrict the tools to just an open source/creative commons approach, but I think this has some potential real mileage. More as I go on.

Compiled Kamaelia: A Mini Axon that compiles to C++ using Shed Skin (+ one minor piece of assistance)

Posted: Michael, 22 Feb 2007 at 15:16
Comments: 2

Well, after getting a mini axon standing" compiling using shedskin yesterday, I've taken the next steps to see if we can get a full mini-axon system compiling. The good news is that, with 1 piece of manual assistance, we can! As a result I have here a compiled version of a mini-axon system.
Yes, you heard right - we have a compilable to executable version of Kamaelia thanks to shedskin.

First of all, what manual change do we need to make? Well, due to the fact we activate various microprocesses in the same way, we need to warn the C++ compiler of this fact, and help shedskin out a bit. The code shedskin creates that's problematic is this:

  • int scheduler::activateMicroprocess(lambda2 some_gen, microprocess *some_obj) {
        __iter<int> *microthread;

        microthread = some_gen(some_obj);
        (this->newqueue)->append(microthread);
        return 0;
    }

This is the fixed code:

  • template<class Klass>
    int scheduler::activateMicroprocess(__iter<int> *(*some_gen)(Klass *), Klass *some_obj) {
        __iter<int> *microthread;
MiniAxon - Standing on Shed Skin - first steps towards a C++ Compiled Kamaelia working

Posted: Michael, 21 Feb 2007 at 14:20
Comments: 0

Well, I looked at shedskin as I mentioned the other day, and now have this working locally. One thing we encourage people new to Kamaelia is to build their own mini-core of the system - Axon. This provides basic microprocesses, basic component communications and scheduling. Whilst this may sound hairy, having run the mini-axon tutorial past many new programmers, including the tutor list, it seems pretty OK. (The tutorial is pretty gentle and provides very small simple steps aimed at someone new to python) The rest of this post after the fold has the meat in.

Entertainment Manchester Review: Bollywood Mikado

Posted: Michael, 18 Feb 2007 at 17:17
Comments: 0

Summary - We ruled!

Hearing of people enjoying the show is great, and makes all the hardwork worthwhile. If you still want to see the show, it's being put on again in Buxton for one night only - March 3rd.

Photos from the show:

Kamaelia for P2P streaming ?

Posted: Michael, 11 Feb 2007 at 14:22
Comments: 0

There was another post on comp.lang.python which asked essentially:

> I am 3 months old to python and I have done some small projects. I
> want to build a Peer to Peer streaming client

I wrote a reply back to the person directly and cc'd the kamaelia list, but the core of the reply is that the core of this relatively simple in Kamaelia, so I thought it worth posting here. The core of an actual peer looks like this:

Backplane("AUDIO").activate()

Pipeline( TCPClient( source_IP, source_port),
          PublishTo("AUDIO"),
).activate()

def PassOnAudio(): return SubscribeTo("AUDIO")

SimpleServer(protocol=PassOnAudio,port=source_port).run()

Whilst this clearly needs a source, and the core of that looks like this:
from Kamaelia.Audio.PyMedia.Input import Input  as _SoundInput
Things I need to look up

Posted: Michael, 11 Feb 2007 at 13:36
Comments: 0

Two things struck me eye from comp.lang.python.announce that I need to look up at some point:

  • Shed Skin by Mark Dufur has a new release with initial support for iterators and generators. This strikes me as a really exciting, since it may mean that we can write a simple version of Kamaelia - ie a miniaxon - that works with shed skin. That opens up some really interesting options! I'll report back in the comments to this post the results of my experiments :)  (I need to reinstall dependencies first)
  • The other was Partial 1.0 by Martin v. Löwis which allows you to define a single class across multiple files. Whilst this probably looks odd I can see all sorts of interesting uses for this from the perspective of exogenous connectors. (Or put another for plugins)

The Mikado: In the style of Bollywood :)

Posted: Michael, 10 Feb 2007 at 21:13
Comments: 0

I'd like to invite anyone reading thing to come next week (its an ideal valentines day out :) and see a Bollywood inspired version of "The Mikado" at the Royal Northern College of Music (map). It's being put on by the award winning MUGSS society, and tickets start at a piffling £5 for concessions or start at £8 for the rest of us :) (more after the fold :)

Upgrade to OpenSuSE 10.2, Topology Visualiser Fixed

Posted: Michael, 10 Feb 2007 at 16:54
Comments: 0

Well, I recently upgrade from SuSE 10.1 to Open SuSE 10.2. The problem with this is the reason I normally use SuSE is because I personally want to focus on writing code, documents, etc. Well, what broke? Well, most of my video players had their libraries removed by the installer, and also firmware support for my wireless card was removed. That's not really any fun, but resolvable. Now I remember why I buy a distribution instead! Anyway, it's been useful for testing with python 2.5 (deprecation of string exceptions) and with updated packages. (Pygame has some minor behavioural differences meaning some minor bugfixes have been needed - this especially impacted the TopologyVisualiser. I've now fixed these issues :-)

Glow in the Dark Dalek Stickers

Posted: Michael, 2 Feb 2007 at 23:56
Comments: 0

Last year at OSCON I was lambasted. Was it my talk was bad? No. Was it because the demo broke? No, I recovered from that (I showed something else cool instead :). What was the piece of sacrilege I did? I had glow in the dark stickers of K9, a TARDIS, a Dalek and Cyberman on my laptop and I had been so thoughtless not to bring some for everyone. OK, the fact that I'd had the stickers on my laptop for months didn't wash. The fact they were freebies off the cover of Dr Who Adventures didn't wash either.

Anyway, in order to make up for it, here's a public service announcement: Dr Who Adventures magazine this week has glow in the dark DALEK!!! stickers on the front. Yes, all new glow in the dark stickers. They're even brighter than last years by the looks of things. (OK, there's also a very shiny K9 and a really large shiny Dalek Zek and a whole load of other stickers too, but I know it's the glow in the dark ones we all really wanted :-) ) Now to try and find out how to get a large supply of these for the next conference....

The Issue with DRM

Posted: Michael, 30 Jan 2007 at 01:43
Comments: 2

It's just struck me that there is an issue with DRM that goes to the core of the way the open source and free software movements run, but not in the way that you might expect. In the following, please read DRM to mean "digital restrictions enforcement", since that's largely all the people who say they need DRM are really thinking about. And whilst we're at it, what should a public broadcaster's policy be? (article is relatively long, so most of it is after the fold)

Procrastination

Posted: Michael, 28 Jan 2007 at 00:28
Comments: 1

I've been doing some reading recently on how to deal with procrastination, in an attempt to deal with my own procrastination better. Along the way I've discovered a few interesting things which I'm finding useful, so I thought its worth blogging about. The post is longer than anticipated so the content is after the fold, if you're willing to spend 5 minutes on it :)

Your favourite RSS writing library?

Posted: Michael, 21 Jan 2007 at 21:23
Comments: 0

Looking around to find a good rss writing library, I can't find one that I like. I can fund several functional ones, but I'm curious to hear of any suggestions anyone has...

Who am I?

Posted: Michael, 19 Jan 2007 at 21:06
Comments: 0

I don't know about anyone else, but I find this relatively scary - Zoominfo trawl the web to find out who you are, collate all the information and put it on the web - all completely automatically. I came across my entry by mistake, and personally I find it relatively disturbing. I know it's one of those things that can't be avoided, but it's really quite an unpleasant thing to see, and makes a complete mockery of privacy laws. On the other hand, it's really quite impressive.

Probably one of the more scary/cool aspects is the way I was tagged by the system - at Foo Camp, I was manually tagged based on the results that Tim O'Reilly and friends could find out about me by searching - zoominfo's tags are actually, in my opinion, more accurate (though still not how I'd tag myself. (Sorry Tim, I couldn't help linking to your zoominfo!)

Apple TV vs Mac Mini

Posted: Michael, 17 Jan 2007 at 10:28
Comments: 0

Brady has a post asking what people think about the apple TV vs Mac Mini. I started writing a reply there, but it grew massively in length, so I've posted it here instead. (most of the post below the cut!) Why the long reply? I've got a Mac Mini plugged into the TV video the composite video connector and can see arguments both ways. At the end of the day though, my take is a Mini rather than Apple TV - I prefer a computer to be a computer rather than an appliance (Actually I'm very tempted by a Dreambox 7020, but can't justify it right now).

End of whitelisting? Spammers spoofing being part of social networks from conversations in usenet?

Posted: Michael, 12 Jan 2007 at 11:19
Comments: 0

Not all my spam is automatically filtered. On one account I've started resorting to automatic filtering, because it's become untenable - however a different one I still manually filter. As a result something caught my eye. I post on usenet periodically about different things on different topics in different conversations (all sorts). That means there are also some names I naturally recognise from others, and I think "ah that's not spam".

After all, if you get an email with the subject "My New Year's resolutions", and on the surface it's from someone you know, you don't mentally mark it as spam. Anyway, what struck me was in the past 48 hours I've recieved about 8 spam emails (with the same content) from email addresses I know and recognise. Given email addresses can't be spoofed so trivially (insanely so), this made me wonder - how come this spam *only* comes from people I know and have been in conversations with on usenet?

And then I realised - have spammers  finally have started resorting to building automatically the social network of people who have chatted on usenet and then send spams whilst spoofing being inside that network? If so that's really quite nasty/inventive. There's no way of knowing if that's the case, but it could start causing real problems for people using whitelisting.

Two webcams that work very happily with Linux

Posted: Michael, 6 Jan 2007 at 13:48
Comments: 3

Well, I had a need to get two webcams, and whilst it wasn't necessary for them to work with linux, I was quite hopeful. Anyway I bought the Logitech Quickcam Chat (£20)and the Technika webcam ("techweb1" - £10) from Tesco. Both appear to work fine with linux. The driver I'm using for both cameras is gspcav1 from Michel Xhaard's website - specifically the gspcav1-20061216.tar.gz tar ball. The Gideo application he links to is also useful for testing.

Quite entertainingly you can use both webcams simultaneously. Personally I think the build quality of the logitech one is higher, but the automatic white balance kicks in with the Technika one resulting in more visually appealing  display (I haven't tested with windows, but I suspect the white balance kicks in there for the logitech).

Overall, I think I personally prefer the technika as a result since for me on my preferred platform, it just works. I'm very tempted now to see if I can create some python bindings for this (which would work with both cameras under linux) since it'd be nice to incorporate a video display into our Kamaelia based whiteboarding app. :-) (Hmm, reminds me, I should put the Kamaelia Whiteboard tutorial from Linux Format online now that the issue with the article in has gone off sale).

Predictions for 2007

Posted: Michael, 3 Jan 2007 at 13:45
Comments: 12

Some people started talking about predictions for 2007, so I decided to come up with some. These won't be in the RSS feed to avoid spamming various aggregators. Feedback and comments welcome :-) Topics covered: Digital Switchover, PS3, DRM, Linux, Windows, Microsoft, Apple, The Web, Politics, Environment, International, Media, Fashion, Gadgets, Predictions. As you might guess from the list, not all of this is entirely serious :)

Manchester Meetup

Posted: Michael, 3 Jan 2007 at 13:28
Comments: 0

Manchester Geekup - where all users of dynamic languages (and others) are welcome to come and share beer, knowledge, food happens againt next tuesday.
Where: The B-Lounge near Piccadilly (2 mins from station)
When: Tuesday 9th Jan, after work. (people tend to arrive from 6 onwards, talks start at 7)
Who: You, me, anyone with an interest in Python, PHP, Perl, Ruby, etc :-)
Talk format: Usually lightning talk or Pecha Kucha style - ie short :)

Supervillains!

Posted: Michael, 2 Jan 2007 at 00:08
Comments: 0

OK, so my superhero was the dubious (occasionally anti-hero) Hulk (which secretly I quite like :-) What's my Supervillain result?

Your results:
You are Dr. Doom
Dr. Doom
 71%
Magneto
 58%
Apocalypse
 58%
Juggernaut
 56%
Dark Phoenix
 54%
Green Goblin
 52%
Lex Luthor
 51%
The Joker
 50%
Kingpin
 49%
Mr. Freeze
 46%
Catwoman
 45%
Venom
 42%
Two-Face
 36%
Riddler
 32%
Mystique
 32%
Poison Ivy
 25%
Blessed with smarts and power but burdened by vanity.
Click here to take the Super Villain Personality Test

Must be new year - have a good one :-)
(Two nice quiz links courtesy of Steve Holden).

Superheroes?

Posted: Michael, 30 Dec 2006 at 13:10
Comments: 0

Steve Holden posted about the "which superhero are you?" quiz. My result?
Your results:
You are Hulk
Hulk
 80%
Spider-Man
 70%
Green Lantern
 65%
Iron Man
 65%
Robin
 55%
Catwoman
 55%
Batman
 50%
Supergirl
 48%
Superman
 45%
The Flash
 35%
Wonder Woman
 28%
You are a wanderer with
amazing strength.
Click here to take the "Which Superhero are you?" quiz...

Exogenous Connectors as an emergent property?

Posted: Michael, 29 Dec 2006 at 00:47
Comments: 0

I've just added basic authentication to the Kamaelia Web Client, since it seems the useful thing to do, and appear to have created a need for exogenous connectors as an emergent property, which is incredibly cool. This will potentially allow:

Pipeline(
    ConsoleReader(eol=""),
    SequentialTransformer(
           HTTPRequest,
           AddHeader(AuthorizationHeader("Username", "password"))
           AddHeader(CacheControlHeader("no-cache"))
    ),
    SimpleHTTPClient(),
    SequentialTransformer(feedparser.parse,
                          pprint.pformat),
    ConsoleEchoer(),
).run()
This would take a URL from the console, turn it into an HTTP request, add a header which happens to be an Authorization header, also add a cache control header, and then pass this onto the HTTP Client. That makes a request, and then passes it on to the next component to be parsed and then put through a pretty printer. It's still transformational, but rather than requiring you to write new components for everything you can write simple transformations. The next step is to figure out how to invert this - how to make it such that Sequential components can be the primary component framework. (this would allow Kamaelia components to be used as a library in "normal" code). (Consider for example taking an RSS feed from a site that publishes links to .torrent files, extracting the ones you like, auto dumping the results to disk for local playback when you get home... )
Basic Authentication - Did you know?

Posted: Michael, 28 Dec 2006 at 19:41
Comments: 1

You may think basic authentication for your site is OK, and depending on how secure you want you data, and how much control you have of your network, it might be. However, consider this (faked) snoop:

GET /Some/Secure/Thing HTTP/1.0
Authorisation: Basic TWljaGFlbDp1bHRyYXNlY3JldA==
How secure is that?
~> python
Python 2.4.2 (#1, May  2 2006, 08:13:46)
[GCC 4.1.0 (SUSE Linux)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import base64
>>> base64.decodestring('TWljaGFlbDp1bHRyYXNlY3JldA==')
'Michael:ultrasecret'
Not very secure at all.
Rudolph The Duck Billed Platypus

Posted: Michael, 24 Dec 2006 at 11:39
Comments: 0

Rudolph the Duck Billed Platypus
Had a very shiny beak
And if you ever saw him
You would even say it squeaked

All of the other platypuses
Used to laugh and call him names
They wouldn't let poor Rudolph
Join in any Duck Bill games

Then one soggy Christmas Eve
Santa came to say
"Rudolph with your beak so bright
Won't you paddle my boat tonight?"

Then all the platypuses loved him
And they shouted out with glee
"Rudolph the Duck Bill Platypus -
You look like a Christmas Tree!"

Live Simple?

Posted: Michael, 17 Dec 2006 at 23:54
Comments: 0

Interesting social site from Philips. I'm not sure how many active users they have (or what social tribes it crosses), but they do have an interesting site. Questions like "is RSS really simple?" Question about DRM. Is time the new currency? Some questions of collaboration creation, which may miss the point. (and yet, depending on how approached, might not). Themes include: Internet & Computing questions, Lifestyle & social (I like the 4 day week idea myself :) - wasn't more leisure time, for whatever purpose, one of the dreams of progress? )

Amplee using Kamaelia Web Server

Posted: Michael, 15 Dec 2006 at 11:27
Comments: 0

Personally I think this is way cool since I didn't write any of the code :)  Sylvain Hellegouarch has written a backend for Amplee that uses the Kamaelia Web Server which I referenced a new tutorial for in a previous post. What is Amplee? From the Amplee site:

  • amplee is a Python implementation of the Atom Publishing Protocol (APP), as specified in the current draft 11.
The integration comes as two files - one is a amplee's demo_kamaelia.py, the other is the file with the Amplee/Kamaelia Web Server Code.

My intention now as a result of seeing this is to look at providing a WSGI interface to using the Kamaelia Web Server. Initially this will be basic integration, but beyond that it would be interesting to expose Kamaelia capabiities to web developers. (Especially given this can run client side) Now I've never used WSGI, but seeing that this works, and seeing that amplee can also use a WSGI based server, this strikes me as an opportunity to say that any help that anyone can give with WSGI integration would be particularly cool.

Kamaelia Cookbook: How to use the webserver in Kamaelia

Posted: Michael, 13 Dec 2006 at 20:21
Comments: 0

After a request on "how do you use Kamaelia's HTTP Server", I sat down and created a cookbook entry detailed how you use Kamaelia's webserver code. So without further ado: How to use Kamaelia's webserver !

After writing that entry, it struck me that it actually makes sense to extend the codebase to simplify the current API, but even so, being able to write a response handler like this:

def EchoHandler(request):
    return Pipeline ( Cat(request), ExampleWrapper() )

is really rather sweet :-) (You can put whatever components you like in between this that you like for example, however wacky)

Project Task Pages - Applying Kamaelia to Project Management

Posted: Michael, 12 Dec 2006 at 18:28
Comments: 0

Well, a bit later than anticipated - I should really be off to this month's Geekup here in Manchester, but I really wanted to get this down sooner rather than later. In a previous post I mentioned technical debt and my hypothesis of project management debt - caused by lack of tracking tools or reporting mechanisms. Well, we'd recently been overloaded with work on Kamaelia so I decided to see if we could apply Kamaelia itself as a project management approach.
Not as a tool, not to create software, but as an approach in and of itself. In Kamaelia, components perform tasks, and take inputs from places and produce outcomes. In this case we did the same, except in the case of a task, the inputs are people and outcomes can be lots of different types of things (code, docs, presentations, etc). Similarly we looked at the core of how we document things - like 1 line synopsis (what is it?), examples (what will be possible by doing this task), benefits (why would you use a component), and applied that as well.
The upshot is a collection of Project Task Pages linked from our Projects section of our developer console.
The core is that given 3 things:

  • Short one line of what the task is designed to achieve/create.
  • A practical, clear result of what will be possible as a result of achieving this task.
  • The context in which this task sits. Has this task any history? Is it the result of any previous tasks - either within the project or outside.
And preferably little details like:
  • Sponsor (who wanted the task achieved)
  • Owner & Developers
We can track more or less anything we like, be it a bug report, or organising a conference (I'm using it for this internally) as well as larger projects. We've got a rather over the top template which describes the "full on" version, where you can track status, and preferably use a task log, but the overall feel won't be too odd to anyone who's been asked to use a bug tracker to track new feature development.
The upshot though of this, is this: can it actually make your life easier? Well, it seems it can. I've been able to produce a summary of the past 9 months work in about 10 minutes, and I've been tracking our involvement in a conference in this way, as well as a dozen other things over the past couple of weeks - something that would normally overwhelm me. For some examples of public PTPs a few links:
So far we're finding this approach, and it does increase the transparency of Kamaelia somewhat as well. (Which is nice given the recent website revamp :) )
If any of this comes over as forced, it's just because I'm enthusiastic and want to share something we're finding useful :)
Manchester Geek/Tech/etc Events coming up

Posted: Michael, 11 Nov 2006 at 23:23
Comments: 0

Some links or else I'll forget

  • Nov 14th - Geek Up (similar to london 2.0 AFAICT)
  • Nov 18th - ManLUG. I suspect me turning up after all these years might be a bit of a shock for some :)
  • Nov 23rd - NW Start Up 2.0 - Starting an internet business? Seasoned entrepreneur? Got the killeridea? Wondering what the 2.0 this and 2.0 that is all about? Lookingfor investment? Looking to invest? ... Come along to the north-west's first and premier 2.0 networking event for an evening of conversation and potential dealmaking.
  • Dec 2nd - DNSCon - DNS9 is the ninth running of the data and network securitycouncil conference. This being the annual meeting of UK securityprofessionals and interested individuals. The UK’s longest running openinformation security conference provides an opportunity to find outabout new threats to information security. Now in Manchester instead ofblackpool.
    running concurrently with...
  • Dec 1st-3rd - Continuity - Continuity is a free security convention open to anyone with a interestin computer security, hacking, phone phreaking, telephony,cryptography, internet security/privacy, urban exploration, socialengineering and related subjects.
  • Also Manchester 2600 - The Manchester 2600 meet takes place on the first friday of everymonth. It is aimed at everybody with an interest in computer security,telephony, hacking, phone phreaking, cryptography, internetsecurity/privacy issues and related subjects.
Flying Spaghetti Monster shows his Noodly Appendages in Germany!

Posted: Michael, 11 Nov 2006 at 19:57
Comments: 0

Awesome:

  • http://www.youtube.com/watch?v=vL7FcvEydqg&eurl=
Technical Debt (links)

Posted: Michael, 10 Nov 2006 at 17:05
Comments: 0

Some links on technical debt. Storing these for later because I've reason come to the conclusion that there's a very similar concept in project management which I've decided to call "management debt", and the question of how to crawl back out of that effectively is probably similar to the one for technical debt. (technical debt is where you cut corners in order to ship - eg testless-code - you will still need the tests later - which will cost the same as before, but also more because the code probably needs changing to be testable or rewritten test first later - so you really are paying back with interest). Management debt is where you just continue writing code, and skip certain aspects of project management in order to get something working. (ie you sacrifice aspects of project management rather than code quality. (eg "code" or "report", "tests" or "report")

"Microsoft will recommend SUSE Linux Enterprise for Windows-Linux solutions"

Posted: Michael, 3 Nov 2006 at 00:12
Comments: 1

Is it April? Novell & Microsoft teaming up to make Linux more attractive for windows users ? (links are to Novell's FAQ, and to Novell & Microsoft's press releases).

Running on someone elses platform - scammers uploading and running PHP phishing on the cheap

Posted: Michael, 2 Nov 2006 at 10:19
Comments: 0

Some people think that in the future, generally speaking, we (a very amorphus indescript "we") will be running all our systems and services of their (another amorphus blob) platform. People point to web services as one key example where this is true. Another place that's pointed at is places that will host mashups. All well and good (well, actually, I'm not personally enamoured with the idea, but that's not what this post is about). However I recently, by accident, discovered what I believe to be a malware example of this. They're doing it for personal enterprise reasons, and they're risking a lot by doing so. Why? Because they're a phishing scammer.

Yep, I received a phishing email, and the fake login.php appears to have been placed on an innocent third party's website - because it allowed uploads (anonymously or otherwise), and the upload location will run arbitrary php files. (Probably due to a bad gallery style application in this case))

How long have people been doing this? Dunno. I tend to file all such scam mails in the circular inbox, but it's a new one on me. Checking inside there shows that all the others I still have copies of are all using cgi-bin's which is a lot harder than just uploading a file and saying "run this".

The scammer needs no serving resources, no resources for sending the emails, in fact the barrier to entry is almost non-existant - just find a site that allows uploads and runs arbitrary uploaded code. People complain about windows machines allowing arbitrary execution of code, but this is far worse - this is directly akin to saying "hey, upload random code to me and do what you like". It's more or less the modern equivalent of putting a machine on the internet running telnet or ssh, but without and password set for root.

It's also rather nasty - someone who runs a server in such a mode, also probably won't be able to do the correct forensics to track down where things came from. Delightful. If anyone's curious, the subject line of this phishing scam is "Please authenticate and update your Amazon.com account by checking the link below immediately" - you've probably got a copy.

As a result, please, unless your system for uploads is designed for handling code, please have someway of ensuring that whatever is uploaded cannot be run. This comment comes around with every mechanism for sharing, and I almost can't believe you have to say it, and yet you do. Hopefully though someone from Django & TurboGears will go "ooh, if we provide a safe mechanism for doing that, everyone will want to compete with us and implement it too " :-) It won't solve all the problems, but it'd be a cool start.

Computer Science Unplugged - Non Computer Based Activities For Children

Posted: Michael, 29 Oct 2006 at 11:25
Comments: 0

Just came across this site - Computer Science Unplugged - which is essentially about teaching the principles of computer science to children, but by definition is probably useful for a wider audience. (They've also published about how well this has been recieved in practice). After all, conveying that information to children has to be clear, interesting, simple and fun. Those are all qualities useful in lots of other surroundings as well - for example when communicating a technical subject to any other group - especially non-technical.

That said, I still think the best single "textbook" on computer science is the Cartoon Guide to Computer Science, however it looks like this "unplugged" book is much more practical for showing ideas to a group or as inspiration for other approaches. Think: Royal Institution Christmas Lectures, or "Think of a Number", rather than "traditional lecture".

Interestingly I came across that site via LULU (the unplugged book is here)- a publishing site which was apparently founded by Bob Young (one of the founders of Redhat). Incidentally lulu also appears to have 3 books on ubuntu that looks quite nice - on desktops, servers and one on packaging. I don't know what they're like, but given their aggregate cost is about the same as 3 beers in london, that's pretty good going!

Open Source at the BBC - when, why, why not and how

Posted: Michael, 28 Oct 2006 at 15:24
Comments: 0

Well, this is the text version of the presentation I gave at Linux World. It should be borne in mind that this has a mixture of both personal opinion, and statements about the BBC. It tries to be as impartial as it can be and is written around the point of view that it should be interesting to businesses (of all kinds), rather than push any particular agenda. Furthermore it's also based on arguments I've seen taken as good business arguments outside the BBC as well inside. As a result this post can't be taken as BBC policy, but is hopefully a useful snapshot of how open source is viewed in a number of areas of the BBC.

A corrollary of this however, is if it looks like there's anythingcontroversial here, it is extremely likely that my description isinaccurate and a formal BBC position would be somewhat different!

Part of the reason for this is because there is no special policy with regard to open source, and if one is formulated, it obviously may not mesh with what's said here. That said, at Linux World, many people did find the presentation useful and interesting, perhaps because of the aims of more business focussed and impartial than perhaps normal. It's probably useful to people running a business for a number of reasons, and engineers working inside a business as for good reasons when to use/improve/originate open source, and when not to do so.

Anyway, without any further preamble, the document: (after the "fold" as some people say!)
LONIX Users Meeting

Posted: Michael, 25 Oct 2006 at 16:02
Comments: 0

I'll be around at the LONIX User's meeting this evening which is described on their next meeting page. Any other Python people interested in meeting up are welcome to say hi! :-)

Linux World, London, Olympia 25-26th October

Posted: Michael, 25 Oct 2006 at 00:20
Comments: 0

Just a note to say that I'm giving two of the "business briefings" at Linux World, Olympia over the next two days, first in the morning of the 25th with a repeat on the 26th. I'm also likely to head towards the London Linux users group meeting there in the evening of the 25th and if any other pythonistas are likely to be around, it'd be good to meet up. I'm planning on putting the contents of my presentation up on this blog after giving the talk. I'm, unsuprisingly, talking about open source at the BBC, use, collaboration, origination, and talking about this in terms intended to be relevant to companies seeking to learn from my experience both at the BBC and in the private sector before joining the BBC. As I say, I'll be putting the content of the talk up after Linux World is over (probably on friday).

Kamaelia in this months Linux Format

Posted: Michael, 19 Oct 2006 at 23:25
Comments: 0

Blatant promotion - I wrote an article for Linux Format on Kamaelia, specifically focussing on the whiteboarding app, but also covering some of the background on how components worth etc, which has just been published today in this month's Linux Format. I think the article/tutorial comes out pretty well myself :-) There's also an interview with yours truly in there with possibly the 2nd scariest photo of me I've ever seen.

Shared Markov Chain Chatterbox

Posted: Michael, 17 Oct 2006 at 00:47
Comments: 0

I wrote this as an example of a relatively simple, but non-trivial network protocol. It creates a server that sits there waiting for connections. Anything that you type at it updates the markov chain for anyone/everyone connected. The protocol handler itself could be bolted into an IRC bot instead so you could have a deranged bot sitting on a channel which talks vaguely on-topic (but relatively - not totally - incoherently) most of the time. I thought I'd blog about it because it makes quite a nice fun/simple introduction to Kamaelia in it's own special way. The markov chain used is based on the one here. (courtesy of a google search)

Technology to beat Pirates on the High Internet? (or services to make people happy?)

Posted: Michael, 13 Oct 2006 at 12:59
Comments: 1

People clamouring for DRM systems continue to miss some key points. Two links to two articles which are interesting counterpoints to each other. One claims technology will win the day pointing at legal services as successes. It misses the reason the legal systems are a success are because of service not technology. The other points out the reason why teens (especially) engage in infringment is because they have no other real option - no one is providing them a useful service. The former group should look at the latter and think about how to deal with this.

What have you got to hide?

Posted: Michael, 12 Oct 2006 at 17:41
Comments: 0

Some people wonder why I don't talk about my private life online, or 'silly things' like whereabouts I live in the world. The response  I generally give is "it's private". Some people however seem to think that in the modern age privacy is a bad idea, that if you want it you must have something to hide - as if privacy isn't a basic human right (Toilets have locks for a reason). Bruce Schneier's written an excellent article about privacy which sums up the debate as liberty vs control. I personally think that it really boils down to privacy matters and if its none of your business, it's none of your business no matter who you think you are, however aside from that final summing up I think its a great article.

RSS Sinks, RSS routers, Simplifying mashing up

Posted: Michael, 6 Oct 2006 at 17:35
Comments: 2

There are many RSS sources on the web, and people are beginning to look at them as a form on unix pipe for the web. Whilst this isn't a bad analogy, it misses a number of things about pipes (especially graphline type pipe systems), which would make things more powerful. We've got a specific need here, so I'm thinking "what's the most Kamaelian way of doing this", and starting from the perspective of "what's missing". The rest of this post discusses the editted highlights of my thoughts.

I got my old email address back!

Posted: Michael, 3 Oct 2006 at 01:32
Comments: 0

An email address I had many moons back I now have again :-) You can now reach me once more at zathrus >A.T< mad >D.O.T< scientist >D.O.T< com :) (Note that's zathrus with a "u" not spelt correctly)
Sorry about the obfuscation, replace >A.T< with @ and >D.O.T< with a . (mutter nasty things about spammers)

OO Prolog?

Posted: Michael, 28 Sep 2006 at 23:51
Comments: 0

Looks like I haven't been paying attention to the prolog world. Since I last looked they've also gone all OO. Just heard about logtalk, which strikes me as interesting and perhaps coming back to. Description from freshmeat: Logtalk is an open source object-oriented extension to the Prologprogramming language. Integrating logic programming withobject-oriented and event-driven programming, it is compatible withmost Prolog compilers. It supports both prototypes and classes. Inaddition, it supports component-based programming throughcategory-based composition.

So not quite the prolog I originally learnt then ... :-) Or perhaps as prolog would say: no. Mind you, I've always thought that if prolog could be made useable by the average person that it would have some really interesting applications for users. After all, being declarative it does allow you to do a lot more by doing a lot less. You've got to wonder what the implications of a tutorial on event based systems in prolog where messages to objects are events have for other systems.

Kamaelia 0.5.0, Axon 1.5.1, Article for LXF

Posted: Michael, 24 Sep 2006 at 11:25
Comments: 0

Well, the release candidates for Kamaelia 0.5.0, Axon 1.5.1 have been created merged with the megabundle and uploaded to sourceforge, next comes the steps of finding what's broken (esp by last minute additions), and fixing. Meanwhile I've also written the Kamaelia article for Linux Format (december issue probably!), which is really cool. It focusses on the whiteboard largely, what you can do with it, and also has a section on how Kamaelia works from basic principles through to building graphlines.

How not to give a talk at a Conference

Posted: Michael, 20 Sep 2006 at 16:31
Comments: 0

I'm currently sitting in the session Scaling Up Open Collaboration and was expecting the two presenters to run a session, to run a talk. What have we got instead? We have a "video". This has instantly turned me off. Almost everyone in the room is now concentraing on their laptops, and not the message of their video. Despite the clear amount of effort and love that they've put in, they've failed completely to get their message across, which is a shame. If you run a talk, talk. I've been in bad talks, depressing talks, but all of these have been more compelling that smeone playing a video at me.

Daleks

Posted: Michael, 21 Aug 2006 at 20:57
Comments: 0

Five million Scary Cybermen jumping through the void,
One fell badly, and whirled and whirled,
Rose phone The Doctor and The Doctor said,
"No more Cyberman taking over the world"

Would you fly to the US right now?

Posted: Michael, 12 Aug 2006 at 00:42
Comments: 1

That's the question on my mind right now. I'm supposed to go to the US for work reasons (Foo Camp) which is undoubtedly (almost?) a once in a lifetime opportunity, but does it outweigh the concerns I have over the fact that I'll have to check in my laptop into the hold ? (having had one data recovery experience this year, I don't want to risk a second) More importantly does it outweight the concerns my family have for my well being - that is they don't want me blown up ? That I really don't know right now. I do know that I'd planned on flying on one of the airlines that had apparently been targeted, which doesn't really inspire me.

The Long Tail Paradox - You don't need a tail

Posted: Michael, 9 Aug 2006 at 09:15
Comments: 0

This is something I've been itching to write for a while, and I'mnot sure I've got my analogies and descriptions right yet, but the gistis right more or less right (perhaps less than I'd like), the long tail sucks, viva le long tail...

The long tail - or rather Zipf's Law - has been getting a lot ofpublicity recently, which is nice in a way, but also suprising inothers. I first came across the long tail 8 years ago when I startedwork at the Janet Web Cache Service in a variety of papers from the 3rdweb caching workshop (which had been held in Manchester just before Istarted working there). The one thing that struck me was the fact thatZipf's law is a fundamental aspect of human behaviour (as fundamentalas fire).

Choose Your Own Programming Adventure!

Posted: Michael, 7 Aug 2006 at 00:45
Comments: 2

I made the offer to write some exercises to teach python programming atOSCON, specifically in the form of being able to create a "choose yourown adventure" style game using python. So, here we start! By the endof this first entry, you will be able to write a basic "choose your ownadventure" computer game, even if you've never written a line of codebefore.

Creative Coding

Posted: Michael, 5 Aug 2006 at 22:32
Comments: 0

I've now got a focus for this blog - creative coding. This is a consequence of spending a week at OSCON, talking to lots of people, making a few commitments for a variety of people, and trip to a bookstore and deciding to buy The Complete Idiot's Guide to Creative Writing, and also Kathy Sierra's keynote talk, where she said "think of something you're passionate about that isn't code". Well, actually there is something I'm passionate about, that is creative, fun, and artistic, and I'm sorry, it's coding. For me coding is creative, and something to be passionate about. This viewpoint firmed itself up somewhere halfway back to the UK, and I decided that that is what this blog should be about.

Changed over to using Dojo Toolkit for editting

Posted: Michael, 5 Aug 2006 at 21:29
Comments: 2

Small changes can make a big difference. For editting of new posts, I'm changing over to using dojo toolkit. I'll see how this goes for a while before rolling out elsewhere on the site.

OSCON 2006, Portland, Oregan

Posted: Michael, 23 Jul 2006 at 17:22
Comments: 0

Well, here I am, in portland, after travelling for nearly 20 hours, I got here. On the clock I left home around 7am and arrived here around 7pm, but travelling over an 8 hours time difference means it was 20 hours. That led me to the rather random thought - sometimes employers bemoan the fact that there's only 24 hours in a day, wishing there were more - well yesterday I had a 32 hour day. If you simply want longer days, put everyone on a boat, and send them westwards. Can't wait until the working day is done? Go the other way round the world. :)

Cerenity: Wiki

Posted: Michael, 4 Jul 2006 at 02:18
Comments: 1

Finally getting around to putting together a proper wiki engine out of the various pieces of code I have, now that the wysiwyg editor in a browser is becoming sufficiently capable. I'll deploy something soon, once I've integrated the capabilities model inside this blog engine with it, but for use behind a firewall, it's sufficient.

Hard drive death

Posted: Michael, 18 Jun 2006 at 23:01
Comments: 3

Hard drive death is a terrible, terrible thing.

My hard drive on my personal machine recently died. The problem with this is I've been using this machine as my main workhorse for the past 12 months, and despite good intentions to back up, good intentions have repeatedly failed to materialise. This means, realistically speaking, if I do nothing, I've lost 12 months of my electronic life....

Create Cool Code

Posted: Michael, 11 Jun 2006 at 14:10
Comments: 0

I hate being asked the question "What do you do for a living?" - it's a question I like to answer literally, to get to the heart of the anwer. It's not the question per se, it's the follow ups I dread. I've now realised this is as much about my answer though as it is their reaction to it.

Foo Camp!

Posted: Michael, 9 Jun 2006 at 02:48
Comments: 0

Well, rather surprisingly, but nicely so, I've been invited to Tim O'Reilly's FOO Camp this year. I've accepted the invite, so hopefully I'll get to go and see a whole bunch of really interesting people, chat about lots of things - hopefully excite them with what we've been doing and get excited about their stuff.

Summer of code

Posted: Michael, 2 May 2006 at 01:15
Comments: 0

So far this seems to be going pretty well. There does seem to be interest in our projects, but not unmanageable at this stage. I've been able to do other things as well, and realistically speaking it's this week that is likely to be the issue. One person seems to be particularly interested in doing one of the projects we view as pretty key, and there's even interest in some ideas that I thought would get a luke warm reaction.

Some links I need to follow up on.

Posted: Michael, 24 Apr 2006 at 00:22
Comments: 0

Been a very interesting evening in many respects. Some links I need to follow up on as a result follow:

Blog Spammers

Posted: Michael, 15 Apr 2006 at 11:43
Comments: 0

Why do they bother? Especially if their posts/replies won't ever show up? There's one very persistent (in the same was a bee bangs its head against the window - ie futile and stupid looking) spammer who keeps on trying to spam here. They've provided sufficient material for spam bayes to block them automatically, so that will be integrated shortly, meaning that no one will see their pointless spams.

Going offline

Posted: Michael, 29 Mar 2006 at 19:51
Comments: 1

This blog will be offline for a while (probably just a few days, hopefully less :-)

Python Meetup Manchester

Posted: Michael, 18 Jan 2006 at 22:32
Comments: 9

Lass O'Gowrie Pub, 7pm, Wed 8th Feb. I don't know if this'll be the first meetup for pythonistas that's happened in Manchester, but it'll be the first one I've been to there - should be fun :-) If you've been to one in London you know pretty much what to expect - a bunch of geeks in a pub talking, well, about python and stuff they find interesting. If you've used/written/found something you think's cool and want to talk about it & show it off please do.

Other projects I'm working on

Posted: Michael, 5 Jan 2006 at 00:54
Comments: 0

Kamaelia (work) (site, bliki)

Cerenity (personal) (site, bliki)

First Post!

Posted: Michael, 5 Jan 2006 at 00:15
Comments: 0

Seems rather obligatory to have a post labelled first post. This is the first post of this blog, which will be my general "stuff" blog. I also have the cerenity blog, which I'd like to move back to being a cerenity blog, and the Kamaelia blog is taking on a very different form, and again, more random, more personal things don't really have a place there.