Monday, October 8, 2007

Can your clustered software platform do this?

Disclamer: I actually know nothing about cluster software environments. And what I call "threads" below aren't actually threads.

Suppose you're developing some sort of clustered multi-processor system. The sorta thing that runs on many computers, and uses fairly complex inter-node interaction with various bits of shared state and whatnot (i.e., something more involved than parallel processing jobs with a message bus in-between). Say, it's a clustered Instant Messaging server, or some sort of a HA database system.

For development, you're running many processes on your workstation, or have some sort of a test cluster thing. All of that is using sample datasets, debug clients, etc.

Now, the problem is that changing code while developing this system is a nightmare. Most of the time, any single change will require recompiling the thing, restarting parts (or all) of it, and then going through all the motions necessary to get it into the desired state. At which point the freshly-written code is debugged, new issues are found, and the whole cycle needs to be repeated. A nightmare.

Well, today I found myself in just such a situation: developing a clustered system that does something or other in many processes spread accross multiple machines. Only (of course) it was all being done in Erlang. And it was a pleasure.

I would be poking at the cluster through Erlang shell (an interactive yoke similar to what one finds in Python or Perl, only this one was running as part of the cluster, so I could easily examine states of various bits of code running on various machines, send them commands, kill & restart threads, etc.) When this poking found an error,
I simply fixed it in code, compiled that, and then issued this command at the shell:

rpc:multicall(nodes(), c, l, [my_module]).

What this says is "on all erlang processes known, purge old version of code for my_module, and replace is it with new version from disk". What it does is, it goes to every
connected Erlang node (and since Erlang clusters tend to organise into fully-connected graphs, that means all nodes in the cluster), finds all threads currently using code from my_module, and substitutes new code from disk instead. And it generally does it without affecting running in-memory state at all. Let me just repeat: I can change code a thread is running without restarting it, while keeping all of its associated in-memory data intact.

This way when I find a bug, or need some new feature, I can add it to my running cluster and have it available straight away, without any sort of restarting or messing necessary. It's very cool.

Monday, September 17, 2007

People who can't stop laughing on youtube.

After a very productive morning, here are some videos of people who cannot stop laughing:

  1. Silly americans
  2. German weather anchor
  3. Barbra Streisand

Tuesday, August 28, 2007

In the news: Stubborn corporation persists at banging its head against a wall.

Back in 2002 or so, when Nintendo was the only kid on the portable multiplayer block (for oh so many years) and getting massive bux out of it, some execs at a non-insignificant mobile phone maker got The Envy. Here they were, busting their ass, trying to make more and more annoying innovative mobile phones, trying to convince punters to buy the latest in glitz, and only really making any money on the hardware sales. Whereas Nintendo would simply barf a device, and then simply sit back and bask in all the revenue coming from every single game cartridge sold. They didn't even need to do anything (apart from an odd marketing event), with all the hard work of coming up with game concepts, carrying them through to completion, and recouping development expenses left to other people. Sweet gig.

No wonder Nokia execs got The Envy. Which compelled them to spend several years, countless millions of whatever money unit they use in Finland, and an immense amount of karma on a series (well, two, actually) of devices under the N-Gage brand. Only to have the whole thing flop, much to the amusement of game journalists and geeks worldwide.

Rather than quietly forget their failure and move on, Nokia kept the ship afloat by turning the brand into a platform running on their smartphones. And fuelling it with monthly press releases, mainly designed to feed Wall Street.

The latest piece of the saga came in the form of an article in The New York Times. It's a fairly entertaining piece about the new generation of N-Gage brand, launching this week. The article is full of enthusiastic observations about all the new and innovative stuff Nokia prepared for its comeback. Let's see:

In 2005 and 2006, members of Nokia’s N-Gage team spent time with Ideo designers in San Francisco. The groups took research trips to six cities around the world, including Barcelona, Tokyo and Shanghai, to study how people played games and what they wanted from games on their phones.

Well, that's fair enough I suppose. Considering how much attention was paid to needs of potential gamer customers in the previous attempt, it'd be foolish not to do at least some due diligence here. And what did they come up with?

The Ideo and Nokia executives concluded that users mainly want to play against their friends and, at the very least, they want to know the skill level of their opponents. ...

Wow. Research shows that players don't really like having their ass kicked by some random Japanese dork with too much free time. Who would've thought?

Here's how I would like to see them arrive at a solution for this perplexing quirk of the psyche: A cool-looking head of creative design director dude, wearing shades that aren't even in the shops yet, gathers his creative design innovation team and presents the problem to them. The whole team then works furiously for a month, going on meditation trips to the Andes, brainstorming in pristine white rooms, experimenting with mind-expanding drugs, and even going as far as to consult children. Then they come back with a beautifully presented document with the solution:

… the new N-Gage permits users to see what games their friends have on their phones and whether they are online. They can also see how many points a person has earned in the game, as well as how much time they devote to solitary play versus group play.

Ahem, apparently in reality they just wrote up a proposal replicating what every other online platform in the universe does, and then probably fudged some timesheets and billed Nokia for sushi deliveries to the office twice a week. But did they stop there? How could they, Nokia has so much money!

The researchers also asked players what their greatest frustrations were. High on the list was buying a game that turned out to be disappointing.

No one ever gets frustrated with how much The Simpsons Movie sucked, after going to see it instead of drinking a few pints. But games now, they are so much different. Who would've thought, without an expensive research effort, that people get a tad annoyed at all the games being sold to them that turn out to be utter shit. So the smart folks at Ideo had to come up with an innovative solution to this problem:

In the new N-Gage service, customers will be able to sample games free before buying them.

Mind boggles. This service is going to be so full of innovation, it'll take the industry by storm. I'm off to buy some Nokia shares right now.

Wednesday, August 22, 2007

Twitter fate

Twitter seems to be the site that pioneer "microblogging", which is just a fancy term for "flaunting one's life on the interweb with minimal effort and maximal impact". It doesn't actually have anything to do with blogging, and instead evolved from those status messages that can be set in most instant messaging systems. Like, you know, workin. send utube links svp.

Of course, the main deficiency of IM status is it's discreetness. People only really see it when going to IM someone, and even then it's only the very last status message that is shown. Which is totally inadequate for an average egghead, who'd rather stand on a busy street corner and scream through a megaphone that he just popped out to buy an apple.

Twitter offered a much better alternative. It renamed status messages to updates and decoupled them from instant messaging. Users were encouraged to create updates through a website, or by messaging in IM, or even through SMS (at a premium rate, of course). Such updates would then be broadcasted to the entire world (through IM & SMS, of course), and stored on a public interweb page, in case anyone missed the broadcast.

This turned out to be a perfect outlet for all attention-starved nerds, who could finally get some release by broadcasting self-indulegent crap and pretending that anyone cares (not unlike this blog). Of course, since such individuals tend to gather in groups, their subconscious desire to keep the fantasy going compell them to react to each other's updates, thus keeping the cycle going.

In addition to Twitter, which allegedly has millions of subscribers, microblogging craze is supported by various Twitter clones (techcrunch et al seem to be promoting Jaiku), as well as big names of time-wastage industry, Facebook and Myspace.

While the two giants don't have the feature implemented quite yet, they are moving in the appropriate direction. Facebook seems particularly close, with an archived, mobile-updatable status update feature. As soon they provide push functionality (which Facebook already sort of does, with RSS feeds), Twitter will be truly dead.

Thursday, May 31, 2007

Web2.0 stupidity of the day: vod:pod

Let me introduce you to vod:podbeta.

Launched in December 2006, vod:pod is a typical silly Rails-based, buzzword-compliant, community-based, mashup website. Basic premise seems to be around users contributing videos they found on other video sharing sites to "pods", which are simply rich tags. There's also functionality to upload videos directly to the site, but there are various limitations on that.

Technically, the website is ok. Going against the grain of Web2.0 philosophies, it is visually busy. There are about a million different action items on each page, all demanding attention with bright colours and contrasting images. Being a Rails site the performance isn't great either.

It does have a few positive things going for it too. It uses AJAX fairly sensibly (that is, almost not at all). Popularity (?) graphs in pod headers are quite cute. Most importantly, cluttered and confusing UI makes it very hard to waste more than about 3 minutes on the site (unless you decide to bitch about it on a blog!).

They claim to be self-funded, and are hiring. Seeing as there are no ads on the site (not that they would generate much revenue, with their visitor numbers) someone is sinking a decent amount of cash into this. And, according to their FAQ, they plan to make money on advertisement and "premium services".

Admirable effort, but no biscuit.

Monday, May 28, 2007

Setting up Mondrian's FoodMart example

So I tried to setup Mondrian's example data mart. It is included in the distribution and referred to by the documentation. It supports various databases, which means it's a Java program, rather than an SQL file. The program creates does all the DDL and then populates it with sample data.

Unfortunately, none of the documentation provides actual working details on how to run that program. After about an hour of screaming at my keyboard, the following command emerged and did the job:

C:\Pentaho\mondrian->java -cp ../workbench-;../workbench-;../workbench-;../workbench-;lib/mondrian.jar;../workbench- mondrian.test.loader.MondrianFoodMartLoader -tables -data -indexes -jdbcDrivers=com.mysql.jdbc.Driver -inputFile=demo/FoodMartCreateData.sql -outputJdbcURL="jdbc:mysql://dbtest/foodmart?user=user&password=blah"

Java is a lovely system.