Goodtimes for Powerset, Hard times for Hadoop?

Yahoo’s troubles and a recent Microsoft acquisition could be bad news for open source software that enables “internet-scale” computing.

Hadoop is a project to build an open source version of the infrastruture that Google uses to process data. It provides a huge filesystem that can be distributed over dozens or even thousands of computers (analogous to GFS), as well as support for processing all that data in parallel in the same way Google does when they build and update their index of the web (using MapReduce). It also provides HBase a distributed database that is built on top of the filesytem in the manner of Google’s BigTable. Hadoop is a spin-off of the Nutch project to build an opensource search engine that could index a significant portion of the web.

Most of the work on Hadoop and HBase has been supported by Yahoo, and a lot of the recent work was supported by a semantic-search startup called Powerset. In fact, a quick look at the personnel on the project shows that it is dominated by people from those two companies.

Given that Yahoo is in turmoil, and has been showing some signs of reconsidering their search business, and given that Powerset was just bought by Microsoft, who likely already has its own infrastructure for these sorts of applications, I have to wonder what will happen with Hadoop.

Netflix Discontinues Profiles: WTF

Like everyone else on Netflix, I got e-mail yesterday letting me know that they’ll be discontinuing their profiles feature. This feature allowed customers to set up separate DVD rental queues for other members of their household under one account, and let the owners of those queues rate movies and get recommendations separately.

We haven’t really used the feature, but I find the move baffling. One commentator suggested it was a move to boost revenue, because it might drive some users to create separate accounts.

That may well be, but in the process of doing so, Netflix is destroying the investment of any customers who made use of profiles and were in the habit of rating DVDs. Presumably these ratings will end up pooled for the whole account., which is likely to reduce the overall quality of ratings for each member of the household (imagine a mother who rented a couple of Kung Fu movies a month, and what will happen when that gets mixed in with the dozen of teen and action movies the rest of the household rents). It also seems to reduce the amount of aggregate information that their recommendation engine has to work with.

It’s puzzling why Netflix is willing to piss off its most dedicated users. Perhaps it is easier to do if they rent more disks and therefore drive up costs. It could also be that Netflix has concluded that the value of the ratings and recommendations is lower than many outsiders thought. It was assumed that users would be less likely to switch to another service because they were getting value from the recommendations derived from their ratings. In addition, new users would start to see immediate value for rating movies because of the vast store of ratings data from other users that Netflix had to work with.

I have to wonder if somehow it has something to do with their attempts to build a video on demand business.

Roku Netflix Player

Last month I tried out Netflix’s streaming video service. I thought the price was right (free with most Netflix subscription plans), but the video quality was poor, like an old VHS tape recorded from TV. The selection was so so as well, but the all-you-can-eat price is right, especially compared to the $3-5/pop it costs to rent something on iTunes.

They.’ve recently addressed another issue. Roku has just released a $99 set top box that plays Netflix on demand. Netflix doesn’t have HD content (and I doubt that most people have internet connections that would be up to streamed HD), but the device is HD ready. I’ve been thinking of replacing the computer hooked to our TV with a smaller, more efficient device, but I’d been leaning back to using a computer so we could use netflix’s streaming. The Roku device solves that problem, but, unfortunately, doesn’t seem to play anything else. Too bad. I’d happily pay another $25-50 for something that would play content from a media server.

WordPress Updates: Ugh

I have to update our WordPress installs. I’m not looking forward to it.

Upgrading wordpress itself isn’t too bad (though doing it 4-5 times gets a little old), but then I have to gather updated plugins (something I may not have to do much longer, since the latest version of WordPress seems to have an integrated updater.

The real PITA though is updating my theme. A few years ago I decided to use K2, which was under active development. It’s still under active development, and it seems like each time I update WordPress, I have to update K2 too. I’ve done some customization (inserting some AdSense code, and integrating code required by a specific plugin, and so I have to propagate my changes to the themes for each blog.

Wish me luck. I think the last time I did this, I only updated a subset of our blogs.

Netflix Watch Instantly: Good News & Bad News

A few months after Netflix launched their “watch instantly” streaming feature I tried to get it running on the computer hooked to our TV. I didn’t get very far though, since it required WinXP, and the computer was (and still is) running windows 2000. In addition, our DSL speed at the time was only 1.5Mbps, which wasn’t enough for their top quality.

I decided to give it another shot though. I haven’t upgraded the TV computer, yet, so I thought I’d try and run it from VMWare Fusion on a Mac laptop I use. The good news is that it works — I thought that DRM restrictions would prevent it from working in a virtual machine.

The bad news is that the quality is truly mediocre, even using our 3Mbit connection, and from what I can tell, a faster connection won’t help.

It’s too bad, because the price is right, unlike the $3.99 rentals on iTunes. I only wish Netflix had the option to queue up a higher quality version to watch later.