My Face In Wired UK

When Wired came out in the UK a couple of months ago, I thought it would be cool to get in there for some of the work I do. The other month I was asked to be in a photo shoot about the work I've been doing in the twitter world as a developer for projects.

If you didn't buy the copy that came out in Nov 09 you can check out the crazy yellow shirt I wore here.

It mentions Twitterfeed, where i've been working with the team to re-architect their back-end to support realtime, we have implemented Pubsubhubbub subscriptions and will be looking into handling RssCloud too soon.

Saving HTTP requests for testing

I have been working with quite a few HTTP API services, so testing has been fun. Setting up lots of mock responses from URL endpoints.

Some gems like, twitter, bitly, etc have the URL's hidden in their guts and without reading around the twitter/bitly api docs you won't know, to help and save time I wrote this module to help override net/http and save the http response. The ruby library net/http seems to be used in most gems.

You can include the http headers (haven't tested fully and loading), for testing redirects/errors. Thought I'd share with others and see if you find it useful.

Ballantyne Landscaping (Landscapes) Review - Wimbledon

I am writing this post hoping that anyone else wanting to use Ballantyne Lanscaping from Wimbledon to think again. And partly to get it off my chest. So here is a review of what happend.

Last summer (2008) we needed a new fence, so we got some professionals in. Ballantyne Lanscaping, a chap called Stephen Ballantyne. The job was completed very quickly and the fence looked good, I asked if I had to treat the wood, Stephen told me 'the wood is treated and good for a couple of years'.

Wrong - One winter and the panels are falling off, it seems most panels on the fence are not even nailed in properly and the bottom board panel is not fixed at all! Cheap materials and a cheap quick job.

I made contact and for the last month he has been saying that he would come over to take a look but today his tune changed and claimed everything he installs is treated and a good job, the neighbor must've knocked it out of place. He then hung up. All I wanted from him was to come and look at the work himself. Like any human being I thought he might want to correct his mistakes and his companies.
Above you can see the bottom panel falling off, it seems to be just wedged between the posts.

This above and below image shows how the panels aren't fully fixed in and not treated properly. If only i was told at the beginning, I don't mind weather proofing it if it wasn't.

How to beat the Pirates?

I have been talking and debating on downloading pirate videos and music for a while now, I think it would stop if there was a quick, easy and cheap alternative. Reading the Digital Britain full report today a point made me want to post this and find out what others think.

People that download the content whether that be music or video are partly doing this out of simplicity and convenience. For example going to the movies to some might be expensive, time consuming, have bad experiences (noises, damn kids) but downloading a DVD copy of film not long after is seen as easy and simple with the sites and tools that are out there. Where are the alternatives? Why can't people go to an application that serves high quality films, cheap with fast bandwidth pipes guaranteeing quick downloads, discounts, etc. If this was available I believe everyone will flock to it, like how purchasing of music through iTunes has taken off.

Reading through the Digital Britain report under the exec summary (not sure if i'll get through the whole thing), point 45 talks about tackling Piracy and downloads, I look forward enthusiastically to:

"The objective of the Government’s policy is therefore three-fold. Firstly, to provide a framework that encourages the growth of legal markets for downloading that are inexpensive, convenient and easily accessible for consumers."

To provide a framework that encourages of legal markets and services is key, couldn't agree more. What do you think?

Guardian Open Platform - My Mashup

This morning I was invited to the Guardian at their new shiny office at Kings Cross for the launch of Open Platform API.

They have created an open API to access all, yes all of their content. It was said at the end that it is in beta and that they will be reviewing registrations for the API key. It's a great playground of information.

They provide methods to search by tags, retrieve items and articles. Even full text search terms too.

This allows developers to mashup applications vertically slicing up data in different ways. What I find brilliant is the tagging they have, you can filter by editorial tags such as /world/barack-obama, /global/reviews, etc. You can read more at their site. So if your trying to create nice content sites, different visualisations of data I do recommend you to look at their API.

I was asked to help with the creation of the Ruby library which is a simple start to make it easy for ruby developers to get up and running. With this library I created a mashup which probably took me a day to create.

Guardian Reviews - Is an application that queries Open Platform for the live music reviews, general reviews based on films and reviews. The data made available by the platform then allowed me to search twitter for relevant conversations that took place that are relevant. It is an example app not really a super duper production quality app :)

Thanks to Tom Armitage for better designs than my bordered divs!

Running an app on Heroku

I don't know if I'm allowed to call it cloud computing but essentially to me that is what Heroku is. They call it a 'Instant Ruby Platform', it creates amazing simplicity to host Ruby applications. The site offers a way better explanation of their architecture, which is a pretty impressive mix of:

• Nginx (Http Server)
• Varnish (Http Cache)
• Some custom Erlang routing magic
• Dino Grid (their way of distributed apps across a processing grid)
• PostgreSQL for database (with replication)
• Memcached caching server

Recently I have been working on an application that i wanted other people to see and provide feedback, so it was a perfect candidate to try this platform.

I started by registering and then to the quickstart guide

Gems

% heroku rake db:migrate

After running the migrate command I got loads of "Missing these required gems" errors, after looking at the heroku docs it seems that they have limited gems installed on platform. So a simple unpack resolved that:

% rake gems:unpack

MySQL and Postgres

So the migrate then got a little further before bombing out on some custom execute MySQL i had used. It's worth remembering that any custom MySQL queries, definitions in migrations, etc won't play well with Heroku.

I changed my schema, committed to git and pushed to heroku, then the migrate ran fine.

Still a 500?

For some reason when running the '% heroku open' (which opens the app in browser) I got my ugly default rails 500 page, to debug further i needed the logs.

% heroku logs

This showed the recent logs from the application, I found that my production config was broken, fixed and pushed up and then all was fine.

Final Words

I think Heroku is brilliant and will be definitely using more in the near future. I will be of course interested in the costing model they will launch, will be it be similar to how google app engine works with requests and processing or a node size and power based approach.

A Frustration with Twitter

Twitter is an amazing application to send micro messages to lots of people that follow you or to those that have changed their settings to see all messages referenced to them. This can be slightly annoying as you only see replies from contacts you follow. I hate this because I don't follow everyone that my friends/connections do.

So when say Mike Butcher from techcrunch uk asks a question or starts a conversation around a topic, i can't see all the replies becuse i don't follow everyone that Mike does. Yes i can go to the Twitter Search function and type @mbites and run a search but then i have to decipher what is a reply to the question in mind and scan through pages of the messages.

There are some mashups around the interweb that provides some functionality around this area - and example would be Tweetree. I find this the best for threading my conversations, but i have to view my whole feed. I wanted something simple to enter an ID, user name or just browse the top/hot conversations on twitter.

There was Quotably earlier in 2008 that provided conversation/thread viewing, at the time I didn't use Twitter much. It has closed down, you can read why here.

A better overview of the situation can be found on Techcrunch where Michael Arrington discusses this topic. Check it out.

Enter my project Tweetracks, an attempt to view threads and store conversations for future search and retrieval (coming later). At the moment it scrapes all messages that are posted to twitter and creates a tree data structure, allowing easy thread views. Eventually I will index all the questions, root messages for conversations, allowing people to search them. This will be quite cool and rich. Lots of existing conversations on interesting topics are held in twitter, but i find it too hard to find them with existing tools.

I will be adding more features soon such as automatically connecting messages that didn't have a reply status id (if twitter allow the calls), ajax refresh to udpate conversations, the ability to view conversations from certain users, being able to filter the conversations using your followers and many more.