LeWeb 2010 - just a few photos from the event

iTunes still loves my second-generation iPod. And one more thing...

Photos of U2 in Helsinki

NYPD Helicopter on top of Empire State Building

Photos of U2 in Helsinki

When you are in Helsinki what is better than to take some photos of old men – better known as…

Amazon S3 and CloudFront with WordPress and DreamHost

Amazon web services

Amazon web services

For the last couple of days I have been testing out the Amazon CloudFront service using my blog as a sandbox. My goal was pretty simple: To set up Amazon CloudFront on my blog, enabling static files like images and styles to be distributed for fast access. This post describes my experimentation with it. It is not just straight forward work, and there are different approaches you may choose among. So I thought I should write a short walk through of the steps that I have taken. If you wonder what Amazon CloudFront is, or looking for information on how it could be used in conjunction with WordPress this post can give you an overview and help you getting started.
[% oiopub-banner-2-right %]

Let’s start with Amazon CloudFront and what this actually is. CloudFront is a Content Distribution Network (CDN). The service distributes your data out to multiple datacenter locations, ensuring faster access times. It works together with another service from Amazon S3 (Simple Storage Service). Amazon S3 is a data storage service where your files is stored initially. To be using CloudFront for distribution you need to have a S3 account and store the files there.

Setting up an Amazon Web Services account
As I have blogged about before, I use Amazon S3 already, so it was not necessary for me to create an Amazon Web Services account. If you do not have an Amazon Web Services account head over here to register. After you have registered, you must enable both the Amazon S3 service and the Amazon CloudFront service to your account (just follow the links and sign up).

With an AWS account and enabled both S3 and CloudFront to this account I was ready to start experimenting. Amazon Web Services is just services. This means that they only offers their services via APIs. Uploading a file to your S3 account must be done by a client. If you are a programmer it’s easy to just develop a client, but there are a lot of clients/programs around. An easy and simple client is S3Fox, a Firefox extension that that lets you upload files to Amazon S3 and manage the files’ access control. I use S3 Browser, a great application that handles the administration of my S3 files. Using S3Fox or S3 Browser makes it easy to test the S3 and CloudFront account, making sure it works. Objects (or files) are organized into buckets that can be created, listed, and retrieved using either a REST-style HTTP interface or a SOAP interface.

Update: As Andy says in a comment to this post (thanks Andy!), another client that you could try out is CloudBerry Explorer for Windows or S3HUB for Mac.

The first thing I had to do was to grab the AWS Access Key and the AWS Secret Access Key. Log in to the AWS account and click on ‘Security Credentials’ in the left menu. The first thing I did was to set up the S3 Browser account with this Access Key and Secret Key. I created my bucket and named it ‘static.larre.com‘, then uploaded some files to be sure everything worked:

Testing S3 bucket

Testing S3 bucket

Setting up CloudFront for my WordPress blog
Now I had an Amazon Web Services account, enabled Amazon S3, installed S3 Browser, created a bucket named ‘static.larre.com’, and uploaded files to this bucket. So far so good. Time to concentrate on the Amazon CloudFront and how to integrate it into my blog, that runs WordPress. There are several approaches that can be used. I have tried a couple of different approaches that I’ll try to describe.

Using Dreamhost
Besides hilarious monthly newsletters DreamHost is a great web hosting company. I use DreamHost for hosting of this blog. Last year DreamHost integrated CloudFront into their hosting service. A pretty amusing blog post from DreamHost about it is here. With CloudFront as a “Goodie-service” smoothly integrated in my hosting environment it is really simple to get started. The step is to create a cloudfront domain filling out this form:

CloudFront Domain

CloudFront Domain

What it does is to create a directory where you can put all files and directories that is going to be stored at S3 and distributed by CloudFront. A very simple admin view where you can handle simple tasks like delete, rescan etc is also part of this “goodies” service. When creating the CloudFront Domain, DreamHost creates a distribution for you at Amazon CloudFront. A distribution specifies the location of the original version of your objects. A distribution has a unique CloudFront.net domain name that you can use to reference your objects through the network of edge locations. If you wish, you can also map your own domain name to your distribution. DreamHost adds a CNAME record (in this case it was ‘static’), that has the cloudfront.net address as value. Using a CNAME enables me to use a domain name (in this case static.larre.com) instead of the domain name CloudFront provides. So looking at my own CloudFront distribution I have the following:

  • http://static.larre.com (my own mapped domain to the distribution)
  • http://d20o1nml9my7kd.cloudfront.net (the unique cloudfront.net domain)
  • http://static.larre.com.s3.amazonaws.com (the origin bucket at S3)

One of the most annoying thing about this service, is the lag from I request a rescan until my files are uploaded to S3. During the test period for the last couple of days it has taken everything from 10 minutes to several hours before DreamHost actually did rescan and upload/delete files in my S3 bucket. When DreamHost had finished scanning the directory and uploaded the files to S3, they where accessible. It seems that the only way to upload files is to do it manually within the DreamHost Admin Panel.

Using the service from Dreamhost may be a bit simple and manually copying all the needed files to the specified directory is not a very elegant solution. A more scalable solution is to make some scripts and jobs that pretty much does the work for you. Here is a blog post that describes one solution to make scripts and jobs for copying static files from Dreamhost to Cloudfront.

Adding a WordPress plugin
To work with my CloudFront Domain (http://static.larre.com), I tested a simple WordPress Plugin: ‘My CDN’. This plugin help you offloading javascript, css and theme files to your own CDN network, but only handle url rewriting not actual file transferring. Installing the plugin, activating it, and then adding three input fields under the settings section, was pretty much what I had to do (here is a screenshot of the settings page). One of the problems with this setup and solution, is that I have to manually put files in the specific directory (as described under the DreamHost section) unless I write a bunch of script-code. I think a better solution is to develop or find an already developed plugin that handles the prefixing of static content and also the upload to S3.

I’ve found one plugin: W3 Total Cache developed by Frederick Townes It’s a plugin improving the user experience of your blog by adding page caching, database caching, minify, content delivery network (CDN) functionality and more. Late last year they implemented integration with AWS and CloudFront. I have used the WP Super Cache until now, but it’s necessary to deactivate it before trying to install and activating the W3 Total Cache. Setting up this plugin is straight forward. After installing the plugin and activated it, I rolled over to the settings sections. First of all I had to activate the CDN and chose Amazon CloudFront as the CDN Type. Under the CDN settings I had to enter my Access Key, Secret Key, bucket and CNAME:

cdn settings

cdn settings

The plugin makes it easy to upload whatever static content you have on your WordPress blog to CloudFront. In the settings sections you specify extensions and files for upload to S3. During the testing period (a couple of days) I have found one issue with this plugin. I use WPtouch. It is a mobile theme, that automatically transforms your WordPress blog into a web-application experience when viewed from an iPhone, iPod touch, Android, or BlackBerry Storm touch mobile device. The default settings for W3 Total Cache captured the mobile devices and bypassed WPtouch. I solved it by adding mobile user agents in the corresponding field under the settings for page cache, minify and CDN.

As far as I can see, the DreamHost integration is not the solution for integrating my WordPress blog with Amazon CloudFront. With other needs than to get static WP content out to the edges it may be a nice functionality. But probably you will have to write some scripts and batch jobs working with the CloudFront directory before you get an elegant and effective solution. The W3 Total Cache plugin seems to be an easy and effective tool to use for mye goal. Setting up an AWS account, enabling S3 and CloudFront and adding a CNAME record is what you have to do before working with the plugin. Everything else seems to be achievable within the settings section.

Everything that is referenced with http://static.larre.com is now hosted by Amazon S3 and distributed out by Amazon CloudFront services.

Tough times for newspapers

In the wake of the financial crisis, the debate around online newspapers and the traditional newspaper model for making money has increased. Today media companies have major financial problems. The newspapers are losing revenue in several areas; subscriptions are declining, classified advertising revenues are sharply reduced, and advertising revenues fall dramatically. The result is powerful cost savings in various editorial with huge layoffs. At the same time a lot of newspapers editions are shut down.

Do you dare to cannibalize big revenues?
One of three big revenues for the newspapers and the media companies was from classifieds. During the last 10 years classifieds has moved from print to online, but the newspapers underestimated the impact of the web and did not see the transformation. Robert Weisman has a great article titled ‘What went wrong?’ where he describes how (and why) The Boston Globe was a victim of its enormous success. They had the opportunity to put their lucrative classified advertising business on the Web, but cannibalization of their huge classified advertising revenues was out of the question. What they “kicked” out of office got picked up by TMP Worldwide and later expanded into Monster.com.

One of the most difficult processes inside an organization is developing something that cannibalize itself. Schibsted, the larges media company in Norway made a strategic move setting up a new organization outside of the newspapers (FINN.no) late in the 90th’s with the goal of building up the largest online classified site in the country. The development from the print business into a multi-media environment for classified business have been a huge success moving revenue from print to online. Today FINN.no (still owned by Schibsted) is the (one) by far dominant classified player in Norway and has almost no competition at all.

Online presence for many years
Most media have been positioned on the web for many years but the challenge is that the revenue is not big enough and the companies do not manage to replace the lost revenue on paper with revenue online. From an end-user perspective it is fascinating to see the lack of innovation at online newspapers. For almost 15 years the news online has just been a copy of the print edition. When looking at the newspapers online today my impression is that newspapers are at the same stage as classified was back in 1996 (classifieds online was just merely a copy of 3 liners from print). Of course there has been some product development and product innovations. The newspapers today are trying out Twitter integration, opening up for comments and debate and to some extent starting to actually link out from articles. But as I see this no huge step has been taken. As an end-user I would like to follow topics that I find interesting inside the articles, I would like to follow persons and places. I would like the information about it to be aggregated and presented in an easy and intuitive context. I would like to look at all the photo’s taken not only the one that the author finds to fit best. I would like to buy or license the photos.

In a great post (in Norwegian) Ragnhild Kr. Olsen describes how much better suited the web is to deliver news with depth, perspective and context than print. Rich user experience built up by linking opportunities and the combination of text, images, audio, video and graphics, dialogue and interaction with the audience gives newspapers a powerful way to convey information. But as she points out the opportunities to do this is exploited in a very modest degree – or not at all.

The web is social, distributed and linked
The media landscape is changing and it’s changing a lot. Breaking news hits social platforms like Twitter with stories and pictures, big exclusives stories goes online first and getting distributed all over the web (look at Guardian), you and I have the ability to produce and distribute higher quality videos than traditional broadcasters and so on. Some weeks ago, Google Chief Executive Eric Schmidt delivered a closing keynote at the Newspaper Assn. of America annual conference in San Diego. Some of his advise to newspapers is that they have to get used to the idea that they are not just generators of trusted, professional content, but also aggregators of the new kinds of information the Web has enabled (the collectively edited knowledge structures like Wikipedia, and user-generated information like blogs, images and online video). Jeff Jarvis the author of What would Google do?, describes the shift from content economy towards a link economy in which content only acquires value if others link to it. One interesting thing about this shift is that today the linking is all about linking documents or web pages.

The creator of the World Wide Web, Tim Berners-Lee’s vision of the Web as a universal medium for data, information, and knowledge exchange (the semantic web) will reinforce this trend and the possibilities and changes deriving out of this is hard to imagine. Here is a great Video of TBL where he talks about the semantic web or open and linked data.

Anders Brenna from Teknisk Ukeblad has written an excellent article (in Norwegian) titled: How is the future of journalism. In his article he discussed the role of journalism in a media landscape in dramatic change, argues that news must be free and discusses how it still is possible to make money.

Know your user and advertising will follow
Online advertising is huge and the spend will increase during the next years (as I have blogged about here). But one of the problems is that the online newspapers are not set up to take advantage of this trend. I believe that in the coming years there will be a lot of innovation in this field. Focus will be on behavioral targeting and social media to get more precise targeting and effect. To take advantage of this you need to fulfill at least a couple of things; First of all you must know your user. How the user is actually using your site is important and also in what mode the user are in at the time of the visit. It is surprisingly few online newspapers that actually know much about their user. How often are you able to actually register or log in to your favorite newspaper online? And then if we look at the mode what is the primary goal of the users visiting the newspapers? Is this mode a good match with more demanding advertisers looking for more effect? Probably not.

The development and innovation in display ads has been next to zero. It’s amazing that I still go into websites being presented with boring banners. It’s about time someone starts to innovate the formats. A couple of months ago I wrote a post about how Google is changing the Yellow Pages Industry They know advertising business better than all of us. Looking at how they manage to erase the differences between the ads and other content is something to transform to other areas. That means you must erase the differences between editorial content and ads.

The lack of innovation should be met by pushing the limit
When I look at online newspapers I often think about the mobile industry. For 15 years the mobile industry as in producers of mobile phone had the ability to innovate. Year after year they produced new phones that was just a copy of older phones. But along came Apple. The interesting thing about Apple’s appearance in the mobile industry is not solely the iPhone as a product. The phone clearly innovated the phone as a product in ways that Nokia, Siemens, SonyEricsson among others was not able to do. But Apple actually turned the mobile industry upside-down by innovating the phone, innovating the business model, innovate packaging, distribution and partnerships. They actually focused on several main areas, looked at how this fit together and combined this with great user experience.

The New York Times article In Europe, Possible Survival Lessons for U.S. Papers highlights the Norgwegian web site VG Nett. VG Nett has a profit margin of more than 30 percent and rivals Google as the most popular Web site in Norway. They have really been innovative, pushing the limits for what a newspaper online could do. They develop new products and services not solely focusing on purely news, and experiments with and engage end-users in clever ways. VG has developed Nettby the second largest Internet-based community in Norway (after Facebook) with more than 980,000 users and profiles (as of January 3, 2009). That a newspaper is behind the largest community online in Norway (besides Facebook) is impressive. VG was the first newspaper to publish photo from the tsunami in 2004. The photo was taken from a phone and sent as a MMS. This seems to have started a strategic move towards embracing user-participation in the Newspaper.

Innovation is not a free lunch
A lot of writers and blogs are saying that newspapers need to innovate. But I hardly read any that tells how they can do it, or writes anything about how to find a healthy business model. Innovation is not an easy task that you just do. It’s difficult. For the last couple of years I’ve been working a lot with innovation. My opinion is that for an organization to be innovative you must have processes and culture that enables ideas and creativity. You must have systems that captures ideas, figuring out what ideas to work on and an organization that has the ability to conceptualize and implement them. Innovation must be a central part of how you work and internal innovation processes and idea management tools must be natural and integrated part of the business.

Tough times must be met by tough decisions
Putting a closed door in front of your business is a dead end. The shift from the content economy towards the link economy will affect distribution strategies. Content need to be distributed and you need to be the best provider of it (look to what Guardian is doing). Pushing the limits for what a newspaper online could do as VG does is probably necessary to lay the foundation for making money in the new media landscape. Everyone doing business on the web, should ensure that they know their users. What is their interests and modes? The lack of information about the users is probably on a collision course with future advertising requirements. To be innovative, innovation must be a strategic and systematic part of your business. Working with culture, idea management, capacity and impact is important.

The key resources in not any longer solely based on the high quality articles and story’s written by journalists but also how it’s packaged, distributed, related to other information and the value added information that the users will contribute with. That is the paradigm shift that newspapers must see!

Thanks to Erlend Schei for great feedback to this blog post and pushing me forward to actually go through and write it.

Why Google should buy Twitter

Twitter is one of the hottest tech-companies on the web right now. The 3 year old company is reaching mainstream and is growing with a breath taking speed. Twitter.com grew 131 percent between February and March to 9.3 million visitors (US). It seems that they will pass LinkedIn in a very short time – just having Facebook and Myspace ahead of them.

Real-Time
A lot of attention is given Twitter in Media and the blogosphere. Read Write Web wrote and article back in January titled Sorry Google You Missed The Real-Time Web. They focus on the fact that Twitter is taking a place in the breaking news area, and that Google always will be late due to the nature of the indexing.

Jeffrey Mann discusses why Twitter should not work hard to find a business model One of his key points is that Twitter’s value is in its content. He like many others focus on the real-time value of the content seeing this as the real value; Peoples comments, opinions and news;

I believe that the real-time element of Twitter is an interesting thing to discuss in relation to how online news will evolve. But I think that the key value of Twitter is not founded on the real-time tweets but the power of the information that you can extract about the users.

Online advertising
Online advertising is huge and the spend will increase during the next years (even if we are in the middle of a financial crisis). Targeted advertising through Adwords and Adsense has been a great success for Google, but it is clearly that they need to innovate. My opinion is that in the coming years there will be a lot of innovation in this field. Focus will be on behavioral targeting and social media to get more precise targeting and effect (this also goes for display ads).

Interest based advertising
Google is clearly seeing this and their focus on interest based advertising gives some glimpse into how they see this evolve. As I see this the information that is created inside Twitter is a powerful element in Google’s development of the advertising technology. Today they have the search history, they have the keywords, they analyze the context inside a document. If they also can get their hands on the users and what users are saying about products and brands among everything else they will be able to make some really huge step forward in innovating the online advertising model.

The Norwegian web changed in 48 hours – how is that possible?

On the 17th of February 2009 FINN.no – one of the largest websites in Norway put out a signal to all it’s Internet Explorer 6 (IE6) users (I’ve just taken the translation from an article in The Register, it’s about 99% correct translated) :

Heads up: You’re using an older version of Internet Explorer.
In order to get the most out of our website, you can get a free update of Internet Explorer. If you’re using a work computer, you should contact your IT administrator.

Actually The register seems to have copied this translation from a blog post by Christian Johansen

At the same time as we put out this message, we released a post on FINN labs (our public feedback and experimental area) which described in detail why we are showing this information to our IE6 users and also gave a list of alternative browsers. The code to display the message was also available.

The background
Before I continue let me say a couple of words about the background for this. FINN labs launched in September 2008 and we have published monthly browser statistics and focused on how the figures are for IE6. In January 2009 IE6 had about 17,1% share. So we discussed if it was possible to give our IE6 users a message to increase the speed of upgrades from this 9 year old browser. So on February 11th, my colleague at FINN labs Erlend Schei put out this message on Twitter:

To everyone with control of mayor Norwegian sites: What about a spring cleaning to get rid of IE6? One week of encouragement on our homepages?

Anders Brenna at Teknisk Ukeblad got this tweet and wrote an article (in Norwegian) where he focused on the problem with IE6 and challenged the websites to do something pointing to the tweet by Erlend.

6 days later we initiated the campaign and Schei put out another tweet:

We have now put out a big green “upgrade IE6” box on FINN.no frontpage. Dagbladet, VG, AP are you willing to join us in this?

This message was pointing to the biggest news-sites in Norway.

One minute later I also put out this tweet:
#IE6 – We are following up our initiative and actually give all the users with IE6 at #finn.no a message on the front page!

The initiative got massive support
Late on the 17th I got this message on Twitter:

I’ll do my best to get it on vg.no as well

VG.no is the biggest website in Norway, and the next morning (18th) vg.no had put a green box on the top of it’s front page. We used Twitter to tell about VG and the reaction was amazing and perhaps scary. One hour earlier A-pressen a big Media Company with over 50 local newspapers gave me a message that they joined and they also blogged about it (in Norwegian). This was actually the breakthrough and it happened in just a few hours, with Twitter as the communication medium. FINN, VG and all the local newspapers has an enormous reach among the population and that means every person in Norway with IE6 will see the upgrade signal. During the 18th more of the biggest websites in Norway joined the campaign and late that day FINN.no sent a press release.

One day later the initiative had spread to more sites and in the evening of 19th it spread to Sweden when Aftonbladet.se – one of the biggest news site in Europe joined in. Late on the 19th when we saw that our initiative had started to spread outside Norway we sent a new press release – this time in English for the international press. Microsoft Norway also sent out a press release where they gave their support to the campaign (MS Sweden has done the same)

During the week people blogged about this and a wiki was established for the campaign where it’s possible to follow the initiative and how it is spreading world wide. Most of the newspapers in Norway was writing about this, and we got a lot of international publicity (for example in Wired)

How is it possible to get a whole Country to unite in just a few hours?
Anders Brenna has blogged about how this could be possible (in Norwegian), and he focus on the following:

  • latent need
  • Debate across the websites (news) and blogs
  • Generous linking and crediting of those who contribute

I really agree upon this, all these elements seem to have been present. In his article he gives thorough description of it. What I would like to point out in this blog post, is how much more momentum and spread this initiative has had in Europe (Norway, Sweden, France, Germany and The Netherlands) than in USA. First of all I think that when FINN.no as one of the biggest players first did this, it was much easier for other to join. The hardest part is actually to get one big player to just start this kind of initiative. Second I think Twitter has had a huge impact in the fast and widely adoption of the campaign. Norway and Sweden is small countries and when big websites initiates this kind of action it spreads rapidly to the national twitter-sphere. It’s easy to get in contact with the right persons on Twitter they encourage it by re-tweeting the information.

Europe vs USA
If we look at the spread we can see that Sweden, France and Germany has some momentum, but in USA nothing happens. Why? I think it is at least two causes. One the twitter-sphere in the US is different from Scandinavia and Europe. That some weird country are starting some campaign is not that interesting in US. If you follow the campaign on twitter (without #) you will see that most of the tweets is from outside the US.

I think this is strange, because this campaign and the huge and massive success of it in Norway and its spread to Europe has been covered a lot (at one time Friday it was on the front page of Wired, Slashdot and Digg at the same time). Also blogs/sites have been covering it (Christopher Blizzard, Ajaxian.com among others.

I think that it’s a big difference in how the Scandinavian (and maybe European) twitter-sphere and the US is working. The twitter-sphere in Scandinavia is quite small so it interesting things get spread rapidly. And of course the people that has the most followers do not have 100k + followers. This means they actually follow the tweets that is directed to them. Maybe the Americans are only looking to San Francisco? Everything outside that area is not worth mention – at least not on Twitter. 😉 But the viral effect is powerful and I think this will continue to spread world wide. Feel free to follow me on twitter – @larre

Update, 26th of february 2009: Teknisk Ukeblad have just received an email from Steve Ballmer supporting this campaign. Her it is (inside the article): Email from Steve Ballmer If you are not willing to scan through the Norwegian article here is the actual message from Steve Ballmer:

Microsoft recommends end users that are browsing the web with Internet Explorer 6 to upgrade today to benefit from numerous improvements including security features and usability enhancements.

Interoperability is key to enabling developers to continue to create great user experiences on the web. Our commitment to the technical community continues with our significant investment in Internet Explorer 8.

We continue to believe in the importance of supporting the end users and encourage the technical community to work with us in securing a good transition for the users that today are using IE6.

Hope that helps

Google is changing the Yellow Pages Industry

In the past years, Google has increased the focus on directory listings (yellow pages). OneBox is a product that combines links to paid content and ads with the usual information in the search results. The search is developed around the needs of the user, and are about to erase the distinction between promotional advertising and catalog. This challenges the established business models, and adding pressure on the industry.

restaurant-sf5


Personalized view of relevant content
Google OneBox (pictured) provides a personalized view of relevant content of various types that match your query. The product is developed for several categories, including maps, travel and directory (Local businesses). In recent years, Google has entered into agreements with local directory publishers or players for access to quality content. This is packaged with content that is retrieved (or crawled) from many different sources on the web, in order to provide a richer search results for end users.

Local Business Ads increases opportunities for advertisers in the directory
The result of the Google directory is ideal for AdWords, the keyword-based advertising system that is the main source of income for the company. It gives advertisers the ability to link ads to place and industry (Local Business Ads). Most of the links from a OneBox result in directory leads to the map. The page is divided into two main modules; The map and a list of text-links that match the result. Int the list is a space for paid content (sponsored links). The paid content (ads) is placed both in the list and on the specific map, so that it receives exactly the same distribution and relevance as the rest of the content. In this way Google builds up a business for the directory listing that is identical to what they have in advertising in general.

The differences between the ads and other content erased
On January 13th Google launched a development of the Local Business Ads, so that it virtually is identical to the rest of the directory contents. This development is erasing the difference between traditional directory content, promotional ads and regular information. This also makes the end users look at the paid content as very relevant.

landingsside2

If you for example search for hotels all functionality in the map is now also available in hotels that have purchased placement. The difference between the paid content and the rest of the content is minimized – especially when the ad is built up with the same type of landing page as a directory entry (see Map). If you want to click on a hotel it opens up an “information bubble” that contains all the functionality that you are getting up if you click on a hotel that has not come through Adwords.

This makes it extremely attractive to advertisers, and also provides a good user experience for end users looking for a restaurant or a hotel.

Game Changer?
As far as I see it, Google changes the rules for the catalog market by entering into local agreements on access to content and to develop products and services around it. Local Business Ads match this perfect and makes it difficult to compete for established directory services players.