How Googlebot reacts to broken links (404 errors)

November 17th, 2008

Well, sometimes you have to learn it the hard way. Initially three days back I was really amazed by the huge increase of traffic I was getting after I had started blogging. I wrote new blog entries and new content and thought there should only be one way for the traffic to go: increase even more. Wrong! Yesterday and today have been far worst than the two days before, which I know thanks to Google Analytics and WordPress.com Stats. So kept myself asking what is going wrong here… And I just found out the answer: I literally pissed off Googlebot with broken links and as a consequece a whole bunch of webpages from my blog was dismissed from the search index.

All this happened because I wanted to optimize my blog for search engines and changed the URL format. Also I was playing around with linking to the blog entry id’s instead of the permalinks… bad idea! If you omit the domain part (absolute path) and work with relative paths the links will work fine from your mainsite of the blog, but won’t once you try to click on one of them from within a blog entry. So I recommend just linking to permalinks and don’t - I mean really NEVER - change your URL structure once it has been set up! My post about installing cakePHP on Mac became quite popular (for my standards of course…) and I’m still losing a lot of traffic because the initial URL all those sites are linking to is gone.

So what had happened? Googlebot started to crawl my site as usual but soon stumbled across a couple of 404 errors (page not found). It seems that at some point this just got too much (17 errors) and Google stopped crawling the site and several pages that still existed were dropped from the index (I checked by querying search terms, where I usually came up pretty high and also used the site: query)! I found this out using the great Google Webmaster Tools which showed me exactly where errors occured. After fixing all the broken links I also used those tools to - hopefully - repair this damage as quick as possible: I used the URL removal tool to mark several directories as outdated and also re-submitted my Sitemap with the working URLs. As I already posted a couple of days ago, Googlebot is fast, so I hope this will be fixed soon.

But as two major learnings from this I will take away: get your links right, have ZERO 404 errors on your blog & get your permalink structure right the first time and NEVER change it! I hope this is useful for some of you (I make the mistakes so you don’t have too…) and if you have other tips on this topic please share by commeting below!

Project WebMoney, Search Engine Optimization , , , ,

Interim Results Project WebMoney

November 14th, 2008

Ok, with all that baking, coding and blogging yesterday I almost forgot about Project WebMoney. Well only almost… Of course I had to check the traffic statistics for my four websites and here are the results:

tobman.com: PR 0, about 120 visitors/day +1200 %

django.at: PR 3, about 20 visitors/day +/-0%

videopodcast.tv: PR 0, about 5 visitors/day +25%

advertbreak.com: PR 0, about 5 visitors/day +25%

If you compare this to the figures just two days ago you will notice one major difference: tobman.com is going through the roof! Two things that are remarkable: First Google’s webcrawler Googlebot is really fast - those new blog entries have been indexed within a couple of hours! How do I know? Because from the referring information of the logs I can already see that people were looking for things such as “setting up cakePHP on Mac OS”. And that article didn’t even exist 24 hours ago… And that’s second: it actually seems that the content I have been providing through my blogging so far seems to be relevant to some people out there. Great. Really great.

What also can be seen that the PageRank hasn’t changed so far on any of the sites. This is not surprising because the public Toolbar PageRank (the one that we can see) is only updated about every 3 months or so. But we see some changes in the traffic of videopodcast.tv & advertbreak.com - although nothing impressive it looks like that some of the readers of my blog actually visited those sites to see what’s on there currently.

The whole project WebMoney hasn’t been started more than 35 hours ago and to be honest those interim results is much more than I expected! Let’s continue the work and see what happens!

Project WebMoney ,

Content is Key

November 13th, 2008

While I’d love to get my hands dirty on SEO as soon as possible there is one major reason why I won’t: it simply wouldn’t work. As long as there is no relevant content on my websites search engine optimization is useless and a waste of time.

The good news is, that I figured out exactly what type of content I want for my three websites (apart from this one) last night in bed.

django.at is going to be a showcase site for websites powered by the django Framework - users can upload their project URL and add some additional infos. The system will then automatically generate a visual snapshot of the website and create an entry in the database. Users can then rate and comment the different django websites and of course there will be different ways to browse the database. I think it would be cool to have urls such as django.at/tobman.com (ie “django framework is used at tobman.com”) but this needs to be checked for technical possibility.

videopodcast.tv is going to be an online directory for videopodcasts, where user upload their podcast URL and data. Users can then rate and comment the different video podcasts and of course there will be different ways to browse the database.

advertbreak.com is going to be an online directory for funny advertisings, where user upload the link to an embeded flash movie (Youtube works great as posted yesterday) and add some stuff. Users can then rate and comment the different advertisings and of course there will be different ways to browse the database.

If you think I’m a little bit confused at the moment let me explain… those repetitions are the real beauty of all this. Because: the three websites are going to be based on exactly the same principles and therefore basically it will be only required to program one website & copy -> paste!

I will show how this can be done as efficiently as possible over the next couple of days using the magic of cakePHP.

Project WebMoney ,

Status Quo

November 12th, 2008

Ok, so here is the situation:

tobman.com: PageRank 0, about 10 visitors/day

django.at: PageRank 3, about 20 visitors/day

videopodcast.tv: PageRank 0, about 4 visitors/day

advertbreak.com: PageRank 0, about 4 visitors/day

So, that’s obviously not much… But a good place to start anyway! The first thing to do is figure out what content each site will host. If you want to try this yourself you probably have the advantage of being able to choose your domain name freely. I’m not that lucky, so I will try to make the best of what I’ve got:

tobman.com

My oldest & bestest domainname. As there really doesn’t seem to exist much information with regards to “Tobman” I have decided to use this as the blog to document this whole project. Blogs can become very popular you know, so this per se is not a bad idea (we’ll see how it turns out).

django.at

django used to be a website for Viennese students (with party pics etc.). It never really had more than 200 members (most friends), but hey - it could have been Facebook back then… well, could have. Anyway there is this new Phyton web framework popular these days. It’s called… django, which is a nice coincidence. I saw that the main webpage of the django framework has a PageRank of 8! Excellent -> django.at will host some info about the django framework

videopodcast.tv

Great domain name! I have worked with Axel on a project on this a couple of years ago. It never really passed the business plan stage, but still I somehow never really felt like cancelling the domain name. It just sounds too good and I thought maybe one day somebody is going to buy it. That still could happen, just the domain itself would be more worth with more traffic. There is this very similiar domainname called videopodcasts.tv (note the s) and it has a respectable page rank of 5. It basically is just a directory of videopodcasts that exist out there. I think that could be done better… so videopodcast.tv is going to be the central source for VideoPodcasts on the net!

advertbreak.com

Last but not least… adbreak. I used this domain together with Mike for a prototype demonstration. This project never really got past the prototype stage, but I forgot to cancel the domain name and just got the new bill a couple of days ago. But I’m glad this happened, because this just brought this whole idea of the project WebMoney up. I see two alternatives for this website. Either it could be a place where info is shared what to do during those oh so boring advert breaks OR (the totally opposite) a website where totally funny advert clips can be shared. Or maybe a combination of those two? I haven’t decided yet…

Ok, that’s the status quo so far. I’m not really firm on SEO (Search Engine Optimization) and all this type of things yet but will try to gather as much info as possible over the next couple of months. In parallel I need to (obviously) create content for the three websites mentioned above. It is somehow going to be easier with toman.com, as I just have to write this blog!

Project WebMoney

Project WebMoney launches

November 12th, 2008

It’s time for a new web project! This domain used to be my personal website for quite some time, then some Domaingrabber bought it, fortunately never sold it and I bought it back. Then this domain became my personal blog for my foreign exchange semester (Facebook was not popular back then…). Recently this domain hosted a funny project of mine: Project 100%. The goal was to make 100% return on investment with stock trading… per month that is.

The project didn’t quite work out (well partly due to the financial crises this year) and it became quiet about this site. That’s about to change! It’s time for Project WebMoney. This is deal:

I currently own four domains where I have direct control over the content. That is (in order of appearance on the web):

Those domains basically do one thing… they cost me money! Just this week I received another bill for advertbreak.com and I thought: this is going nowhere. As I’m currently working on a new project (vooch) I had this brilliant idea today: why not try to actually make something of those websites and create some heavy traffic which then should bring 1.) traffic useable for vooch and 2.) money through Google AdWords.

Project WebMoney was born. My intention is to create content for those domains mentioned above during the next months and try to dramatically increase the traffic they get. This project will be documented via this blog.

Project WebMoney , , ,