A couple of days ago I wrote about how to get your blog indexed by Live Search & Yahoo! Search. While the process itself was quite straightforward the result is really disappointing. At the moment Live Search still hasn’t updated its index although it has downloaded my sitemap already a couple of days ago.

With Yahoo! the situation is a little bit more satisfying. Yahoo Slurp (their crawler) has accessed the site and some pages appear on the index already. The real problem is that I’m really bad ranked for some reason. When you search for tobman (my current Google Nr. 1 position) you find my blog on the 7th result page… That’s really bad. Even worse: if you search for other really high ranked posts (e.g. cakephp mac or cakephp breadcrumbs) the result is not the relevant post but just the general link to tobman.com. Of course this is not appealing to anybody searching for those terms. As a reason I currently get 0 traffic from Live Search and Yahoo! Search.

And then on the other hand there is Google. What a pleasure! My blog currently has 26 posts, 25 of them are indexed on Google! It was really interesting to see that after my Index fiasko Google gained “trust” from day to day and added more of my posts to its index. It now takes roughly one hour for a new post to appear on the Google search engine. I think this is really amazing and just shows from a webmaster’s point of view why Google is just so superior to Live Search & Yahoo! Search.

I will try to find out if there is anything that I can do about improving results with Live Search & Yahoo! Search and will let you know of course! If you have experience yourself please feel free to comment this post!

Search Engine Optimization Google, LiveSearch, Yahoo, Yahoo Slurp

So far I have been only working on getting my site being listed at Google. I had a look at my Google Analytics stats today and saw that 100% of the traffic generated through search engines comes from Google. So I asked myself - maybe there are other search engines out there that should link to my site as well? I personally never use a different search engine than Google so the first difficulty  was to find out, what other search engines I could have a look at.

Of course I knew that there was some search service offered by Microsoft and of course I knew that there was Yahoo out there. But here’s the funny thing: the result pages of Yahoo’s search site look so similiar to Google that in fact I thought they were using a rebranded Google service. But that’s not the case… So first I head a look at whether my site is listed in Yahoo or not. Shock! It doesn’t seem to be… or at least when I tried popular search terms that give me high ranks with Google my site never showed up at the front. Time for submitting my site to Yahoo!.

I clicked on their link where it says Submit Website and was overwhelmed by all the different options. Next surprise: you can pay for your site being listed at Yahoo! Of course I don’t want to do this, so I guess Submit Your Site for Free is the best alternative. I chose to submit a site feed (my SML sitemap) and had to sign up for a Yahoo ID… At least the sign up form was very web 2.0 with a lot of Ajax an fancy stuff.

Ok, once signed up and everything you get to the Yahoo Site Explorer, which is basically the same as Google’s Webmaster Tools just for the Yahoo search engine. To be honest I think the design is a little bit more professional than Google’s, but there is not as much functionality. Here is what I did so far:

• I verified my site (using a Meta-Tag)
• I added my sitemap as feed
• I checked the statistics: currently 0 pages from my blog are indexed by Yahoo
• I deleted some old blog entries using the Delete URL feature

Deleting sites is called “Site actions” and you are restricted to 25 actions per website. I don’t know whether just for the moment, or forever, but currently I only have 21 delete requests left. So much for Yahoo… I then moved on to try the search engine of Microsoft. First I had to find out that this is called Live Search nowadays. There is a link for webmasters on their site as well and it gets you to the Webmaster Center. Again, this is very similiar to Google and Yahoo, but much less functionality and the design is really outdated as well. Here is what I did at Live Search:

• I verified my site using the XML file method - they check the verification every time, so if you don’t want to have the annoying Meta Tag in your HTML code all the time I recommend using the XML file
• I added my sitemap

That’s it. Really nothing more that you can do. One interesting thing is the so called “Page Score”. I assume it’s something similiar to Google’s pagerank, but I ranked really high: 4 out of 5. I don’t know whether this is something to be happy about, or something that makes me wonder, how on earth MS calculates this score…

Search Engine Optimization Google, Live Search, Microsoft, Yahoo

The main reason why I blog is money. And to be honest I’m currently doing a really bad job in that sense (zero income from blogging so far). I asked myself today whether I actually have the right approach in order to succeed. I’m currently blogging mostly about cakePHP and search engine optimization. I get the most traffic out of my cakePHP posts because some of them are ranked really high on Google. For example, if you searched for

cakephp breadcrumbs

cakephp mac

you would find posts from my blog about those topics ranked 3rd and 6th on Google respectively. Those are really good ranks and I’m more than happy with that. If somebody searches for those terms I’m pretty confident that they eventually happen to visit my blog as well. But does this mean my blog will be successful? Does this mean my blog will make a lot of money through advertising? Unfortunately the answer to both questions is no - big time, no. So today I had some thoughts about this and found some great infos that can help you (and me that is) to start a blog that will actually become popular over time and will also be able to generate some cash flow!

How to get rich with a blog - a 5 step tutorial

Ok, this header is a little bit catchy, but essentially this is the main theme I’m going to cover with this post. I already wrote an article in the past, esentially saying that content is the key to your success. Now this time I am not only going to clarify what good content actually should be, but also give you an extensive step-by-step explenation of how to start a blog with successful content that will actually get you some cash!

1. Get content that people care about

As mentioned before, I have some articles about cakePHP online. Is this a hugely popular topic? Your (and my) guess would probably be no (maybe you do not even know what cakePHP is … it’s a framework for building websites… if you don’t know what a framework is you can probably tell why cakePHP is not going to be a major topic for most of the people out there). But there is a more advanced way to find out. Google Insights for Search - another free and great tool brought to us by Google. Google Insights for Search allows to monitor what is being searched for at Google and you can also determine how popular a specific search term is. If you want to give this a try with “cakePHP” follow this link. As you can see there has been a huge increase in search requests for cakePHP over the past three years. You can also see where most of the requests are coming from - Bangladesh and other Far Eastern Asian Countries. I would assume this is because a lot of outsourced developing of websites is going on there. Anyway, the question now is - cakePHP has been on the rise lately, but is it really popular? To answer this have a lookt at this link. This link shows you the 10 most requested search terms in 2008 in the United States. As you can tell - no cakePHP anywhere.

Now Google Insights for Search is a really great tool to identify content that is currently requested by users all over the world. While it is not possible to get the most requested search terms worldwide you can drill down to your specific target market and obtain data for it. Also keep an eye on the rising searches figures - they tell you what keywords are currently on the rise! I would suggest you look for content that is popular and has a rising trend. Play around with the different tools! Try different key words, different countries, different time ranges and so forth. There is only one thing to note: Google does not provide absolute search request figures but instead normalizes the data. They say that they do this in order to keep the data comparable between different countries of different size. I would guess they do this, because otherwise it would be obvious how much traffic Google gets in different countries and what the hot topics really are. But because this data is not available you have to play around and follow your guts feeling to find different areas of content that are really popular at the moment.

2. Get content advertisers care about

Say you have come up with a list of 5 different content topics that seem promising. You see an uprising search trend in your target markets and believe those are blockbuster themes. Before you go on to produce content in those fields hold on one second. While it is great to get a blog with a lot of readers it’s even better to get a blog that earns you a lot of money. So you should check first, whether the topics you have identified are actually relevant to advertising.

Let’s get back to our example “cakePHP”. Our first check is to fire up Google and search for cakePHP. If you do so have a look at the advertisings that are displayed. If nothing has changed since this was written you will see absolutely no advertising. This is not a good sign because it means that nobody actually pays for advertising related to the keyword cakePHP. But there is an even more detailed way to find out whether a topic is hot for advertisers or not. Check out this link which leads to the Google AdWords Keyword Tool. Enter cakePHP and have a look at the results. This nice little tool not only tells you what Cost per Click to expect for cakePHP ($0.45 when writing this), but also other keywords to consider. What you want to see here are keywords that have a high CPC, because this means that advertisers care about this topic! If you think 45 cents is a lot enter “insurance” as a keyword… you will see CPC estimates up to €25!

Now the only thing you have to do is find a topic for content that is both popular and highly requested by advertisers!

3. Create your content

After you have decided what topic you want to go with you will have to create a website that provides relevant content in this area. You basically have to options - create a website or blog. If you are new to programming you might want to stick to the second alternative. It is actually really easy to start writing a blog and I have wrote a short article on this here. If you decide to build a new website there are dozens or hundreds alternatives out there. You could use an Open Source CMS for example or design static webpages with Word or Dreamweaver and the likes. If you are looking for something more powerful you will have to program. I can recommend cakePHP as a framework for building websites with PHP & MySQL but there are tons of alternatives out there. I have written a short article on how to start building websites with cakePHP.

Once you have your blog or website set up you will actually have to create the content. And really I think now is the first time that you should worry about how you can create the content yourself. I assume the topic you have chosen is not “PhD programs in biotech” or something similar, where you would probably have a hard time writing content about. I assume your topic is going to be a popular one and in this case most of the people will understand it and now something about it. If you don’t know anything about it, it doesn’t matter at all. See, I had almost no clue about Search Engine Optimization just a couple of days ago. Still I’m writing articles about Search Engine Optimization. And this is easily possible because I’m writing about the new things I learn and things I have tried out along the way. You don’t have to be an expert already on your topic. You just have to show interest and learn! Read other blogs about your topic, read articles, books and so forth and present your readers the exciting information that you will find along your way. Eventually you may become an expert in the field that you are blogging about.

I think it’s even an advantage if you write about something you haven’t been too much involved in the past. As an example consider you were writing about car insurances. Car insurances are a complex topic but nothing like rocket science. If you document your way of finding an optimal car insurance, points to consider, price comparisons, terms that should be closely watched etc. I believe this would appeal to a lot of readers because most of them won’t be experts on car insurances either.

4. Get your content out

What I mean by this is, that you also have to spread the word. The great area of content you identified before and the great content that you produced for this is not of any help, if nobody has the chance to actually consume this content. This is basically the field of so called Search Engine Optimization. There are tons of guides out there on how to improve the ranking of your site with search engines and how to make your blog or website more popular. I have written a short introduction on search engine optimization for blogs here and this should give you a good start.

I would also strongly recommend using two other great Google tools: Google Webmaster Tools and Google Analytics. Both tools help you to monitor and improve how your website or blog interacts with searchengines and your users. I have written a short article on both of them here and here.

5. Maximize your revenue income

This is the last step. You have identified a promising topic. You have created a blog or website, optimized it for being found on the Internet and traffic is starting to flow in. The question now is how to turn this traffic into money. The answer -once more- is Google. They have this great service called AdSense that allows webmasters to include text advertising on their site.

Of course there are many other alternatives for online advertising including ad-networks or individual deals with advertisers. To be honest, I haven’t gotten to step 5 of starting a popular blog to earn cash myself (as mentioned in the beginning) so I cannot really give an advice on this yet. As soon as I have experience with this, I will write an article and post a link here. In the meantime feel free to post any comments and thoughts on this (and others of course)!

Summary

So to sum this up for today: Use Google Insights for Search to indentify content that is popular. Use Google AdWords to see what of this content is most relevant to advertisers. Decide what content you want to go with and create that content. Use Search Engine Optimization, Google Webmaster Tools and Google Analytics to get your content out to the people. Finally use Google AdSense and specific advertising deals to maximize your revenue income, sit back and enjoy! Did you notice that Google almost covers the entire value chain in this process? There is only one thing that’s missing: a free Blog service by Google!

Blogging adwords, cakePHP, Cost per click, Google, Google AdWords Keyword Tool, Google Insights for Search

If you have followed my blog lately, you will know that I had some problems with entries of my blog being dropped from the Google Index. This was several days back and I assumed it had to do with a lot of 404ers Googlebot was getting. I tried to fix this with re-submiting the Sitemap and requesting old content to be removed. This took some while, but eventually today it seemed to work. Well, at least partly.

The first removal request with the Google Webmaster Tools took around 2 days. The second request only 10 hours or so and my third request was processed within a couple of hours! I really don’t know why it took longer in the beginning and was super fast at the end, but now the number of pages on tobman.com listed in the Google index dropped from around 120 to 15! Great news! You can check this yourself using this search request:

site:www.tobman.com

Of course you should try to run this query with your domain as well. The goal is just to have relevant content being indexed. While it worked out to get rid of outdated content I still have the problem that not all of my sites are being indexed. It’s much better now than a couple of days back, but currently out of my 22 posts only 12 are indexed. At the Webmaster Tools it even says that 0 out of the 23 links submitted with the sitemap are indexed. It seems that it takes some time for this report to update (although I think I have never seen anything else than 0 so far).

I sat down before and tried to figure out what the reasons for not being indexed could be. I honestly could not find any regularity. I had a look at the number of onsite and offsite links and whether pictures are included in the post or not - this doesn’t seem to affect the indexing behaviour. The only thing that might the case that posts not indexed seem to be a little shorter than others. Maybe Google thinks their content is too short and therefore not worth being listed? I don’t know… I will try to relink to some of the “lost” posts, just as to my great post on why content is the key and see whether this changes anything. If not I will try to lengthen those posts as an alternative approach.

Just as a sidenote: this has been the fourth blog entry on search engine optimization in a row. I hope you find it useful… If not, don’t worry I have some new ideas for cakePHP posts and will try to write them as soon as possible!

Search Engine Optimization Google, Google Webmaster Tools, Index, Search

As you might have read I’m currently struggling with Google to get my current sites indexed and my outdated and deleted pages out of the index. Today some progress was made. First of all the first content removal request was processed! Hooray! I had to wait two days (Google indicated that it takes between 3-5 business days usually … so I’m really asking myself, what are they doing? Are they processing thos requests manually?) and the result: well the pages are still in the index. BUT it just might have been my fault, because I chose to remove the page /tag/ instead of the directory /tag/. The latter one should include all subdirectories, so I posted another removal request today.

But I have also done something else - I updated my robots.txt. It now looks like this:

User-Agent: *
Allow: /
Disallow: /blog/category/
Disallow: /blog/tag/
Disallow: /*?
Disallow: /blog/2008/

Basically the first line says that whoever is reading this (some bot usually) you better take note of what follows. In principle every directory on the host is allowed to be indexed, BUT don’t even try to index categories, tags and the 2008 archive! I also have excluded any page with a question mark - those pages are usually the ones from a search request. I’m doing all this to get rid of the outdated content that is still in the Google index and also to try to minimize the duplicate content, as this is supposedly not good for page ranks.

I have also modified the looks of the blog today a little bit. Tweaked the headers (now on the mainpage the blog name is <h1> and the titles of the entries are <h2> and on every single entry view the blog title is <h3> and the title of the entry is <h1>) and filled some titles with some keywords. We’ll see how all this works out in the coming days I suppose!

Blogging, Search Engine Optimization Google, Robots Exclusion Standard

I posted yesterday how some of my pages have disappeared from Googles index. I also mentioned how I tried to fix this issue (re-submitting sitemap, request broken links to be removed). Now, one day later the situation is as such:

This data from the Google Webmaster Tool actually means that currently not a single of my blog entries is listed in Google. And this in return explains one thing: why currently my site traffic is almost down to ZERO! I think this shows very impressively how dependet website and blog owners are of Google. If you’re not listed in the search engine nobody will find you! Of course this is especially true in the beginning of starting a blog where you don’t have any subscribers, other sites that link to your blog etc.

Now as you might imagine this situation is a little bit frustrating for me. But right now there is actually nothing I can do about it! The requests for content removal (all the broken links) is still pending, I double checked my robots.txt and meta information - all this seems to be correct. So right now the only thing I can do is to wait… I will keep you informed on how things evolve!

Blogging, Search Engine Optimization Google, Google Search, Google Webmaster Tool

Well, sometimes you have to learn it the hard way. Initially three days back I was really amazed by the huge increase of traffic I was getting after I had started blogging. I wrote new blog entries and new content and thought there should only be one way for the traffic to go: increase even more. Wrong! Yesterday and today have been far worst than the two days before, which I know thanks to Google Analytics and WordPress.com Stats. So kept myself asking what is going wrong here… And I just found out the answer: I literally pissed off Googlebot with broken links and as a consequece a whole bunch of webpages from my blog was dismissed from the search index.

All this happened because I wanted to optimize my blog for search engines and changed the URL format. Also I was playing around with linking to the blog entry id’s instead of the permalinks… bad idea! If you omit the domain part (absolute path) and work with relative paths the links will work fine from your mainsite of the blog, but won’t once you try to click on one of them from within a blog entry. So I recommend just linking to permalinks and don’t - I mean really NEVER - change your URL structure once it has been set up! My post about installing cakePHP on Mac became quite popular (for my standards of course…) and I’m still losing a lot of traffic because the initial URL all those sites are linking to is gone.

So what had happened? Googlebot started to crawl my site as usual but soon stumbled across a couple of 404 errors (page not found). It seems that at some point this just got too much (17 errors) and Google stopped crawling the site and several pages that still existed were dropped from the index (I checked by querying search terms, where I usually came up pretty high and also used the site: query)! I found this out using the great Google Webmaster Tools which showed me exactly where errors occured. After fixing all the broken links I also used those tools to - hopefully - repair this damage as quick as possible: I used the URL removal tool to mark several directories as outdated and also re-submitted my Sitemap with the working URLs. As I already posted a couple of days ago, Googlebot is fast, so I hope this will be fixed soon.

But as two major learnings from this I will take away: get your links right, have ZERO 404 errors on your blog & get your permalink structure right the first time and NEVER change it! I hope this is useful for some of you (I make the mistakes so you don’t have too…) and if you have other tips on this topic please share by commeting below!

Project WebMoney, Search Engine Optimization 404, error page, Google, Googlebot, indexing