Importance of a Site Map

Lots and lots of fervor lately about site maps, and how they are supposed to help get your site indexed etc. Well they are right, but here are a few tips about how to go about it, in a way that makes sense.

First off, your site map needs to be in two formats, one for the viewer, and one ( since they demand it that way) for the engines, Google in particular. The first format should be, of course, html or other form that's viewer friendly in any browser, as this is for your viewer to see.

Other formats, the most common being xml, is for the engines. This is the same format as is commonly seen in blogs and RSS feeds, with several major changes. The proper format for a site map for the engines, has codes all over it that say, when the page was made, how often it changes, etc. etc. Lots of variables you can put in it, more on that in a moment.

Ok now, I'm sure your asking ... ok what's the big deal here all of a sudden with site maps ?

One, the viewer:

A site map is a place for your viewer to go, that has the whole site laid out, in order, so they can find what they want. It's like handing your viewer the keys to your car and telling them, drive wherever they want. A plain text based list, of all your urls,  nothing fancy, nothing they have to do, or mess with, no java lists or fold up menus.  Just a simple ordered list that tells them, by category, just what your site contains. Have a look at mine  if you like, to see what I am talking about.

Alphabetical order is preferred, but not totally mandatory, it depends on your content .

The reason being, again is simple, if they want say, a page on Frogs, they would look under F,  first off, but any other categorical breakdown that makes sense, say animals, or amphibians, would work just as well, provided it's simple to understand, and in an order that can be scanned by eye easily.

For example, my own is broken down into the same Galleries, and sectors, as appear on the site itself, along with all the sub pages and their direct links. Things that might otherwise be hidden from view, are all out in plain sight, in an easy to read and follow format.

Speaking of plain sight, put the link to the site map right upfront, make it easy to find. If your site has navigation that travels with the viewer, put it in the same place every time, and make it simple to access. That way, if they are ever lost they can hit the link. Having it next to your search is also a good idea. It gives your viewer options on how to seek what they are after.

Now, if it happens you have a 5 to 10 page site, you might not need this, as everything you have can easily be found. However, once the site gets up into hundreds of pages, which is easier to do than you might think, then finding something can be a real challenge. Site searches are put in for the same reason, to help your viewer find what they are looking for. This is particulaly important for a business site.

Now, the fact is, if your viewer has to access your site map often, there is something wrong with your navigation, as things are too hard to find. Put a hit counter or other tracker on the site map page, and if its getting hit a lot, check your over all navigation layout as it may need to be revamped.

The engines:

Most people, wont use the site map, if your navigation is good, but for those who do need it, it becomes indispensable. And the ones who need it most ? Search engines. I mean, think about it, all your urls, all the pages you want the engine to index, all in one place, one gulp and the spider has it all. Which is why some of the big boys are pushing for you to put one up, their way.

Now, if you go to say, Googles site map maker, prepare to be confused, as you nearly have to be a programer to use what they offer, in fact you almost have to be one to understand the directions. There is, thankfully, a whole variety of alternatives, in the form of online site map generators. They have grown to be a rather large commodity, just plug in the words site map generator, in any engine, and you will have links in plenty to choose from.

Ok, why the special format ?

Well, recall from other articles, most search engine spiders are really pretty simple affairs, compared to your average browser. They are not able to read much more than text and not great gobs of that at once, so ... the xml format, gives them the bare basics they need, what urls, how often it changes, what are the most important pages, etc. all in one shot.  However, it's in a format that your typical browser can't read, at least not in a manner that makes sense. If you try and view an xml page with a normal browser, it looks like this

  <?xml version="1.0" encoding="UTF-8" ?>
- <urlset xmlns="">
- <url>
- <url>

The only link that is "live" here is the one for Google, which is fine in this case, as it's a Google site map. But as you can see, there is no information here that your average viewer could use. Can  you see the codes I was talking about ? Changefreq, meaning how often will this page change, which can range from daily, to never. When was it last modified ? What's its priority rating and more. Lots and lots of goodies, that tell the engine some direct details about the pages, that your viewer would not be interested in, but the engines would like very much to know.

( Note: Most online generators will explain these terms in greater detail as to what they do, are for , and which ones you need, or can have on the site map )

It also gives you a bit more control over which pages they come back and scan often, vs the ones, that once they have them, will never change, so scanning them again is pointless. You can see where this information would be very useful to the engines, as it saves them a lot of useless scanning of pages they already have, that have not, and perhaps will never change. So they can focus their time, and effort on those you want, and need them to update.

Note: You do not want the spiders to have to hit the pages any more than needed, as recall every time they do, that counts as a hit and a lot of hosts, have data transfer limits. The spiders visit is no different than any other viewer as far as data download, and its visits can, if its doing its job right, will roll over your entire site. Which, if the site runs into hundreds of pages, that's a lot of data transfer, for it to index things, that might be identical to what's already indexed.

Now, of course, this requires that you think about your pages, and how often do they, or will they change, but remember, the site map is not set in stone, you can change any of the variables, and resubmit the xml page, as the needs of your site changes.

This page is most often directly submitted to the engines, you pointedly tell them where it is on your host, as your viewers will never see it. In fact, most often it's not hooked into your site at all. It's just a file floating around in your site directory, but if you tell the engines where it is, and most of the big boys have a means for you to do that, they have no problem finding it.

The spiders are not the sharpest tool in the shed, and to be blunt, need things taken down to a simple level. To understand why, go to any spider emulator, which will take any html page, and show you what a spider sees, which will give you an idea of what you need to present, to feed them properly. It's why the best pages, as far as indexing, are often the most simple. They are easier for the spider to understand and catalog.

So your site map, even if you don't create one in the special format, is your best tool for getting the pages of your site indexed properly.

Updating: Understand, that once you put one up, you need to try to keep it current, not just for your viewer, but for the spiders. Once those little bots know where your site map is, they will hit that first, before scanning the rest of the site. So keeping it up to date with your current urls and links is important.

Adding one in their special format and submitting that ? Same thing, it needs to be updated as the site is updated, to keep it current. It needs to become habit, put up a new page, add it to the site map, both formats, if it happens you use them both.

If you get into the habit of updating the site map, each and every time you update the site, you will never get "behind" and have it out of date.

More soon


Return to how to design a website