Is There Any Use In Using Keywords In The URL?

Is there any value in using keywords in the URLs of web pages? Would a search engine look at keywords that you might include in the addresses of your pages, and associate those keywords with the content of your pages in the search engine’s index?

If so, how would a search engine go about looking at the web addresses indicated in the URLs to your pages, and break them down into meaningful parts to identify keywords?

Breaking URLs down into parts may also play a role in how the pages of a web site might be crawled by a search engine.

A newly published Yahoo patent application gives us some ideas on how it might extract keywords from the URLs of pages, and rank them, as well as using information uncovered in the process to determine which pages to crawl first from a web site.

Techniques for Tokenizing URLs
Invented by Krishna Leela Poola and Arun Ramanujapuram
Assigned to Yahoo
US Patent Application 20090083266
Published March 26, 2009
Filed November 6, 2007

A search engine will look at many different signals to determine what a page on the Web is about, and attempt to rank pages based upon keywords that might be an indication of the subject matter or content of those pages.

Many of those keywords are extracted from the content of pages themselves, but a search engine can look at other information associated with pages, such as the addresses of the pages.

Keywords may also be extracted from the URLs of pages, by using an algorithm that can break the URL into components, understanding the structure of those URLs, and removing candidate keywords from the different parts found within the URL.

Parts of URLs

The patent application provides a definition for different parts of URLs:

Scheme - This section of a URL identifies the internet protocol used to access a resource, such as HTTP or FTP

Authority - The part of a URL that identifies the host server where the documents or resources are located, or the domain name.

Path - This is the information following the slash character after the authority, or domain name, and it identifies the specific page or resource

Query arguments - A string that may appear in a path that can be broken down into name and value pairs, such as “category=shirts”

Fragments - A fragment identifies a subsection within a page that might be pointed to in a URL, ususally started with the “#” symbol

An example of these five different components from the patent filing:

http://www.yahoo.com:80/shopping/search?kw=blaupunkt#desc

In this URL, the scheme is “http”

The authority is “www.yahoo.com:80″ which shows the domain, and also includes a port number of “80″ in this instance.

The path is technically everything after that first single slash: “shopping/search?kw=blaupunkt#desc”

A query argument shown in this example is “kw=blaupunkt”

A fragment from this URL is #desc

Tokenizing URLs for Keywords and Web Crawling

The patent application describes a way that it might break down URLs into parts, or components, to extract keywords from URLs. Those keywords could be used to categorize pages for web search, and to understand what pages are about when providing advertisements for those pages.

This breaking down of URLs into components, and even smaller parts is referred to as “tokenizing URLs.” In addition to helping a search engine find keywords in URLs, it can have an impact on the indexing of the pages of a web site:

The tokens generated by URL tokenization may also be assigned with features of the web document to improve the efficiency of a web search. Tokenizing URLs is also the first step when clustering URLs of a website. Clustering URLs allows the identification of portions of a web document that hold more relevance. Thus, when a website is crawled by a search engine, some portions of web documents may be white-listed and should be crawled, while other portions may be black-listed and should not be crawled. This leads to more efficient web crawling.

Conclusion

Yahoo provides a fair amount of detail in the patent filing on how URLs can be broken down into components, and how keywords can be extracted from those components, as well as provided different rankings. If you’re interested in how the URLs of your site might be treated under this process, it’s worth spending some time with the patent filing itself to get a grasp of the technical details. Keep in mind that the processes from this patent application may not be the ones that Yahoo may presently be using at this time,

A cautionary note - changing the URLs to your pages, especially if those URLs have been around for a while and are indexed by search engines, is an undertaking that shouldn’t be started without careful consideration, and without using a cautious approach that keeps the risk behind such a change to a minimum. Such an approach can include using proper redirects (permanent 301 redirects) to any new URLs for external links pointed to pages of the site, actually changing URLs in internal links to the new addresses upon the site itself, and other technical methods that might help a site retain its rankings in search engines. How a search engine might react to changes to the URLs of the pages of a site can vary from one search engine to another, and traffic to the pages of a site may be negatively impacted by such a change for a period of time regardless of how carefully such a change is implemented.

12 Content Strategies To Make SEO Great

Its been said time and again, to search engines, “content is king”, but when it really comes down to it, many companies say they don’t have the time, patience, or budget for content creation. It begs the question; why then should they be ranked high in the search results? Maybe, just maybe, they’re not thinking about it logically and aren’t considering all the options at their disposal.

Here are 12 different types of content (not necessarily mutually exclusive) that when implemented properly, can drive tremendous volumes of visitors, links, and even business:

1. widgets/badges - for those unfamiliar with widgets, a widget is typically an interactive tool, that can be embedded in a web page. Since they’re interactive, often they can help potential clients find solutions quicker than they otherwise might have, making them a very powerful online marketing tool. Sometimes widgets are also made available for download to other websites, making them a powerful link acquisition tool too. Below is an example of one such widget.


2. videos - a good video is difficult to beat. People like to hear and see things for themselves, as opposed to reading about them, or being told about them. Video makes this possible, and accordingly is very popular. At the same time, video can be instructional, entertaining, and/or informational. When well done, videos will drive tremendous volumes of traffic, links, and/or potential clients to a site … and offers additional opportunities to appear in the search results


3. images - if a picture can say a 1000 words, then its also possible for images to generate 1000 visitors or links. We’ve seen many client sites, who’s images drive 20%+ of their overall website traffic volumes. In many cases, images are the reason people visit websites … they need to see what they’re buying! All that to say, do not underestimate the power of good quality images!


4. research - people often buy from websites they’ve heard about before, because familiarity breeds loyalty and trust. One means of breeding familiarity and trust is to perform and publish unique research that others in your industry or potential clients will find useful. When well thought out, timely, and topical, such research can generate thousands of links, visitors, and potential clients. Good research also helps to establish you as an authority in your space.


5. awards - everyone likes to receive cudos, or acknowledgement of a job well done! Now image that you as a site owner offer the ability for others to take those cudos, and post them on their site for all to see. Its a powerful tool, often generating thousands of visitors and/or links.


6. contests - while not for every business, contests can generate alot of relevant links, and accordingly help a site rank much better. The key is to ensure that the contest somehow relates to your business.


7. news - sites that are routinely first to publish industry news, are often visited more frequently, generate more links, and are viewed more as industry authorities. Alternatively, those running news stories after the fact, typically don’t experience all these benefits, unless they add some value or unique intepretation angle. Of course, one way to ensure the news is new and fresh is to make the news (aka a publicity stunt) … so get creative!


8. blog posts - perhaps one of the most popular techniques to create content, and one of the most poorly done! Most don’t understand the subtle nuances of creating blog content. I’ve posted about it previously (12 Common Blogging Mistakes to Avoid) as have many others, but the majority still continue to write blog content as if it were for another form of offline media.


Blog content generally consists of one of the four types listed below:


a. editorial/opinion based

b. entertaining

c. resource/educational

d. news (see #7 above)

The type you choose should be based on your objectives.


9. do something “remarkable” - if a company truly does something remarkable and unique, and publishes it on their site, the content will take on a life of its own. What does this mean? Well, if you’re a sausage manufacturer, manufacture the largest sausage on the planet. If you own a burger joint, offer a $2700 burger on the menu. Even if no one ever orders it, its something to mention on your website, and something interesting that others will notice and talk about (which results in links, visitors, and often clients). Consider this term alone “most expensive martini” … there are 1200 mentions on it in Google, and presumably a good number of those mentions link to it!


10. create an ebook - much easier said than done, but creating a ebook can result in perceived authority, visitors, links, and …. business!


11. make and distribute applications - if you can find a way to make potential clients lives easier (online or offline), it can have a profound impact. Creating an online application such as a WordPress plugin, a calculator, or industry software, and hosting it for download via your site, can increase awareness of your site, your perceived authority in the industry, attract large volumes of visitors, links, and yes, even business.

12. user generated content - content added by others, is content that you didn’t have to add yourself. With most business owner’s arguing they don’t have the time or budget for content, user generated content (UGC) would appear to be a logical solution. Although, most business owners then become concerned with their inability to control the content. Precautions can be taken however in most situations, making UGC a solution worth investigating.

Google Releases Major Blog Search Blogroll Algorithm Update

The much awaited update to the Google Blog Search blogroll detector algorithm has finally been pushed through.

An updated Google Groups thread has Googler, Jeremy Hylton saying:

We have launched a ranking change that reduces the number of results that are returned because of blogroll matches. There are still problems to work out, but this change appears to be a big improvement over our earlier fix. We had originally planned to launch an experiment for link: queries, but decide more recently to release this change first. We are still working on the link: change and expect to have that ready in a few more weeks.

We did expect to see an update for how Google Blog Search responds to the link query, but as Jeremy said, that won't be released yet for the next few weeks. But the blogroll matching detector, which matches for keywords in the blogroll sections of sites, should no longer return results for those keywords.

Why does this matter? Well, lets say you are like me and you track who links or mentions you via Google Blog Search. If someone has the Search Engine Roundtable in the blogroll, and the do a daily blog post, even if that blog post doesn't mention the Search Engine Roundtable, blog search would show that new blog post as a match. Why? Because it is in the blogroll and Google thinks it is part of the content of the blog post. Google said they fixed this issue but they do want feedback at the Google Groups thread.

Is Flash Being Indexed By Search Engines?

It seems that once a year for the past five years or so, a search engine representative will stand up at a conference and announce that they are now indexing Flash. Website designers jump with glee, and SEOs lament that all the work they did educating their clients on why they need to curb their use of Flash will be for naught.

Think of it this way, if Flash were a food, it would be an herb or a spice. Let’s say it’s basil. While basil is great in pesto, it’s not something that you’d typically make an entire meal out of. You use it to add flavor to your tortellini. And yet designers across the world—as well as CEOs who want to appear cool—want huge chunks of their websites in Flash. Sigh.

Of course if the search engines truly are indexing Flash this time around, then what’s the problem?

Flash sites certainly can do some neat tricks. People can interact with them in ways they can’t with a typical plain-Jane HTML website. Interaction is certainly good, as it can make a site “sticky.” And people pass around fun websites to their friends so they can make the little balls spin, or play with the cool virtual cubes. This is great for some websites which are looking to entertain people.

If, on the other hand, your company wants people to actually understand what you do when they come to your website, how does playing with the spinning balls further this goal? I don’t personally find it endearing to have to guess what’s behind each ball or cube when I mouse over it and it makes a funny sound or explodes and perhaps shows a single word or cute saying. If I’m looking to kill time, I might visit your site. But if I’m wondering if you’re a good fit for my needs at the moment, I don’t want to do a puzzle to figure this out. Instead, I am likely to seek out your competitor that provides me with information, rather than games.

As you can tell, I’m not a fan of cool Flash sites for a typical business. But what about those that have a more basic Flash site that does indeed provide information? For them, I would ask, why Flash? If you don’t need to allow your visitors to interact with your website, then why not just use HTML with Flash accents? Because even if the search engines are indexing the information contained in Flash (more on this in a bit), there are other reasons not to use it. First and foremost, not every browser has Flash installed. In fact, currently on an iPhone, Flash shows up as a little blue cube. Second, many Flash-based sites use only one URL for the entire site. Besides the search engine implications of that, it is also a nightmare for bookmarking, as well as for most web analytics programs.

So, even if search engines are indeed indexing the information in Flash, my recommendation is still to not design your entire website with it.

With that out of the way, I just rolled up my sleeves and started looking at how Google is currently treating Flash and whether websites can do just as well in the search results even if important information is contained within Flash.


Guess what? Most Flash is still highly invisible in Google.

To test this, I went directly to some Flash sites, and pasted exact words contained within their Flash into Google (using quotes) and most of the sites did not show up in the search results. Upon review of the source code of two sites that did show up I found that one was using the phrase I looked for within their Meta description (see… meta descriptions actually DO count for something with Google, despite what some will tell you). The other site had lots of text within the code beneath the Flash so that search engines and browsers without Flash would have some meaningful info. While that’s a good alternative, the info they had was very different from the info contained in their Flash. That’s a bit of a dangerous game to play with the search engines if you ask me. If that info is good enough for search engines and iPhones, why isn’t it good enough for your most important users?

The other thing I noticed was that even on pages that used Flash sparingly, if they had words in the Flash files, a search for those exact words would not typically pull up the HTML page that the Flash was embedded on. However, when performing a search for the exact phrase plus only .swf filetypes (which is what most Flash files use as an extension), the Flash files themselves do show up in the results. This tells me that the Google reps were kind of telling the truth about Flash being indexed, but that it doesn’t do website owners much good in a real world setting.

Google searches for the brand names of Flash sites do bring up the sites in the search results (note that their brand is typically in the Title tag). So if that’s all you care about, then using all Flash shouldn’t be a problem for you. However, if you’re interested in showing up in the search results for people who may not have already heard of you, i.e., those seeking out exactly what you offer rather than your brand name, then you may want to rethink the cool factor and go for the smart one instead!

Way Of Google Analyzing The Top Keywords On The Site

Google recently filed a patent application that deals with the keywords that Google finds on your web pages.

Google's new patent filing describes a way for website owners to view the top phrases that Google assigned to their website. The patent shows that Google finds the most important keywords on a website with a phrase-based indexing system and it describes a method that could allow website owners to add additional related keywords.

How does Google find the top keywords of your website?

All major search engines index web pages based on the individual words that they find on the page. If certain words and phrases appear together on the same page, search engines assign a topic that is related to these words to the page.
For example, the words "Paris" and "Hilton" are associated with a woman instead of a city and a hotel, the words "Tiger" and "Woods" are associated with golf.

Google's patent application indicates that Google might plan to tell you what they believe are the top keywords for your website and let you suggest changes to these phrases.

How can Google find the relation between words?

Google has billions of web pages in its index. If Google finds that many web pages contain both the word "Paris" and the word "Hilton" then Google might assume that these keywords are related. The other words on these pages could give Google a hint that this special word combination is about a woman.

Words that frequently appear very close to each other could get a tighter connection. Google has a lot of algorithms that allow them to calculate the relation between different words.

What does this mean for your website?

Google does not allow you to suggest your keywords through a form yet. That means that you must use other methods to tell Google for which keywords you want to be listed on Google's result pages. That's why search engine optimization is so important.

Here are some things that you can do to show search engines that your site is relevant to a special topic:

• Use a meaningful site architecture
  

Use a logical system to organize your website content. Create content sections that deal with different parts of your main topic and make sure that everything that is related to your topic is mentioned on your web pages.

Make sure that your web pages are put in the right categories on your website and that it's easy to find the different categories.

• Create web pages that use different relevant search terms
  

If you want to get high rankings for the keyword "shoes" then it's not enough to mention the keyword "shoes" on your website.

You must also use related keywords such as "sneakers", "boots", "sandals", "footwear", etc. to show Google that your website is relevant to the general topic.

• Find out why other pages rank higher than yours
  

If you ever asked yourself why another page has been ranked higher than yours although you perfectly optimized your pages for your search terms then you should analyze the inbound links of the top ranked pages.

The number and the authority of inbound links are important. However, it's also important that the links come from semantically and topically related pages.

Don't focus on a single keyword when optimizing your pages. Modern search engine algorithms require you to create a website that has been optimized for many different but related search terms.