Ad Hoc Keyword Research is Dead

Determining an optimal keyword list can be a very involved process and is always considered the lynchpin of any successful search marketing campaign. Gone are the days of focusing your search marketing efforts on a phrase list consisting of one or two keyword terms. These “short-tail” phrases might prove to be over saturated in the search engines results or are lacking in specific relevancy to your message thus leading to a poor conversion potential. Finding those “long-tail” terms will always lead to a great ROI potential, and greater user retention.

The Process –In a Nutshell

The phraseology process has in recent years has truly become an art form and each analyst will have a core set of tools and different methods of data collection when crafting their final targeted terms.

The first step of this process typically includes the collecting of related search terms en masse. Generating large groups of search terms varying in length are gathered using keyword search tools and one should utilize more than one set of tools. Relying on one keyword tool can be detrimental to the end result when assessing your keyword targets. It is always a safe bet to back up your search data comparing the results between multiple sources. As one can imagine different search data sources can lead to very different results.

Most reputable search marketing firms will have a gambit of in-house developed tools at their disposal however here is a sample of commonly used search / keyword tools:

• Google’s Keyword Tool and Insights to Search
• Keyword Discovery
• Word Tracker
• Wordstream
• Keyword Spy
• SEOmoz – Keyword Difficulty Tool

At the end of this phase you should take a close look at historical analytics to see what terms have delivered the most relevant targets in the past. It is imperative not to destroy the great traffic you have enjoyed in the past by diluting it with new terms. Build on past successes instead of just replacing it with another term.
Once your core phrase targets have been collected and loosely organized the paring down phase of the process kicks in. The removal of unachievable terms (due competition) and others that have a lower relevancy or traffic potential should be performed. The results should deliver you with somewhat of a refined core list to work from.

The final goal of course is to have a tightly refined targeted list to apply to your search engine optimization strategies and this is where the historical knowledge of the types of keywords that convert comes in to play.
Assessing the realistic expectations of the phrases you wish to employ is prudent. Can your site compete with the others targeting that term? Are you able to do what it takes to add value to your site in order to target that particular term? These are the questions that need to be addresses before committing to the final targets.

These tips are truly a very broad look at what is needed to come up with solid targets that will over time deliver you with all that huge traffic increases you are looking for. An experienced keyword analyst will have the insight to provide a solid targeted list and the search / historical data to back up the choices.

The Company Branding and Messaging Misconception

The right mindset is essential to come up with the final keyword selection that is going to deliver you with those “money” terms.  All too often traditional marketing individuals have a difficult time understanding that what has works for years on a billboard does not necessary translate into a winning term via search. This is not to say that one should not pay close attention to fully understanding a company’s brick and mortar business and the existing marketing initiatives. The more intimate you are with the business messaging, product or service will obviously have a large impact on the phraseology process.

Marketing managers and business owners will also sometimes have strong views on what terms they wish to be found under. All too often these terms are not searched by web searchers in great numbers and in some cases can deliver irrelevant results.

For example, an auto insurance company might be apprehensive with the keyword suggestion “cheap auto insurance quotes” with the thinking that the word “cheap” degrades the quality of their business and perhaps places a misconception on the quality of their service. They may push you on this and suggest that “inexpensive automobile insurance quotes” is what they want you to utilize in your strategies. Nine times out of ten once they realize that people just don’t search that way and your suggestions will come to fruition.

Mark Johnstone

Search Engine Marketing Analyst and Project Manager
Google AdWords Qualified Individual
Metamend – Search Engine Marketing Experts

Once during a rather large Intellectual property enforcement, one respondent replied with a firey e-mail stating we were persecuting him, and he was just an honest businessman who wasn’t doing anything wrong. I began to compose a reply immediately, calmly and logically presenting supporting data for our claim, and before I could finish my response and hit send, I was greeted with another e-mail from the same person. This time the man was borderline depressed and apologetic, and remarkably, only a few minutes had elapsed. I deleted my first e-mail response that I was composing, since the whole tone had changed, and before I could re-write another e-mail I was greeted with yet a third response. This was puzzling since I had yet to respond at all? The tone of this e-mail was a willingness to work with us if we could provide sufficient proof of Trademark, good faith usage, etc. I decided this time my response was getting written, hell or high water. I sent out the response with all necessary proof, and I heard nothing back for the rest of the day.
The next morning the man was back with a vengeance. He was going to figure out how to destroy the company, ruin my life and make us pay for having the audacity to call him on his actions, according to the e-mail. I replied calmly and logically, staying on the issue at hand and ignoring his fantastical word imagery and terroristic threats. I reminded him of the law that he was actually breaking, only from the intellectual property standpoint, and I attached supporting evidence that we would use to defend our claim and rights. I sent the e-mail. He replied with a statement along the lines of : “you’ll be hearing from my attorney.” and it ended.
Later that day we received an e-mail containing an authcode for domain transfer and an apologetic e-mail for causing so much difficulty.  To this day, I am unsure if that issue honestly ever resolved in that man’s mind or if he is still plotting my demise?
Contrary to the last respondent, there is no need to get overly excited if you receive a cease and desist letter. It is honestly a courtesy extended to you in the hopes of preventing expensive legal action, no matter how it may feel. Here are some tips if you are a lucky recipient of a Cease and Desist:

Tips

1. Really read the letter. Many times people in their haste either throw the letter away or pick out one element and get hung up on it. Getting hung up on one point also may lead to you to inadvertently escalating due your frustration over simply being in this situation. Without reading the letter fully, you have no idea what you have done, to whom, or how long you have to respond to these claims.

2. Respond! If you Choose to ignore the issue, that is usually taken as an admission of guilt by the offended party! In fact, what you are doing is “escalating”, whether by accident or design. Throwing the letter away can lead to you getting sued, and doesn’t rectify the situation at all. This is akin to an Ostrich putting its head in the sand.  Having said that, I recommend composing a response after seeking advice as needed, and then walking away for 30 minutes to an hour before sending. Perhaps even wait until the next day or obviously let your legal counsel respond. In most cases, emotions will hang you, not the plaintiff, so re-read and remove the emotion from your response before sending.

3. Remain open. Take a look at what they are saying, and get comfortable with the idea that you may be in the wrong, even by accident. Most Cease and Desist letters simply want you to do just that; stop the behavior they have a problem with.

4. Remedy the situation. If they have a point, and you can fix this situation, you probably should. If they are wrong, then get professional counsel ASAP and counter the accusation by using a professional with evidence of your own.

Remember, do not let your emotions get the best of you.

Show me Where these Dead Links are Coming From

Okay, here’s a snip of the new feature;

The ‘Linked From’ link triggers a nifty little lightbox type window with the sources of the faulty links. Each page listed represents each page where the invalid link was detected. They may be internal links which you’ll need to correct, or they may be links from external sources which you’ll now know exactly where they are originating from. The link source report is found under the URLs in Sitemaps errors and Not Found errors.

Don’t Ignore Your Crawl Errors

Once again, the moral here is that in this cut throat SEO game every little bit counts and if the Search Engines, in their infinite wisdom ( except ‘Live’ and it’s Anti Cloaking Bot :) ) are going to be kind enough to offer up some helpful advice, you’d be silly not to take notice. With this new feature be sure to completely audit each and every item detailed here as they all have the potential to represent a new lead, prospective client, referral or that last back link which bumps you into the next major PR increment. Keep your stick on the ice and don’t ignore your broken links!

The Root Cause

Well aware of the problems that IE6 and IE7 created for web developers, the IE8 development team took a hard look at the root causes of these problems. Unsurprisingly the core group of issues began with IE6; more specifically with the introduction of the DOCTYPE switch. The DOCTYPE switch was a method used by IE to alter the box model that is used in calculating the dimension and relative position of elements in a web document, depending on whether the page declared a DOCTYPE. If a web page declared a valid DOCTYPE it would be rendered in “standards mode” using the box model outlined in the CSS2 specification, otherwise the page would use the rather creative interpretation of the CSS2 specification employed by IE5.5. The intention of this behaviour was to avoid breaking the countless sites that were designed for IE5.5 (yes, it’s interpretation of CSS2 was that creative), while allowing developers to “opt-in” to the new standards mode.

Non-Standard Standards

The standards mode introduced in IE6 brought Internet Explorer’s implementation of CSS2 closer to the actual W3C specification, but introduced a new class of CSS peculiarities, many of which were hard to identify and would often result in drastic inconsistencies in web pages when compared to other browsers such as Firefox, Opera and Safari. This rose to a multitude of well-documented IE6 “CSS hacks” and the trend of creating a distinct IE6-only style-sheet which led to popular, somewhat tongue-in-cheek, belief that one had to design a web page twice: once for IE6 and once for everything else. IE6 had tipped the balance in favour of backwards compatibility.

Breaking Changes

When IE7 came along the use of CSS had become the norm and the adoption of web standards was continuing to spread, so IE7 was developed with more standards-centric approach, resulting in the first version of Internet Explorer that was in reaching distance of true standards compliance. However, this improved support came with a price, as many of the web pages that utilized the IE6-specific CSS workarounds were suddenly broken in this latest version of IE. Now the balance was tipped towards interoperability, at the expense of backwards-compatibility.

The Perfect Balance?

With IE8 well into development, things are looking good for this next major version of Microsoft’s flagship browser. With Internet Explorer 8 recently passing the Acid2 test, the IE8 development team proved that they mean business when it comes to improving IE’s standard’s compliance. They also learned from the problems that arose from the last two releases of Internet Explorer, and working together with The Web Standards Project, have come up with as solution that could provide that perfect balance between backwards-compatibility and interoperability with web standards.

The solution comes in the form of an IE8 specific meta declaration that allows a web developer to specify what rendering engine IE8 should use for a given page. This declaration will specify the exact version of Internet Explorer that the page is targeting, and IE8 will display the page using the rendering engine from that version.

meta http-equiv="X-UA-Compatible" content="IE=8" />

meta http-equiv="X-UA-Compatible" content="IE=7" />

The ‘Question’ related to the rash of bizarre search referrals that MSN was spreading around earlier this year.

The question was pretty much ‘where the heck is all of this search traffic coming from suddenly?’. Okay, suddenly meaning towards the end of August of this year, just like everyone else. So for an SEO Firm getting hits for the keyword ’search’, this could appear pretty flattering. Did I do something magical to suddenly send the MSNLive Bot into heat? Did someone hijack a PPC campain and we’re going to get a nasty bill from the folks at MSN Adcenter? After playing with those ideas for three to four seconds, it became brutally obvious that this was just bogus bot traffic masking itself with a search referral, thus triggering an entry into our Search Metrics. If I didn’t know any better I could possibly call that cloaking… The plot thickens. After spending some time reflecting on our Enquisite data a few items still didn’t ad up.

1. Enquisite was tagging this traffic as arriving from MSN Live. No one could recreate a human click from this engine using ’search’.
2. It was always the same keyword.
3. The same keyword hit dozens of alternate URL’s. One would think that specific traffic, though not very focused, would generally arrive at a single related page. (same phrase, same search engine)

The geo tagging was returning an ‘unresolved’ so I grabbed the raw logs and compared the referring strings. Here are the bits that matter.

• 65.55.165.51 (Ip of the ’searcher’)
• http://search.live.com/results.aspx?q=search&mrt=en-us&FORM=LIVSOP (Appears to be the referring SERP)
• http://www.metamend.com/what-is-cloaking.html (Target of ’search result’ – Irony?)

This is one of many examples. All were from the same Class A IP block. Have a look for yourself ARIN – 65.55.165.13. So if you caught the Owner there everything would quickly ad up. So then the question becomes “Why is MSN hammering my site with bogus search referrals which are completely skewing my metrics tools and tracking systems. As you can see from the screen above, we are still receiving traffic at time of writing.

Once I found the SERP string I started searching for other victims online and came across several informative posts including this rather enjoyable Post with an equally thoughtful title.

Early this month (I did mention how late I am getting to this right?) MSN posted on it’s brand new Live Search Webmaster Center Blog about this whole topic. They’ve done a pretty lousy job of responding to this epidemic of bad traffic, but hopefully it will result in some warfare in the SE marketshare. Anyhow all paths on this topic, up until the MSN post, ended at this discussion at Webmaster World. They’ve stated that these issues should be disappearing however we’re still seeing the traffic 2 weeks after their announcement so we’ll see how that goes.

Later.

This outlines the importance of frequently logging into Webmaster Tools to keep up to date on your sites search health and not assuming that this useful tool is “set and forget”. Google seems to be having periodic website verification and Sitemaps errors of late, has anyone else has noticed this?

The website concerned has around 4000+ article pages indexed, with each article page consisting of templated boilerplate sections and the main article text pulled from a HTML include file. Unfortunately on this particular day the directory containing the article include text files was accidentally deleted from the web server. Every article page subsequently served reported an ASP sever execute error as the article text include file directory was missing. The article include directory was only restored from a backup around 12 hours later.

Within several hours of the problem being reported it was obvious that the search engines had taken notice. According to the Enquisite search metrics data for the site, there was a definite and almost immediate reduction in search referrals. As it turned out, there was around 50% less search engine referrals that day compared to the preceding and following days. This demonstrated that the article pages had a temporary reduction in ranking, resulting in the loss of search traffic for that particular day. So what happened from a search engine point of view?

We can assume that if the search spiders visited that day they noticed that the article pages lacked content and provided no value. It is also possible that the search engines took into account the users’ probable immediate and quick bounce from every article back to the search results page. Either of those explanations could result in the temporary loss of ranking witnessed for the article pages. What is certain is that the search engines reacted quickly to the problem by punishing each articles’ ranking which resulted in a devastating loss of traffic.

Live Search obviously has a mountain to climb. Their index has been the brunt of criticism for far a good long time now. So much so that was witness a presenter as SES San Jose in ‘06 refer to their search index irrelevant and ‘who cares’ -ouch. Still, top positioning doesn’t mean that you’ll generate a ton of leads as there just aren’t enough MSN searchers at the moment. Even though IE ships with Live as the ‘default search privider’ it’s clear that it’s time for MSN Search to put up or… well. buy Yahoo! (I love that rumour) Hopefully this new buzz will encourage a few of us to wander back to the MSN for a while in hopes that a meaningful algorithm update is also in the works.

As long as you provide a HTML alternative of the same content for the spiders, and non-JavaScript enabled browsers, then the search engines will not have a problem with your Ajax pages. This will also allow you to reach an audience who have accessibility issues, such as the visually impaired or those using mobile devices.

As part of our initial examination of a client site our SEOs turn off JavaScript whilst browsing or use the Lynx text only browser to determine if all of the content and links can be seen by the spiders. Although I find Lynx to be quite useful for browser accessibility issues, it does not give a true representation of how a search engine spider will see a web page. By not displaying linked images that have a blank or missing alt attribute, Lynx can give the mistaken impression that some links are not spiderable. To get the most realistic impression, there are many “Search Engine Spider Simulators” that will show the text content and links that are visible to a spider after all of the HTML and JavaScript is removed from the page source code.

Although Ajax facilitates powerfully interactive web sites, the same old problems occur for the search engines as with Flash or JavaScript presented content. “Web 2.0″ may be the latest buzz word in web site design, but having already dealt with Flash and JavaScript for some time, SEOs know what is required.

Do you have a multitude of sites that you administer through Google Webmaster tools? We all know it can sometimes be difficult to keep all the sitemap.xml files up-to-date and uploaded to all those seperate web-server FTP accounts, especially if you do not have any sort of automated system to do it for you.

In the past you could only store a Sitemap on its respective host, but this restriction has now been lifted. As long as you can ‘prove’ you control the host, by the verification process in Webmaster tools, you can store the Sitemap on any other host that you control. This means that you could store all the individual Sitemaps on your own server or even have URLs for multiple websites contained within one Sitemap file.
There is one caveat however, in that this will only work for Google at the moment, although they do assure us that they are talking to the other search engines that support the Sitemap Protocol.
I would imagine that the simplest implementation would take the form of a change to the Sitemap declaration within the robots.txt file. As the site ‘owner’ would legitimately have access to the robots.txt anyway, this would neatly resolve the security issue that Google was concerned about. Check out the original Google Sitemap Cross Submissions post.

You can now control the country association of your content on a per-domain, per-sub-domain or a per-directory level. It’s as simple as adding a new website entry to your dashboard for the domain, sub-domain or directory that contains the international content, then using the “set geographic target” interface from the “tools” menu to specify the country.

Although this will work well for sites that determine their international pages using a separate domain or directory structure, it will not help dynamic sites that use a URL parameter for this purpose. Perhaps this is a future addition that we can look forward to. For more information read the Country Associations for your Content post from the Google Blog.