Revealing your Competitor’s FULL External Relevance Profile – One of my best kept secrets

Revealing your Competitor’s FULL External Relevance Profile – One of my best kept secrets

October 17th, 2007 @ // 45 Comments

Revealing your Competitor’s FULL External Relevance Profile – One of my best kept secrets

Most SEOs know that winning the battle for top search terms is in the links. They regularly check the back links of top rankings sites to get a clear picture of the “external relevance profile” of those sites. From this they learn:

  1. How many incoming links the site has

  2. The source of such links

  3. The anchor text of those incoming links

If you are familiar with Google Webmaster Central you know that you’ve got almost everything you need when you pull up the external links report, including all the incoming links and link text that Googlebot is aware of. The problem is that this data is available only for your own site, not for competitive analysis of your rivals. Also, most sites competing for top terms have thousands of links, yet competitive link analysis tools are currently limited to just a thousand results.

There are a couple of ways around these problems. I’m going to share one of my most closely guarded SEO tricks… Warning, this requires a little programming!

Option 1: Get access to your competitor’s Google Webmaster Central account by setting up a phishing site, breaking into his/her home, or bribing the technical support staff to place a Google file validating you as the owner.

Okay, obviously that option was a joke. :-) Now, let me explain the real deal. The idea is to use Yahoo’s Site Explorer and Yahoo’s powerful link command on the competitor. In order to pass the one thousand links mark, we need to add a keyword to the link search. For example link:http://sitename.com keyword1, link:http://sitename.com keyword2, etc. Save all the links, concatenate them together and filter the duplicates. You can use the keywords you have already generated in your keyword research analysis for this process.

For example, let’s see how we can extract SEOmoz’s two million links from Yahoo.

1. Extract a list of relevant keywords using Google’s tool

2. Use Yahoo Site Explorer to determine the total number of inbound links (You need to be logged in to Yahoo). Choose “Show Links Except from this domain to Entire Site”

3. Use Yahoo Search link command and add keywords to extract subsets of incoming links. One thousand at a time. Here I do a search adding “seo” and another adding “social media”

Although you can get the full list of links using this trick, the list includes links that are ‘nofollow’ that clearly do not pass any link juice. There are also sitewide links (blogroll links, and links included on every page of other sites), links coming from the same domain, IP block (class C), or owner (from the registrar information), as well as links that are not ‘nofollow,’ but which Google may be preventing from passing link juice. Any of these may increase your link count, but will not necessarily provide the same value as links coming from multiple, unaffiliated sites.

To take things to the next (useful) level, we need first to carefully organize the links by domain, IP block and ownership so that we can identify what I call unique link sources. Second, we need to filter out the links that are ‘nofollow.’ Finally, we need to visit the pages where the links are and see if they pass link juice, as I explained in the post about testing paid links.

Once the process is complete, we can collect all the anchor text coming from those links to get a complete external relevance profile. Now you can see how and why your competitor is listed and use it to form a solid link-building strategy. You have a highly-targeted list of link sources to pursue.

As you probably guessed by now, this is something that is far easier to do with a tool (stay tunned ;-) .)

I was a little bit hesitant to share this, so I would really appreciate if you let me know in the comment if you found this useful.


Category : Blog

45 Comments → “Revealing your Competitor’s FULL External Relevance Profile – One of my best kept secrets”


  1. Jeremy

    6 years ago

    This is very useful. PLEASE…Tell me more.

    Reply

  2. juerg

    6 years ago

    Very useful and interesting stuff–but: How long did it take you – assuming you were usin n o tool to perform this task and – you guessed it- how long does it take you w i t h tool…

    Reply

  3. Hamlet Batista

    6 years ago

    juerg – I am too lazy to do this type of stuff manually. Using Yahoo's API and a sizeable pool of web proxies (to overcome Yahoo's API limit) you can do this in several hours.

    I plan to share the tool, but I need to work more on it first.

    Reply

  4. juerg

    6 years ago

    Great. I was hoping you were lazy enough to come up with some code for us to share …. many thanks in advance and looking forward

    Reply
  5. [...] Sphinns – Revealing your Competitor’s FULL External Relevance Profile – One of my best kept secrets at Ham… -  Ever want to know step by step how to run a competitive analysis?  Now you do.  The only [...]

    Reply

  6. Sam Daams

    6 years ago

    This is VERY useful stuff Hamlet. It also shows that in order to build up a serious site in a seriously competitive field takes work, lots of it. Too many people forget that. If you're going after a competitive field you should expect to be burning a lot of hours building up your site and it's reputation before you see any results! An automatic tool to do a lot of this would be valuable to a lot of serious SEO – you could probably charge for it or maybe license via SEOMOZ (not sure if they do that stuff though)?

    By the way, I just noticed today the 'top posters' area. Is there any reason my name links back to a post on this site or is that a bug?

    Reply

  7. Jez

    6 years ago

    Good post, as I read it I thought it was leading somewhere ;-)

    Do you plan to integrate this with ranksense? (I do not remember this being a feature)

    How scalable do you think a published solution would be, given the requirement for proxies?

    Reply

  8. Web Design Newcastle

    6 years ago

    Nice tip there. I never knew you could add keywords to the Link: search.

    Reply

  9. Hamlet Batista

    6 years ago

    Sam, Jez

    This functionality will be part of the advanced edition of RankSense that is in the works.

    I'm working on a free web tool off of the prototype so that you guys have something useful to play with. I plan to use it as a viral marketing piece to promote RankSense ;-)

    BTW: Consider applying to RankSense's private beta if you haven't done so yet

    Reply

  10. 5ubliminal

    6 years ago

    If I feel generous, which I usually don't ;), I might release a software quite soon to do what you talk about here including some detailed reports.

    Maybe …

    Reply

  11. Hamlet Batista

    6 years ago

    Hey 5ubliminal! Do you want to make some extra bucks? Send me an email.

    I am always looking for help and you seem to be a very talented programmer.

    Reply

  12. Hamlet Batista

    6 years ago

    By the way, I just noticed today the ‘top posters’ area. Is there any reason my name links back to a post on this site or is that a bug?

    Sam – I forgot to answer this.

    It seems it is a bug on the plugin. I'll check if there is a new version and see if that fixes it.

    Reply

  13. juerg

    6 years ago

    BTW: Consider applying to RankSense’s private beta if you haven’t done so yet."
    Hi Hamlet
    did apply some time ago, but no reply yet..??
    Can you please look for arranging this. I am on a new and urgent project and would be more than pleased to check this thing out…
    Thank you

    Reply

  14. David Hopkins

    6 years ago

    Moving over to Safari:

    This is certainly a useful method. I have an extra tip to get a few more keywords. I recently found an excellent tool that logs millions of queries on Google.com to allow for some advanced keyword research. The tool (Seodigger.com) allows you to find out what keywords a domain is on the first two pages of Google. You could use this to get some more keywords to get more of the site's incoming links.

    What do you use to stop Yahoo temoprarily banning your IP When harvesting Yahoo data I always pick up the HTTP header code to make sure i've not been blocked. If i've been blocked I use sleep() to stop for a bit. I also sleep 10 seconds between each HTTP request. Do you have a better method

    In ref. to Sam's point that this shows how much work you need to do, if only SEO clients could be made to understand this P. I also think you would benefit much more by releasing the tool for free. Obviously you would only be able to make the script available. Something like this would take your site out P

    What sort of programmers are you looking for If I can be of any use, I know (X)HTML, CSS, DTD, JavaScript, DOM, AJAX, JSON, XML, XSL, XSL-FO, XPath, XLink, XInclude, PHP 4&5, SQL, SVG (basic), XUL & FF extensions building (basic). Also do logo, interface design, icons and have made a number of scrapping tools and search engine widgets. Can also put you in touch with some people who know Perl, Python, C, Java. Some of these are in Poland, so lower payment demand but have no search engine knowledge.

    Reply

  15. shor

    6 years ago

    Another gem of a post Hamlet.
    Waiting for your post on the automation of this technique!

    Reply

  16. Hamlet Batista

    6 years ago

    juerg – I will make sure you get it so you can play with it this weekend

    The tool allows you to Seodigger.com find out what keywords a domain is on the first two pages of Google . You could use this to get some more keywords to get more of the site’s incoming links.

    Nice find, David. I am glad you got the blank comments problem sorted out :-)

    <bockquote>
    What do you use to stop Yahoo temoprarily banning your IP When harvesting Yahoo data I always pick up the HTTP header code to make sure i’ve not been blocked. If i’ve been blocked I use sleep() to stop for a bit. I also sleep 10 seconds between each HTTP request. Do you have a better method

    I prefer to use the API instead of scrapping. The API gives you 5k queries per IP. You just need to use multiple proxies ;-)

    What sort of programmers are you looking for

    I can use you skills for client-side development. Please send me an email to discuss further. We prefer to code the server apps in Python/Django.

    RankSense is a Windows application and is coded mostly in C++, but for that development I don't use contractors only internal staff.

    shor – thanks for stopping by and congrats on making the Digg homepage! Very clever post and title ;-)

    The follow up post will make available the tool that does this. Stay tunned!</bockquote>

    Reply

  17. Hugo Peppers

    6 years ago

    I'm on the edge of my seat waiting for this tool.

    Reply

  18. Matt Davies

    6 years ago

    Genius. I'll be keeping my eyes open for a tool, Hamlet. This is the sort of competitor research I'd love to do for my clients, but doing it manually would just be too time consuming.

    Reply

  19. Hamlet Batista

    6 years ago

    Matt, Hugo thanks. I am glad there is interest in such tool. We already started working on it. It will take about a week to complete and make available.

    Reply

  20. juerg

    6 years ago

    Hamlet, can you please arrange for me to get access to ranksense. Would be delighted to check it out starting this weekend
    Thank you

    Reply
  21. [...] Revealing your Competitor’s FULL External Relevance Profile [...]

    Reply

  22. Hamlet Batista

    6 years ago

    juerg – sorry for the delay. I sent you an invitation code by email. Please allow for a few hours to get access to the latest build that is being published Today.

    Thanks in advance for your quality feedback.

    Reply
  23. [...] Check it out! While looking through the blogosphere we stumbled on an interesting post today.Here’s a quick excerpt…One of my best kept secrets at Hamlet Batista dot Com using Google suggest per site and Yahoo Site Explorer and some data crunching and sorting. [...]

    Reply

  24. juerg

    6 years ago

    Hamlet
    thank you for your support. Tried to donwload/ access, but was declined, since account already created. Cannot provide another e-mail address…Can you please check and confirm. Hope I didnt miss anything…
    Many thanks
    Juerg

    Reply

  25. juerg

    6 years ago

    Hamlet
    sorry, forgot to mention: Just for your info:Get mailer-daemon failure notice when e-mailing to nemedia for help ([email protected]). Am using latest firefox. Obviously this is not my day. Will have a latte macchiato and wait for your reply…
    Kind regards
    juerg

    Reply

  26. vingold

    6 years ago

    Exceptional post!

    I need to print it out and give it a try when I have a few free hours.

    I look forward to more!

    This was really good.

    Reply

  27. Hamlet Batista

    6 years ago

    Juerg – I deleted your existing accounts. Please try again with the invitation code I sent you. I apologize for the inconveniences. Thanks for alerting me about the email address issue. That address goes to several emails accounts (including mine). I guess one of them is incorrect.

    Vingold – thanks. I am glad you liked it.

    Reply

  28. john

    6 years ago

    I'd be interested in the tool as well. Sounds like an edge up on competition.

    Reply

  29. SEONewbie

    6 years ago

    This is really interesting. I'm looking for ways to stand out at my company and doing this may justify a much wanted raise.

    I do wish there was a way I could tell if those websites that linked to a competitor were Affiliates. Any suggestions?

    **hoping a free tool is available shortly**

    Reply

  30. Amanda

    6 years ago

    I can't wait to see what you come up with and test it out. Don't make us wait too long!

    Reply

  31. Hamlet Batista

    6 years ago

    SEONewbie – one way to tell if the backlinks are from affiliates is to check for HTTP redirects and cookies. I'll add this to the next update to the tool.

    Amanda – we are giving the finishing touches to the tool :-) Thanks for your patience.

    Reply

  32. Vishal

    6 years ago

    very very informaive post. i will be implementing your directions. perhaps even code a system so that this can be shared without the complications… so anyone can come to my site page and insert the data and see the result, instead of repeating the tedious procedure for each domain / keyword. thanks!

    Reply
  33. [...] Revealing your Competitor’s FULL External Relevance Profile – One of my best kept secrets [...]

    Reply

  34. Nikhil

    6 years ago

    Hello Hamlet,

    Can i get invitation code for ranksense as well . I have already submitted there my email.?

    Thanks
    Nikhil

    Reply

  35. Hamlet Batista

    6 years ago

    Nikhil – We're sending out the rest of the invites this week. Stay tuned. Thanks for your interest.

    Reply

  36. Seo Design Solutions

    6 years ago

    Hey Hamlet:

    I know what you mean about sharing the good stuff. I have been giving away my techniques for months now for competitive link building and the like, but this is great information for link hunting. Good stuff…By the way, I hope you win an award for the recent nomination for best SEO Research Blog. Thanks for the tips.

    Reply
  37. [...] profile for them. Hamlet Batista unleashed something that you will not find in any ebook with Revealing your Competitor’s FULL External Relevance Profile. Also, Earner’s Blog continues his series on link building with a good reminder to think [...]

    Reply
  38. [...] Revealing your Competitor’s FULL External Relevance Profile – One of my best kept secrets Hamlet Batista | 10/17/07 [...]

    Reply
  39. [...] Revealing your Competitor’s FULL External Relevance Profile – One of my best kept secrets Hamlet Batista | 10/17/07 [...]

    Reply

  40. losangelesseo

    6 years ago

    is it possible to query the site with multiple keywords at once? like link://www.site.com keyword1,keyword2…

    Reply
  41. [...] Revealing your Competitor’s FULL External Relevance Profile – One of my best kept secrets Hamlet Batista | 10/17/07 [...]

    Reply
  42. [...] Revealing your Competitor’s FULL External Relevance Profile – Hamlet Batista [...]

    Reply

  43. Manual Directory Sub

    6 years ago

    Hello Hamlet,

    Can i get invitation code for ranksense as well.

    I would certainly be interested in the RankSense. Can you join me in your email newsletter so that I will be able to know when you will launch future version of RankSense.

    Thanks
    Padhamanabha

    Reply

  44. Beth

    6 years ago

    Hi Hamlet,

    Could I also receive an invitation to check out your beta Ranksense?

    Thanks,

    Beth

    Reply

  45. Aaron Hall

    5 years ago

    This is very useful. Although I knew about Yahoo's tools, your article helps to effectively apply those tools using a practical step-by-step method.

    Reply

Leave a Reply


three − 1 =

Latest Posts

Testimonials

"This is one of the greatest SEO minds around. After just one year working for him, I am finding great success with my own companies. If you can understand 10% of what he says, you'll be successful"

MIchael Payne — Majority Partner, Profit Bridge, Inc.

Subscribe Now