Comments on: Keeping Ad Tracking and Dead URLs out of Yahoo! Search http://www.ysearchblog.com/2007/02/26/keeping-ad-tracking-and-dead-urls-out-of-yahoo-search/ Fri, 20 Nov 2009 18:49:51 +0000 http://wordpress.org/?v=2.8.4 hourly 1 By: Justin http://www.ysearchblog.com/2007/02/26/keeping-ad-tracking-and-dead-urls-out-of-yahoo-search/comment-page-1/#comment-3942 Justin Sat, 30 Jun 2007 17:26:26 +0000 http://ysearchblog.com/blog/2007/02/26/keeping-ad-tracking-and-dead-urls-out-of-yahoo-search/#comment-3942 I think that Yahoo! Slurp should not crawl dead URLs. Once it finds that a link is dead remove it from the indexed page that it once used to be. This will help from people searching for something and not being able to access it. I think that Yahoo! Slurp should not crawl dead URLs. Once it finds that a link is dead remove it from the indexed page that it once used to be. This will help from people searching for something and not being able to access it.

]]> By: Conficio http://www.ysearchblog.com/2007/02/26/keeping-ad-tracking-and-dead-urls-out-of-yahoo-search/comment-page-1/#comment-3941 Conficio Sat, 02 Jun 2007 19:18:29 +0000 http://ysearchblog.com/blog/2007/02/26/keeping-ad-tracking-and-dead-urls-out-of-yahoo-search/#comment-3941 Sorry, I misspoke. Off course I mean the HTTP spec not the HTML spec. Sorry, I misspoke. Off course I mean the HTTP spec not the HTML spec.

]]> By: conficio http://www.ysearchblog.com/2007/02/26/keeping-ad-tracking-and-dead-urls-out-of-yahoo-search/comment-page-1/#comment-3940 conficio Sat, 02 Jun 2007 18:04:41 +0000 http://ysearchblog.com/blog/2007/02/26/keeping-ad-tracking-and-dead-urls-out-of-yahoo-search/#comment-3940 Are you serious, really serious? The HTML spec clearly says return code 410 is "gone", and you are treating 404 like gone? <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html" rel="nofollow">http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html</a> the explanation of 404 says it is temporary or unknown it also indicates "The 410 (Gone) status code SHOULD be used if the server knows, through some internally configurable mechanism, that an old resource is permanently unavailable and has no forwarding address." Yahoo slurp is free to treat a 404 page as if removed, but I can't understand why you teach people to send a 404 if a 410 return code is appropriate. Are you serious, really serious?

The HTML spec clearly says return code 410 is “gone”, and you are treating 404 like gone?

http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html

the explanation of 404 says it is temporary or unknown it also indicates “The 410 (Gone) status code SHOULD be used if the server knows, through some internally configurable mechanism, that an old resource is permanently unavailable and has no forwarding address.”

Yahoo slurp is free to treat a 404 page as if removed, but I can’t understand why you teach people to send a 404 if a 410 return code is appropriate.

]]> By: Justin Stanley http://www.ysearchblog.com/2007/02/26/keeping-ad-tracking-and-dead-urls-out-of-yahoo-search/comment-page-1/#comment-3939 Justin Stanley Fri, 18 May 2007 04:08:53 +0000 http://ysearchblog.com/blog/2007/02/26/keeping-ad-tracking-and-dead-urls-out-of-yahoo-search/#comment-3939 Thank god for this service. Its very annoying when you come across dead urls. Most likely it was something that you needed and of course its not there. Nice step in the right direction. Thank god for this service. Its very annoying when you come across dead urls. Most likely it was something that you needed and of course its not there. Nice step in the right direction.

]]> By: Decorative Throw Pillows & Blankets http://www.ysearchblog.com/2007/02/26/keeping-ad-tracking-and-dead-urls-out-of-yahoo-search/comment-page-1/#comment-3938 Decorative Throw Pillows & Blankets Wed, 16 May 2007 18:49:56 +0000 http://ysearchblog.com/blog/2007/02/26/keeping-ad-tracking-and-dead-urls-out-of-yahoo-search/#comment-3938 Slurp should automatically not crawl dead urls after it crawls them several times and found dead. Slurp should automatically not crawl dead urls after it crawls them several times and found dead.

]]> By: Cubic Zirconia CZ Jewelry http://www.ysearchblog.com/2007/02/26/keeping-ad-tracking-and-dead-urls-out-of-yahoo-search/comment-page-1/#comment-3937 Cubic Zirconia CZ Jewelry Wed, 16 May 2007 18:48:14 +0000 http://ysearchblog.com/blog/2007/02/26/keeping-ad-tracking-and-dead-urls-out-of-yahoo-search/#comment-3937 It should not be a limit for how many dead urls we can remove..... It should not be a limit for how many dead urls we can remove…..

]]> By: Richly Scented Candles http://www.ysearchblog.com/2007/02/26/keeping-ad-tracking-and-dead-urls-out-of-yahoo-search/comment-page-1/#comment-3936 Richly Scented Candles Wed, 16 May 2007 18:46:42 +0000 http://ysearchblog.com/blog/2007/02/26/keeping-ad-tracking-and-dead-urls-out-of-yahoo-search/#comment-3936 Thanks for the advise on dead urls. Thanks for the advise on dead urls.

]]> By: Exclusive Home Furnishings Catalog http://www.ysearchblog.com/2007/02/26/keeping-ad-tracking-and-dead-urls-out-of-yahoo-search/comment-page-1/#comment-3935 Exclusive Home Furnishings Catalog Wed, 16 May 2007 18:44:53 +0000 http://ysearchblog.com/blog/2007/02/26/keeping-ad-tracking-and-dead-urls-out-of-yahoo-search/#comment-3935 Something is still wrong with the indexing system. Some of my sites are now nowhere to be found on the search results. Something is still wrong with the indexing system. Some of my sites are now nowhere to be found on the search results.

]]> By: Brian Harris http://www.ysearchblog.com/2007/02/26/keeping-ad-tracking-and-dead-urls-out-of-yahoo-search/comment-page-1/#comment-3934 Brian Harris Mon, 12 Mar 2007 16:50:07 +0000 http://ysearchblog.com/blog/2007/02/26/keeping-ad-tracking-and-dead-urls-out-of-yahoo-search/#comment-3934 please fix 301 redirects! you should be treating them as a 404 for the old page and include the new page. please fix 301 redirects! you should be treating them as a 404 for the old page and include the new page.

]]> By: Gareth http://www.ysearchblog.com/2007/02/26/keeping-ad-tracking-and-dead-urls-out-of-yahoo-search/comment-page-1/#comment-3933 Gareth Thu, 01 Mar 2007 08:30:46 +0000 http://ysearchblog.com/blog/2007/02/26/keeping-ad-tracking-and-dead-urls-out-of-yahoo-search/#comment-3933 Back in Oct we had server issues, basically we were giving out a lot of incorrect server headers for about a week due to server load issues. In Nov we saw what we expected a lot of pages being incorrectly indexed and our SERP's dropped. we thought it would take time to get these incorrect URI’s to disappear but as of this moment me are still showing thousands of pages that are either 404, redirects PPC URI's or pages blocked by a robots.txt file. From my Stats I’m seeing 20K visits a day from Slurps I’ve not checked every ip address that the Bots coming from but I’m assuming they are correct ip's. The bot visits these pages get given the correct server response whether that’s a 404 or 301 or 200 then visits the robots.txt file and gets given the disallowed folders. But yet we are 3 and a bit months in since we had issues with the server and Yahoo is still indexing rubbish and my serp’s haven't returned. Thanks for your advice on this subject but precautions have been in place for almost 4 months and Slurps isn't listening. Back in Oct we had server issues, basically we were giving out a lot of incorrect server headers for about a week due to server load issues. In Nov we saw what we expected a lot of pages being incorrectly indexed and our SERP’s dropped. we thought it would take time to get these incorrect URI’s to disappear but as of this moment me are still showing thousands of pages that are either 404, redirects PPC URI’s or pages blocked by a robots.txt file. From my Stats I’m seeing 20K visits a day from Slurps I’ve not checked every ip address that the Bots coming from but I’m assuming they are correct ip’s. The bot visits these pages get given the correct server response whether that’s a 404 or 301 or 200 then visits the robots.txt file and gets given the disallowed folders. But yet we are 3 and a bit months in since we had issues with the server and Yahoo is still indexing rubbish and my serp’s haven’t returned. Thanks for your advice on this subject but precautions have been in place for almost 4 months and Slurps isn’t listening.

]]>