MSN now crawling JavaScript
Last April, Google began crawling JavaScript to get at more data and now MSN has quietly launched a new spider to crawl JavaScript. This is great for all the sites that use JavaScript in their navigation, but could pose a problem for some sites wanting to keep certain information out of the search engines. They'll need to exclude the new robot (along with Googlebot) in their robots.txt file.



0 Comments:
Post a Comment
Comments welcome, however any off-topic posts will be deleted by moderator.
<< Home