be "410 Gone".
I've been trying to fix it for two days and finally ended up back here.
- Code: Select all
http://site/cars?page=1
http://site/cars?page=2
http://site/cars?page=10&from=100
These do exist....
.... and I've recently banned them with robots.txt since Yahoo Slurp will not find anything new there, only lots of duplicate contents by changing the numbers...
However, Slurp seems to read robots.txt very seldom.
Now I see Slurp is indexing nonexistant pages
- Code: Select all
http://site/node/321
is an article, just one page... but Slurp has started thinking there are many pages of it:
- Code: Select all
http://site/node?page=2
http://site/node?page=312&from=100
This prepending of ?something after almost any URL on a PHP website is possible. [Link removed by richardk: /posting.php?hello_whats_happening_?]I don't know why Slurp started doing it on my site.
I've been trying to mod_rewrite it away but have not been successful. Here are a couple of attempts that do NOT work. Can anyone please help?
- Code: Select all
RewriteRule ^node/(.*)?page - [G,L]
- Code: Select all
RewriteCond %{QUERY_STRING} node/([0-9]+)\?page
RewriteRule - [G,L]
- Code: Select all
RewriteCond %{QUERY_STRING} \?node/([0-9]+)
RewriteCond %{QUERY_STRING} page=([0-9]+)
RewriteRule - [G,L]