Google has removed all indexed translated pages

[pepito69]

Hi,
Google has removed from the index all translated URL:


site:www.egiptoforo.com/forums/pt

site:www.egiptoforo.com/forums/en


etc...


I found someone here in the forum that had the same problem but due to a mysql error. I don't know what's happening in my forum.
I didn't change de robots.txt ...i happened today. I had indexed for example: 47.000 url in hebrew, 20.000 in french, etc....

Please help

Thanks

[kamilkurczak]

do you have any e-mail from google or mysql errors in log file? please write all informations.

[pepito69]

No i did not recived any email from google and there are no mysql errors in my log.
It was yesterday, and i did not touch anything.

[basketmen]

as i know, the pages indexed inside a folder can removed if the pages are cannot accessed by google for long times (ie 24 hours or more)

from my experience, i have about 4,000 pages inside a folder domain.com/d/ using other mod (no related with vbet), but because that mod are broken for few days, after i check again in google site:domain.com/d/ all indexed pages are gone

try to get links from other sites to your each translated forum home page too, ie : get links from other sites to egiptoforo.com/forums/pt/ and egiptoforo.com/forums/en/




i think the problem is not in vbet, but in the forum/site/server

double check all thing, check google webmaster tools too https://www.google.com/webmasters/tools/

[pepito69]

as i know, the pages indexed inside a folder can removed if the pages are cannot accessed by google for long times (ie 24 hours or more)

This is not the case.

i had a lot of external links and a lot of pages.

In webmaster tools i have the classic problems in vbet: Low response and duplicate metatags and tittles. Nothing new....

I'm thinking in disable the mod and it was important for me. but i think that google don't like automatic content as they say in webmaster guide. Google is moving against that in my opinion. Hope to be wrong because Vbet, could be a wonderfull mod despite it well known problems and limitations.
Look, i had thousends of URL indexed in translated versions and now nothing!

[basketmen]

This is not the case.

i had a lot of external links and a lot of pages.

In webmaster tools i have the classic problems in vbet: Low response and duplicate metatags and tittles. Nothing new....

I'm thinking in disable the mod and it was important for me. but i think that google don't like automatic content as they say in webmaster guide. Google is moving against that in my opinion. Hope to be wrong because Vbet, could be a wonderfull mod despite it well known problems and limitations.
Look, i had thousends of URL indexed in translated versions and now nothing!
we can wait and see, if there is other member that using vbet too get same problem, i see very many people using vbet specially the free version, because the threads views in this support forum are very many, and from vb.org Installed information


if the automatic pages are good quality, i think not many the pages indexed are reduced

and it is better if there is Manual translate feature too, to increase the translate quality, like i requested here (http://www.vbenterprisetranslator.com/forum/feature-requests/1033-manual-editing-other-language-translation-result-cache-admin.html), so its just like wikipedia, the article are same only translated manual to another language by the member

[kamilkurczak]

please check your e-mail or log file and find some mysql errors or something like that.
Do yu have a robots.txt file? can you paste a full content of this file?

[pepito69]

You can look it here:
http://www.egiptoforo.com/robots.txt

[dsimms]

You can look it here:
http://www.egiptoforo.com/robots.txt

I think your robots is overkill. I have never seen such
a huge robots in my life. I think you should just use
a standard robots that will keep spiders from indexing
some of your forum directories, and thats it. You are
trying to keep out every spider in the world, and I do
not think that is a good thing...what are you afraid of?

Now that I think about this. I am on a vps, and I got hacker
email report, and a google bot was on the banned ip list. Is
it possible that your server has banned some bot ip's and
if that is the case and they could not come back and spider,
then would they not be de-indexed? I found it very weird
that a google bot ended up on my banned ip list...so what
could a google spider be doing to have gotten banned?

[pepito69]

The problem is not robots.txt
Thanks for your advice but the problem is only in trasnlated ones not on main forum. But i will simplify the robots...
Thanks again but this mistery stills on.

[dsimms]

The problem is not robots.txt
Thanks for your advice but the problem is only in trasnlated ones not on main forum. But i will simplify the robots...
Thanks again but this mistery stills on.

I am curious to know also :)

as I am sure those who run free/paid versions would
also be concerned/worried that they could be deindexed also.

So the question is:

Was this your fault, or googles fault.

How old is your domain?

[pepito69]

Hi,
my domain is 4 years old. And never had a problem before with google.
I have to say that i had a lot of URL indexed in translated languages. More than vbet forum and other i have seen.

[pepito69]

I uninstalled vbet mod, will delete every rule relationed to vbet, wait 4 days and reinstall it from zero.

[dsimms]

I uninstalled vbet mod, will delete every rule relationed to vbet, wait 4 days and reinstall it from zero.

I am not sure if that is going to help, so everytime you see some
happened you are just going to uninstall, and start over?

Google seems to have serious issues with others getting ahead of
anything, and everytime something appears to be good, then
google dumps all over you, google pretty much controls our destinty,
and google has not been known to play fair either...google pretty
much controls the internet, what happens on the intenet, and
what they want to see happen on the internet, but I am not an expert,
but I hear the same ole stories...I have a hot website, now all of
a sudden google does not show me love anymore...and domains
that were stars and getting traffic, turn out to get very little...

My domain is very, very new...it was down like 2 days, and I lost
about 70 backlinks, I am not sure if that was already coming, or
did I lose those backlinks because I was down for 2 days, and
with my site being so new, why am I getting fast backlinks anyway?

Google is strange, but they are not here to help, they
are here to make money, and help themselves first....

and sometimes, even the smallest site change can take you out....since most of us
do not really know how the system works, then we make a minor site change, then we start
losing tons of traffic, and our first thought is to blame google.

[pepito69]

i will tell you in few weeks ,i have too much time... thanks anyway.
There is a possibility to be my fault, i have to discover it.

[dsimms]

i will tell you in few weeks ,i have too much time... thanks anyway.
There is a possibility to be my fault, i have to discover it.

as i mentioned before my site is very new. it has peaked about
1400 links in just about two weeks then stopped. I am not sure
what is going on now, it seems to be losing like 20-50 links per
day now, so I am not sure what it is at fault, hopefully it is not
vbet, but who knows at this point. I have disabled a mod that has
been suspected of being black hat, so I hope that it is the problem,
now that I have disabled the mod it make day a few days/weeks
to find out if my links stop dropping and start increasing again, but
then again it may be nothing at all but sandbox, or google dance....
but with the case of a 4 year old site, you should not be sandboxed.

How do you know that just vbet links are being removed?

[pepito69]

How do you know that just vbet links are being removed?

Easy, just type in google search site:www.yourdomain.com/forums/it

The changes in the number of links indexed by google is due to probably, different google servers, they are not indexing the same number of URL, and there are more than 50 servers.

everytime you do:
site:www.yourdomain.com
it connects to a single google server.

[vBET]

but i think that google don't like automatic content as they say in webmaster guide. Google is moving against that in my opinion. Hope to be wrong because Vbet, could be a wonderfull mod despite it well known problems and limitations.
Look, i had thousends of URL indexed in translated versions and now nothing!

Please tell us where in webmaster tools is written that Google doesn't like translated sites?... Also here content is NOT automatic - translation is automatic - the content is authentic from your very threads written by your very users.
Google is not against translated context - the proof is lot of forums using vBET and being indexed. You can simply check it in example on our site. If Google was against it then we wouldn't be indexed. Even this very thread which is new is already indexed in other languages (I checked for Polish) what easily proofs that Google is indexing translated pages.

So this is your issue not general one.

At this moment vBET is not working on your forum - so we are not able to work on your issue nor give you any other hint that - please turn on vBET. Whatever was the reason of your issue, by turning off vBET you made it permanent. You will never have indexed translated pages if you will not have translated pages on your forum. As I wrote - we do not know now what was the reason. It could be even something temporary on Google side what could get back as it was without any changes. Maybe... Now it is not the issue anymore - now you have no vBET so at this moment it is normal thing that you do not have indexed translated pages - because now you do not have translated pages. Please first turn vBET on.

[vBET]

I'm checking the thread from beginning so sorry is fomething was already written. I checked your robots.txt and I saw there wird content like:
Disallow: /forums/pt/*mode
As I know robots.txt do not support wildcards like * - this could be the part of the problem. It shouldn’t, because it should say that links with exact URL beginning: /forums/pt/*mode shouldn’t be indexed, but maybe something changes in Google robots and now they do not enter any links starting by: /forums/pt/ (in example because * signs are not allowed and URL reading stops at this sign giving you not allowed for indexing all your translated pages). That would be some explanation of your situation since I see rules like that for all your languages.

Once again it shouldn't be the reason here, but other way you shouldn't use * signs in disallow rules, so maybe it has something common with your issue. Please remove/comment those rules.

Also check what webmaster tools is telling you about link which was blocked by robots.txt. Of course now - because you disabled vBET - we can have wrong webmaster tools analyze, because translated links do not exists anymore on your forum. So once again - first turn on vBET, after that at this moment we need to wait, because webmaster tools do not show actual data - historical (and never knows when it is updated). So first turn vBET. Then if in 2 weeks nothing changes we will investigate is more. It would be easier if vBET would be all the time turned on, now we will have additional thing - Google reaction time...

[vBET]

The problem is not robots.txt
Thanks for your advice but the problem is only in trasnlated ones not on main forum. But i will simplify the robots...
Thanks again but this mistery stills on.

robots.txt still not simplified and it IS potential issue maker. And if you read my previous post here you will see potential explanation why only translated pages was not indexed because of your robots.txt - potential - I cannot definitively say that it was the issue maker, but you also shouldn't definitively say that it is not, especially that you have there illegal statements.

[vBET]

I uninstalled vbet mod, will delete every rule relationed to vbet, wait 4 days and reinstall it from zero.

Bad idea. You physically make translated links not working and not existing on your forum. URLs that do not exists cannot be indexed.

[vBET]

Easy, just type in google search site:www.yourdomain.com/forums/it

The changes in the number of links indexed by google is due to probably, different google servers, they are not indexing the same number of URL, and there are more than 50 servers.

everytime you do:
site:www.yourdomain.com
it connects to a single google server.

Please keep in mind that:
1. Just as you said Google has lot of servers on whole the world and those have to periodically redistribute data between each others. That is why some pages can be saw already indexed on one continent but not yet on other.
2. Keep in mind that Google shows APROXIMATE number of results what means that:
a) it is not real number
b) number of results can change even when real number of indexed pages didn't
c) number of results can be still the same even if number of indexed pages changed
d) Google uses heuristics not algorithms, so results are really not deterministic - most probably you will not see this on top results, because of caches and so on, but if it could be visible when getting on to really low results
e) magic can happen ;)

Seriously - please read my several last posts in this thread – I hope it can help :)

[vBET]

I just looked for wildcards in robots.txt (since according to my previous knowledge it shouldn't be in robots.txt) and found this article which says it should work:

http://seogadget.co.uk/wildcards-in-robots-txt/
Still I found very interesting authors sentence:

Some SEO’s don’t agree with the use of wildcards, or at least they see it as slightly risky or far from best practise.

I will make some more research in this area.

EDITED.
Here I found more about wildcard in robots.txt:
http://tools.seobook.com/robots-txt/ - according to information there Google and Yahoo allows to use wildcards.

Still - please make sure that you aren’t blocking all your translated content by some of wildcard rule. Please use webmaster tools to determine (first make vBET running again). As wrote author of last mentioned article:

Consider related URLs if you use Robots.txt wildcards: I accidentally cost myself over $10,000 in profit with one robots.txt error!

[dsimms]

Bad idea. You physically make translated links not working and not existing on your forum. URLs that do not exists cannot be indexed.

I did tell him upfront to simplify his robot.txt - I am not an expert, but
I thought right off the bat to have such a crazy robots.txt did not
make since. a robot.txt needs to be simple, not overkill....

I think I also mentioned to him that it was not a good idea either to
remove the mod as that would not solve the problem, because what
happens if it starts all over again, does he then consider removing
the mod once again, since he did not solve this the first time, it
just does not make since just to un-install the mod without solving the issue.