PDA

View Full Version : Solved Google not following URL's with special characters



tavenger5
30-11-12, 16:23
In Google webmaster tools I have around 46k URL's that are not being followed by Google. The only thing that they all have in common are special characters in the URL. Here's an example:

http://cellphoneforums.net/fr/general-service-provider-forum/t348277-ting-code-promotionnel-rabais-de-50-$-sur-n-importe-quel-t%C3%A9l%C3%A9phone.html

notice how the special characters change - Google doesn't seem to like this

vBET
01-12-12, 14:09
Google has no issue with special characters in URL for sure. As I see the URL you are giving here is actually working. Maybe the issue was historical (for example translation provider could response too slow or something like that) - it is hard to say.

What is Google error message? Can you provide more examples of such URLs?

tavenger5
01-12-12, 17:28
In webmaster tools google is reporting URL's like the above as "not followed". This is because when Google goes to the URL it is redirected too many time. In this case it is because of the t%C3%A9l%C3%A9 characters.

tavenger5
02-12-12, 14:13
Here's some more examples:



http://cellphoneforums.net/ru/apple-******/t347236-%D0%9A%D0%B0%D0%BA-%D1%84%D0%B0%D0%B1%D1%80%D0%B8%D0%BA%D0%B0-%D1%80%D0%B0%D0%B7%D0%B1%D0%BB%D0%BE%D0%BA%D0%B8%D1%80%D0%BE%D0%B2%D0%B0%D1%82%D1%8C-at-t-******-5-%D0%BE%D1%82-apple-%D0%B4%D0%BB%D1%8F-%D0%B8%D1%81%D0%BF%D0%BE%D0%BB%D1%8C%D0%B7%D0%BE%D0%B2%D0%B0%D0%BD%D0%B8%D1%8F-%D0%BD%D0%B0-***-%D1%81%D0%B5%D1%82%D1%8F%D1%85-%D0%B4%D1%80%D1%83%D0%B3%D0%B8%D1%85-%D0%B4%D0%BB%D1%8F-$19-99-a-2.html

http://cellphoneforums.net/fr/theunlock-ca/t347946-at-t-******-4s-4-3-gs-**-d%C3%A9blocage-usine-d%C3%A9marrage-@-$-5-a.html

http://cellphoneforums.net/es/*********/t300436-caballero-*********-jinete-*-mensaje-kitt-entrante-*-*-remix-*-y-*-tema-*-4.html

END

vBET
03-12-12, 20:58
In webmaster tools google is reporting URL's like the above as "not followed". This is because when Google goes to the URL it is redirected too many time. In this case it is because of the t%C3%A9l%C3%A9 characters.

In case of too many redirects Google shows such information as I remember. Also the t%C3%A9l%C3%A9 do not make the redirects - this is STANDARD way of including special characters in URL. if you put it like this some browsers will display it nice URL field, some not (IE will display it as it is). In example under IE the only thing which changes in the URL you gage first time is $ - and we must check is it redirect or just browser changes it in URL field (we will confirm it).

In last post you gave some other examples - as I see some of those has also $ included. Only last one do not have it but it is cut by our filters so maybe it has $ also (can you confirm?). Please check does the links have $ included - this is the thing which I suspect at this moment. IT is not % notation - this is standard and Google handles it well.

tavenger5
05-12-12, 15:49
Yes, a lot of the URL's either have $ or * characters in them.

Examples:


http://cellphoneforums.net/de/general-service-provider-forum/t348277-ting-*****-code-$-50-rabatt-auf-jedem-handy.html
http://cellphoneforums.net/fr/google-android/t348250-best-android-phone-for-up-tp-150$.html
http://cellphoneforums.net/es/virgin-mobile/t349005-beware-de-****ty-servicio-a-trav-s-de-m-vil-virgen.html

Marcin Kalak
07-12-12, 15:16
I tested these cases on our forum, they do not cause redirect, but we can have a different configuration than you.
Please PM me access details to Admin CP and FTP. I will check what is going on there :)

Marcin Kalak
08-01-13, 22:10
No response - considered the issue is gone.

tavenger5
19-07-13, 18:06
I just wanted to bump this back up since the number of URL's has grown after the last update. I'm guessing this is because more URL's are being redirected since I turned off translations of new threads.

The response code that Google is returning is a 303

Priority URL Response Code Detected

1
de/alt-cellular-verizon/t350300-virgin-mobile-$25-plan-vs-all-others.html
303
7/16/13

2
es/alt-cellular-verizon/t350546-*********s-comet-entry-level-android-phone-wi-fi-quad-bandgsm-1700-2100-w-cdma-personal-hotspot-$140.html
303
7/14/13

3
de/alt-cellular-verizon/t350546-*********s-comet-entry-level-android-phone-wi-fi-quad-bandgsm-1700-2100-w-cdma-personal-hotspot-$140.html
303
7/14/13

4
fr/alt-cellular-verizon/t350300-virgin-mobile-$25-plan-vs-all-others.html
303
7/6/13

5
pt/new-member-introductions/t312961-*********~.html
303
7/14/13

6
es/sprint-pcs/t309992-owe-$-sprint-account-can-i-still-sell-phone.html
303
7/5/13

7
de/alt-cellular-attws/t351912-virgin-mobile-begin-offering-$40-unlimited-prepaid-**-broadbandservice-tomorrow.html
303
7/10/13

8
fr/alt-cellular-verizon/t350290-wi-fi-phones.html
301
7/7/13

9
de/virgin-mobile/t361401-konto-vor%C3%BCbergehend-gesperrt-my-balance-nicht-wegen-amt-%C3%9Cbertrifft-400-$.html
303
7/9/13

10
fr/alt-cellular-verizon/t351058-new-verizon-data-plan-details.html
301
7/16/13

11
es/sale-wanted/t361652-f-s-nuevo-*******-yo9190-galaxia-s4-mini-unlocked-$350.html
303
7/9/13

12
de/alt-cellular-attws/t352133-pageplus-vs-tracfone.html
301
7/16/13

13
de/**/t324455-how-get-rid-****y-old-phones.html
303
7/12/13

14
fr/alt-cellular-verizon/t350871-obscenely-large-bill.html
301
7/16/13

15
it/alt-cellular-verizon/t350744-failure-ring-incoming-calls.html
303
7/13/13

16
pt/sale-wanted/t361652-f-s-new-*******-i9190-galaxy-s4-mini-unlocked-$350.html
303
7/13/13

17
pt/*********/t295085-jay-z-*brooklyn-we-go-hard-remix*-ringtone.html
303
7/14/13

18
pt/sale-wanted/t361652-f-s-novo-*******-i9190-galaxy-s4-mini-desbloqueado-$-350-a.html
303
7/13/13

19
es/alt-cellular-attws/t351880-virgin-mobile-$25-plan-vs-all-others.html
303
7/16/13

20
fr/sale-wanted/t361652-f-s-i9190-galaxy-*******-new-s4-mini-unlocked-350-$.html
303
7/13/13

21
de/alt-cellular-verizon/t350643-verizon-android-phones-flood-onto-craigslist.html
301
7/6/13

22
es/alt-cellular-verizon/t351005-verizon-charge-$30-upgrade-your-phone.html
303
7/12/13

23
fr/new-member-introductions/t312961-*********~.html
303
7/15/13

24
de/sale-wanted/t361652-f-s-i9190-galaxy-*******-new-s4-mini-unlocked-$-350-a.html
303
7/14/13

25
es/*********/t295085-jay-z-*brooklyn-we-go-hard-remix*-ringtone.html
303
7/16/13

tavenger5
19-07-13, 18:13
In the first url listed above
de/alt-cellular-verizon/t350300-virgin-mobile-$25-plan-vs-all-others.html
turns into
(German) Virgin Mobile $25 plan vs all others (http://cellphoneforums.net/de/alt-cellular-verizon/t350300-virgin-mobile-25-%24-plan-vs-alle-anderen.html)

Note that the problem is with special characters being escaped. If you look at the header for this page you'll see:
HTTP/1.1 303 See Other

Marcin Kalak
22-07-13, 14:03
Are you testing it as a guest?
Have you cleaned up the guest cache?
Do you have disabled translation new threads?
Which link causes header 303?

tavenger5
28-07-13, 20:37
Are you testing it as a guest?
Have you cleaned up the guest cache?
Do you have disabled translation new threads?
Which link causes header 303?
Yes
Yes
No
all of the URL's listed are getting 303 errors in Google Webmaster tools

Marcin Kalak
29-07-13, 11:12
The latest version of vBET should have been translated links under the flags for guests and should not be redirected.
Make sure that you have enabled Guest cache (http://www.vbenterprisetranslator.com/forum/vbet4-troubleshooting/413-faq-3.html#post13517).
You make sure that the files are created in the guest cache in the appropriate folders.

Before each test, clear the files in the guest cache and have an enabled translation.

tavenger5
29-07-13, 20:39
Right - I'm using the latest version and my guest cache is enabled and working properly.

The problem has something to do with the way vbet handles url encoding for special characters. Note that in the example provided on the previous page the $ in the URL stays, but in the translated URL's it changes to %24
Original:
http://cellphoneforums.net/alt-cellular-verizon/t350300-virgin-mobile-$25-plan-vs-all-others.html
http://cellphoneforums.net/de/alt-cellular-verizon/t350300-virgin-mobile-$25-plan-vs-all-others.html <--- watch it change

vBET
29-07-13, 22:55
It suppose to be changed, because it was not translated, so it's redirected to translated one (note that after redirection there are different - translated words and also $ changes its place - result of translation).
Please see this happens everywhere - even when there is no special character in URL. Example:

http://cellphoneforums.net/games/t358163-angry-birds-game-your-favorite.html
Check what is under the flag and how looks URL at the end. There also is redirection.

The issue is that you have not translated links under the flags, when those should be translated. Please send access details by PM to Marcin Kalak and he will check why that happens. So we know what the issue is and now we have to check why it happens and correct it. We are not able to do this on our test environment, because there everything is OK.

Marcin Kalak
05-08-13, 09:42
No response - considered the issue is gone. If not please write here.

tavenger5
15-08-13, 01:41
Sent PM. Sorry for the delay.

Marcin Kalak
27-08-13, 21:23
The issue is that you do not have our rules in .htaccess.
Add to your .htaccess file:

RewriteEngine On

RewriteRule ^/?(..|zh-CN|zh-TW)/$ vbenterprisetranslator_seo.php?vbet_lang=$1&redirected=/ [L,QSA]
RewriteRule ^/?(..|zh-CN|zh-TW)/(.*)?$ vbenterprisetranslator_seo.php?vbet_lang=$1&redirected=/$2 [L,QSA]

RewriteCond %{REQUEST_URI} !(admincp/|modcp/|vbseo_sitemap/|cron)
RewriteRule ^((archive/)?(.*\.php(/.*)?))$ vbenterprisetranslator_seo.php [L,QSA]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !^(admincp|modcp|clientscript|cpstyles|images)/
RewriteRule $ vbenterprisetranslator_seo.php [L,QSA]

tavenger5
31-08-13, 01:33
The issue is that you do not have our rules in .htaccess.
Add to your .htaccess file:

I do, they are just included in the apache pre_vituralhost file.

Marcin Kalak
31-08-13, 10:46
Please paste the contents of that file here or in PM.

tavenger5
31-08-13, 13:21
PM sent. Thanks!

Marcin Kalak
19-09-13, 11:25
No replies to PM - considered the issue is gone. If not, please write here or reply to PM.

AfrikaansAlbanianArabicBelarusianBulgarianCatalanChineseCroatianCzechDanishDutchEnglishEstonianFilipinoFinnishFrenchGalicianGermanGreekHaitian CreoleHebrewHindiHungarianIcelandicIndonesianIrishItalianJapaneseKoreanLatvianLithuanianMacedonianMalayMalteseNorwegianPersianPolishPortugueseRomanianRussianSerbianSlovakSlovenianSpanishSwahiliSwedishTaiwaneseThaiTurkishUkrainianVietnameseWelshYiddish
Translations delivered by vBET 4.9.2