PDA

View Full Version : Solved Google caching thousands of duplicate pages with /lang/lang/lang issue



ctrenks
25-01-11, 05:28
Ok, Look at the page
http://www.allfreechips.com/ga/gl/afc_contests/selectpayment.html

as its a duplicate of

http://www.allfreechips.com/ga/afc_contests/selectpayment.html


Long time ago when we installed the vBet we had an issue where it would allow multiple lang "folders" so now I have a very large SEO problem with thousands of duplicate pages.

the abouve should not be /ga/gl/ it should only be /ga/ and that them becomes a duplicate

I needt to re-direct all these incoorect folders to the correct single folder asap.

Please fill me in on a .htaccess mod to correct this please.

Thank you!
Chris

kamilkurczak
25-01-11, 12:19
Hello,

I have solution for you - please check this solution:
http://www.vbenterprisetranslator.com/forum/vbet4-bug-reporting/1384-googlebot-has-discovered-errors.html#post6864
:)

ctrenks
25-01-11, 17:06
I did the simple upgrade from 3.3.3 to 3.3.6 and now I get an error on any thread view!

Help !


http://www.allfreechips.com/phpBB2/promotions/6987-check-your-emails-villento-casino-accounts-40-freespins.html

ctrenks
25-01-11, 17:08
Agg and if i turn it off, the forum homepage does not appear! Somthing is way out of whack

ctrenks
25-01-11, 17:18
Im in a real bad spot here! please help me out!

ctrenks
25-01-11, 17:32
OK, I was able to re-upload the old vbrt and im back to working! I would like to get to the latest version though so I can remove these multi lang pages

ctrenks
25-01-11, 17:44
I made a 404 error trigger in htaccess, although I would really prefer to R=301 redirect these pages


RewriteRule ^/?(af|sq|ar|be|bg|ca|zh-CN|hr|cs|da|nl|en|et|tl|fi|fr|gl|de|el|iw|hi|hu|is|id|ga|it|ja|ko|lv|lt|mk|ms|mt|no|fa|pl|pt|ro|ru|sr|sk|sl|es|sw|sv|zh-TW|th|tr|uk|vi|cy|yi)/(af|sq|ar|be|bg|ca|zh-CN|hr|cs|da|nl|en|et|tl|fi|fr|gl|de|el|iw|hi|hu|is|id|ga|it|ja|ko|lv|lt|mk|ms|mt|no|fa|pl|pt|ro|ru|sr|sk|sl|es|sw|sv|zh-TW|th|tr|uk|vi|cy|yi)* http://www.allfreechips.com/Allfreechips [L,QSA]


/Allfreechips then triggers my google 404 error page.

ctrenks
26-01-11, 03:19
Alos noted my sitemap is broke, and if i run manually i get this

Warning: strtr() [function.strtr]: The second argument is not an array in /home/allfreec/public_html/phpBB2/includes/functions_vbseo_url.php on line 488


and that line is



if (false !== strpos($text, '[langtitle=')) {
$text = strtr($text, $GLOBALS['vbet_custom_char_replacement']);
global $stylevar;
if ($stylevar['charset'] != 'utf-8') {
$text = preg_replace('/&#([0-9]+);/esiU', 'convert_int_to_utf8(\'\\1\')', $text);
}


When i try to view the sitemap i get a intermal error.

kamilkurczak
26-01-11, 11:16
Hello,

Now I don't know where is the issue. Still with double lang folder in url or not?
ahhh sorry because yesterday I gave you a solution for 4 version of vB but you have 3;) my mistake, sorry once again.

here you have a solution to eliminate double redirect from url:
1. open and make a copy of this file:
forum root/includes/vbenterprisetranslator_functions_hooks.php
2. in this file find:

function vbet_global_start(&$template_hook) {
3. in this function find:

if (!$vbulletin->userinfo['userid'] && $vbulletin->options['vbenterprisetranslator_guestfilecache_use']
4. put BEFORE this code:

if ('REQUEST_URI' == $vbulletin->options['vbenterprisetranslator_linkstype'] && $_REQUEST['goto'] != 'newpost' && $_REQUEST['language'])
{
require_once(DIR. '/includes/vbenterprisetranslator_functions_url_tracking.php');
$url = str_replace('/'.$_REQUEST['language'].'/', '/', $_SERVER['REQUEST_URI']);

if(preg_match('/\/('.VBET_LANGUAGESMATCH.')\//',$url))
{

$url = preg_replace('/('.VBET_LANGUAGESMATCH.')\//','',$url);
exec_header_redirect($url);
}
}

Secondly. I don't know what issue do you have when you would like to upgrade vBET. please give me some example. Please check all configuration settings in admincp->vBET->main (url engine etc.) and paste here your .htaccess file.
If you still can't upgrade to new version - please sent to me by PM your admincp and FTP details and I will fix it.

Thirdly. Did you integrate your sitemap generator with vBET? here you have a solution step by step how to integrate this mod: http://www.vbenterprisetranslator.com/forum/vbet-integration-other-plugins/98-sitemap-generator-integration.html

ctrenks
26-01-11, 17:06
Yes, multiple issues :)

One, I want to 301 all the pages with the error, now that code you posted does a rediredt, but is it an actual hard 301 redirect?


2nd, Upgrading to the newer 3.x version everything worked except i could not view the threads, just got a blank page.

3rd my sitemap was integrated when i had the paid installation of vbet a while back, so now it generates a sitemap but you get a Internal Error when you try to access it, and also the error i reported abouve when you generate a sitemap.

ctrenks
26-01-11, 19:02
redirect 301 /phpBB2/phpdice.php http://www.allfreechips.com/afc_contests/dice.html
# Comment the following line (add '#' at the beginning)
# to disable mod_rewrite functions.
# Please note: you still need to disable the hack in
# the vBSEO control panel to stop url rewrites.
RewriteEngine On

RewriteRule ^((urllist|sitemap_).*\.(xml|txt)(\.gz)?)$ vbseo_sitemap/vbseo_getsitemap.php?sitemap=$1 [L]

RewriteRule ^/?(af|sq|ar|be|bg|ca|zh-CN|hr|cs|da|nl|en|et|tl|fi|fr|gl|de|el|iw|hi|hu|is|id|ga|it|ja|ko|lv|lt|mk|ms|mt|no|fa|pl|pt|ro|ru|sr|sk|sl|es|sw|sv|zh-TW|th|tr|uk|vi|cy|yi)/$ vbenterprisetranslator_seo.php?vbet_lang=$1&redirected=/phpBB2/ [L,QSA]
RewriteRule ^/?(af|sq|ar|be|bg|ca|zh-CN|hr|cs|da|nl|en|et|tl|fi|fr|gl|de|el|iw|hi|hu|is|id|ga|it|ja|ko|lv|lt|mk|ms|mt|no|fa|pl|pt|ro|ru|sr|sk|sl|es|sw|sv|zh-TW|th|tr|uk|vi|cy|yi)/(.*)?$ vbenterprisetranslator_seo.php?vbet_lang=$1&redirected=/phpBB2/$2 [L,QSA]

RewriteCond %{REQUEST_URI} !(admincp/|modcp/|vbseo_sitemap/|cron)
RewriteRule ^((archive/)?(.*\.php(/.*)?))$ vbenterprisetranslator_seo.php [L,QSA]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !^(admincp|modcp|clientscript|cpstyles|images)/
RewriteRule $ vbenterprisetranslator_seo.php [L,QSA]

RewriteCond %{REQUEST_URI} !(admincp/|modcp/|cron|vbseo_sitemap)
RewriteRule ^((archive/)?(.*\.php(/.*)?))$ vbseo.php [L,QSA]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !/(admincp|modcp|clientscript|cpstyles|images)/
RewriteRule ^(.+)$ vbseo.php [L,QSA]

kamilkurczak
27-01-11, 11:17
hello,

Firstly. please give an example with url with this issue. If you have double language in url - you should have a site with "invalid thread .."
Please, go back all the changes you made and give me an example, If you can.

This site is not working for me: http://www.allfreechips.com/ga/gl/afc_contests/selectpayment.html. You should make this subdirectory translated. You should make .htaccess file from here: http://www.vbenterprisetranslator.com/forum/troubleshooting/6-faq-2.html#post54

After upgrade - did you set all settings in admincp->vBET->main ?

Your .htaccess is ok.

ctrenks
27-01-11, 15:38
hello,

Firstly. please give an example with url with this issue. If you have double language in url - you should have a site with "invalid thread .."
Please, go back all the changes you made and give me an example, If you can.

This site is not working for me: http://www.allfreechips.com/ga/gl/afc_contests/selectpayment.html. You should make this subdirectory translated. You should make .htaccess file from here: http://www.vbenterprisetranslator.com/forum/troubleshooting/6-faq-2.html#post54

After upgrade - did you set all settings in admincp->vBET->main ?

Your .htaccess is ok.

Currently I I made a re-direct to 404 all the double lang issues, i want to make a 301 though so ill use your posted code and see if that works.

When i did the upgrade, i uploaded the files and imported the xml, and this is where every thread just showed a blank cannot connect to page IE error.

I did no further configuration in the vBet control panel as the settings all looked unchanged.

ctrenks
27-01-11, 15:39
hete is my root .htaccess that has the re-direct i made



<Files ~ "\.inc$">
Order allow,deny
Deny from all
Satisfy All
</Files>


Options +FollowSymlinks
RewriteEngine on
RewriteBase /

RewriteCond %{HTTP_HOST} ^allfreechips\.com
RewriteRule (.*) http://www.allfreechips.com/$1 [R=301,L]



#Those rules have to be in domain main folder
RewriteRule ^/?(af|sq|ar|be|bg|ca|zh-CN|hr|cs|da|nl|en|et|tl|fi|fr|gl|de|el|iw|hi|hu|is|id|ga|it|ja|ko|lv|lt|mk|ms|mt|no|fa|pl|pt|ro|ru|sr|sk|sl|es|sw|sv|zh-TW|th|tr|uk|vi|cy|yi)/(af|sq|ar|be|bg|ca|zh-CN|hr|cs|da|nl|en|et|tl|fi|fr|gl|de|el|iw|hi|hu|is|id|ga|it|ja|ko|lv|lt|mk|ms|mt|no|fa|pl|pt|ro|ru|sr|sk|sl|es|sw|sv|zh-TW|th|tr|uk|vi|cy|yi)* http://www.allfreechips.com/Allfreechips [L,QSA]
RewriteRule ^/?(af|sq|ar|be|bg|ca|zh-CN|hr|cs|da|nl|en|et|tl|fi|fr|gl|de|el|iw|hi|hu|is|id|ga|it|ja|ko|lv|lt|mk|ms|mt|no|fa|pl|pt|ro|ru|sr|sk|sl|es|sw|sv|zh-TW|th|tr|uk|vi|cy|yi)/$ index.php?language=$1&vbet_outside=true [L,QSA]
RewriteRule ^/?(af|sq|ar|be|bg|ca|zh-CN|hr|cs|da|nl|en|et|tl|fi|fr|gl|de|el|iw|hi|hu|is|id|ga|it|ja|ko|lv|lt|mk|ms|mt|no|fa|pl|pt|ro|ru|sr|sk|sl|es|sw|sv|zh-TW|th|tr|uk|vi|cy|yi)/(.*)?$ $2?language=$1&vbet_outside=true [L,QSA]




#redirect 301 /_review/gold-rock-casino.html http://www.allfreechips.com/casino_review/500/21-grand-casino.html
RewriteRule [.]*play_casino([0-9]*) cportal/tclick.php?%{QUERY_STRING}&id=$1 [QSA]
RewriteRule [.]*play_banner([0-9]*) cportal/bclick.php?%{QUERY_STRING}&id=$1 [QSA]
RewriteRule [.]*casino_review/([0-9]*) casino_guide/info?id=%{QUERY_STRING}&id=$1 [QSA]
RewriteRule [.]*bingo_review/([0-9]*) casino_guide/info?id=%{QUERY_STRING}&id=$1 [QSA]
RewriteRule [.]*poker_review/([0-9]*) casino_guide/info?id=%{QUERY_STRING}&id=$1 [QSA]
RewriteRule [.]*skill_review/([0-9]*) casino_guide/info?id=%{QUERY_STRING}&id=$1 [QSA]
RewriteRule [.]*-afcinfo([0-9]*) info.php?%{QUERY_STRING}&id=$1 [QSA]
RewriteRule [.]*-resources([0-9]*) links.php?%{QUERY_STRING}&id=$1 [QSA]
RewriteRule [.]*game_review/([0-9]*) casino_guide/games?id=%{QUERY_STRING}&id=$1 [QSA]
RewriteRule [.]*casino_guide/([a-z0-9-]*) index.php?%{QUERY_STRING}&pageid=$1 [QSA]
RewriteRule [.]*poker_guide/([a-z-]*) index.php?%{QUERY_STRING}&pageid=$1 [QSA]
RewriteRule [.]*afc_contests/([a-z]*) index.php?%{QUERY_STRING}&pageid=$1 [QSA]
RewriteRule [.]*skill_games_guide/([a-z-]*) index.php?%{QUERY_STRING}&pageid=$1 [QSA]
RewriteRule [.]*bingo_guide/([a-z-]*) index.php?%{QUERY_STRING}&pageid=$1 [QSA]
RewriteRule [.]*skill_guide/([a-z-]*) index.php?%{QUERY_STRING}&pageid=$1 [QSA]
RewriteRule [.]*casino-banks/([a-z0-9-]*) casino_guide/binfo?id=%{QUERY_STRING}&id=$1 [QSA]
RewriteRule [.]*casino-banking/([a-z-]*) index.php?%{QUERY_STRING}&pageid=$1 [QSA]
RewriteRule [.]*casino_test/([0-9]*) casino_guide/info2?id=%{QUERY_STRING}&id=$1 [QSA]

RewriteRule [.]404.html /404-error-page.php [QSA]
RewriteRule [.]*onlineslots/software/([a-z-]*) index.php?type=online&pageid=slotselect&software=$1 [QSA]
RewriteRule [.]*onlineslots/features/([a-z-]*) index.php?type=online&pageid=slotselect&features=$1 [QSA]
RewriteRule [.]*onlineslots/reels/([0-9]*) index.php?type=online&pageid=slotselect&reels=$1 [QSA]
RewriteRule [.]*onlineslots/lines/([0-9]*) index.php?type=online&pageid=slotselect&lines=$1 [QSA]
RewriteRule [.]*onlineslots index.php?type=online&pageid=slotselect [QSA]


RewriteRule [.]*slots/software/([a-z-]*) index.php?type=offline&pageid=slotselect&software=$1 [QSA]
RewriteRule [.]*slots/ index.php?type=offline&pageid=slotselect [QSA]





#This rule must be in each directory which have to be supported (without forum directory itself)
RewriteCond %{QUERY_STRING} !vbet_outside=true
RewriteRule ^(.*)$ $1?vbet_outside=true [L,QSA]

kamilkurczak
27-01-11, 20:17
please send to me you admincp anf FTP details on PM and I will fix it. Thanks.

kamilkurczak
28-01-11, 10:36
Hello,

Firstly. I can not manage your website on admincp as admin. In your admincp on my account I have only menu to vBET and vBSEO, but I need menu to plugins part also.

Secondly, when I try to go to admincp->vBET->main I have this message:

Sorry, you don't have permission to access the administrative controls on this page.

If you need to access this page, ask your lead administrator to enable your permissions for this page using the Administrator Permissions section of the control panel.

You forgot about your FTP host name. Please send it to me by PM.

IMPORTANT
Please update your vBSEO to the latest version and upgrade vBET once again.
After upgrade vBSEO remember to edit two vBSEO files (everything is described in readme.html file in do-not-upload folder in vBET pack)

Thanks

ctrenks
28-01-11, 19:11
OK, Everything (except vbet) is updated. I did the mods for the sitemap and it now creates 4 files with 40,000 records but i still can not access them (internal server error)

your access should be ok now as well.

ctrenks
28-01-11, 20:02
Fixed the sitemap :)

ctrenks
28-01-11, 20:09
OK, On a roll, i actually upgraded the vbet as well and it works as desired. Should I remove the cache or just let that go?

ctrenks
28-01-11, 20:16
OK, I updated the forum root/includes/vbenterprisetranslator_functions_hooks.php
file by placing the code BEFORE the searched string so i have



if ('REQUEST_URI' == $vbulletin->options['vbenterprisetranslator_linkstype'] && $_REQUEST['goto'] != 'newpost' && $_REQUEST['language'])
{
require_once(DIR. '/includes/vbenterprisetranslator_functions_url_tracking.php');
$url = str_replace('/'.$_REQUEST['language'].'/', '/', $_SERVER['REQUEST_URI']);

if(preg_match('/\/('.VBET_LANGUAGESMATCH.')\//',$url))
{

$url = preg_replace('/('.VBET_LANGUAGESMATCH.')\//','',$url);
exec_header_redirect($url);
}
}


if (!$vbulletin->userinfo['userid'] && $vbulletin->options['vbenterprisetranslator_guestfilecache_use']) {

but its not re-directing the issue pages?

ctrenks
28-01-11, 21:14
if i add the code AFTER the If is unregistered it appears to re-direct to the forum root, not the actual page minus the lang?

ctrenks
28-01-11, 22:41
Also, can you explan how I remove the Automatic translations supported by vBET as I paif for the de-branding as well.

Thanks!

kamilkurczak
29-01-11, 13:50
hello,

about this: http://www.vbenterprisetranslator.com/forum/troubleshooting/1406-google-caching-thousands-duplicate-pages-lang-lang-lang-issue-2.html#post6972
this is not good solution for your. you can put it into code but if you are using vBSEO - this shouldn't work.
I will fix it for you after weekend.

Yes, please try to clear all cache.
about BF version - PM was sent :)

kamilkurczak
31-01-11, 11:25
and I still don't have a FTP host name of your server

ctrenks
31-01-11, 14:57
Sent PM with the details

kamilkurczak
01-02-11, 14:05
ok Thanks for this details. I am still working on it. Thanks for you patience :)
It will be fixed!

ctrenks
01-02-11, 22:03
Looks like the biggest issue is outside the forums, in the VBCMPS

kamilkurczak
02-02-11, 12:00
Hello,
This thread is very long now and I wolud like to order it ;).

I notice this:
1. page and translation on forum (in phpBB2 folder) everything works good, right?

2. The issue is here:
if I would like to go by this url:

http://www.allfreechips.com/ga/gl/afc_contests/selectpayment.html
this should be redirect to:

http://www.allfreechips.com/ga/afc_contests/selectpayment.html right? This is the main issue, right?

Secondly. I have an access to the FTP but I have no files here, only .ftpquota file, so I can't do it. Please check mu access data.

ctrenks
02-02-11, 16:44
Ill check your access, yes, right now it appears VBSEO is making the forum pages 401, I would like to have vbseo disreguard those and make them 301 Redirects as well.

kamilkurczak
03-02-11, 09:20
ok I can reproduce it on my test forum, so I will fix it without our access. Thanks.

I will move this thread into bu section.

ctrenks
03-02-11, 14:27
you should have access? I re-did the ftp yesterday morning

kamilkurczak
04-02-11, 11:02
Solved!
Already done on your server - can you confirm it?
Examples:

http://www.allfreechips.com/ja/de/pl/afc_contests/dice.html

or


http://www.allfreechips.com/ga/gl/afc_contests/selectpayment.html

Can you confirm it?

If yes - solved and will be included in the next relase.

ctrenks
04-02-11, 14:49
I can confirm the Non forum urls are re-directing to the base lang, Excelent! The forums are doing a 404 error on these, not sure if that is VBSEO taking over and doing that though?

kamilkurczak
07-02-11, 11:50
so about this issue - solved. About second part of your post - can you explain your question better? I don't understand. Thanks :)

ctrenks
07-02-11, 14:26
Yes, the link in the forum for to 404 error pages instead of re-directing to the correct URL



http://www.allfreechips.com/ga/gl/phpBB2/rules-read-before-posting/2893-posting-rules.html

will redirect to http://www.allfreechips.com/phpBB2/

ctrenks
07-02-11, 14:32
Also had a spammer from russia yesterday, Had posts with russian titles and it really screwed up the forum, it actually showed the PHP code from the backend as the posts and removed all the forum controls under it. It wasnt a hach post but it did a job on the forum until I cnahed the title to a normal english one. I wish I would have kept the title for you but its a possible bug to watch out for.

here is a cached version, not as bad as it was on the english one but you can see it distorting the forum above the post as well.



http://www.allfreechips.com/phpBB2/promotions/7272-ru-%D0%9F%D1%80%D0%B8%D0%B3%D0%BB%D0%B0%D1%88%D0%B0%D0%B5%D0%BC-%D0%92%D0%B0%D1%81-%D0%B2-%D0%98%D0%BD%D1%82%D0%B5%D1%80%D0%BD%D0%B5%D1%82-%D0%9A%D0%B0%D0%B7%D0%B8%D0%BD%D0%BE-bluechips-%5B-langtitl.html

kamilkurczak
07-02-11, 14:45
hello,

All urls in main forum folder (on your forum - in folder phpBB2) are created by vBSEO (We can only integrate vBET with vBSEO - to have translated urls), so vBSEO is making a 404 redirect when urls is wrong (We can't change it).
Urls to the pages outside forum folder are made by vBET so I can modify it and redirect a double language to the correct one. I hope this is understandable now.

About the second one: probably the spamer tried to add code or url into post title. This is not a bug.

Thanks :) solved

ctrenks
07-02-11, 15:20
So adding a url to the title brakes the forum now? I mean it destroys the page the post is on, and even screws up the thread list

Youngster
08-02-11, 11:33
the thread is closed .

Created new thread with this issue ( with this spammer), and we will try to make it on our test forums .

Give us your site url , we will check it.
Paste example of the post that had crased your forum

AfrikaansAlbanianArabicBelarusianBulgarianCatalanChineseCroatianCzechDanishDutchEnglishEstonianFilipinoFinnishFrenchGalicianGermanGreekHaitian CreoleHebrewHindiHungarianIcelandicIndonesianIrishItalianJapaneseKoreanLatvianLithuanianMacedonianMalayMalteseNorwegianPersianPolishPortugueseRomanianRussianSerbianSlovakSlovenianSpanishSwahiliSwedishTaiwaneseThaiTurkishUkrainianVietnameseWelshYiddish
Translations delivered by vB Enterprise Translator 4.10.1