PDA

View Full Version : Costs optimization - how vBET allows to use machine translation FREE and/or CHEAPER



vBET
22-10-13, 20:29
Below you will find list of hints how to use translations completelly FREE or cheaper if some languages are not supported by FREE API's.

Disable not necessary languages (http://www.vbenterprisetranslator.com/forum/vbet4-general-discussions/3753-costs-optimization-how-vbet-allows-use-machine-translation-free-cheaper.html#post14026)
Use many machine translation providers (APIs) (http://www.vbenterprisetranslator.com/forum/vbet-announcements/3753-costs-optimization-how-vbet-allows-use-machine-translation-free-cheaper.html#post14027)
Use free language detection or not at all (http://www.vbenterprisetranslator.com/forum/vbet-announcements/3753-costs-optimization-how-vbet-allows-use-machine-translation-free-cheaper.html#post14028)
Check all configurations described here (http://www.vbenterprisetranslator.com/forum/vbet-announcements/3753-costs-optimization-how-vbet-allows-use-machine-translation-free-cheaper.html#post14029)
Use database cache (http://www.vbenterprisetranslator.com/forum/vbet-announcements/3753-costs-optimization-how-vbet-allows-use-machine-translation-free-cheaper.html#post14030)
Do not clear database cache (http://www.vbenterprisetranslator.com/forum/vbet-announcements/3753-costs-optimization-how-vbet-allows-use-machine-translation-free-cheaper.html#post14031)
Do not translate numbers (http://www.vbenterprisetranslator.com/forum/vbet-announcements/3753-costs-optimization-how-vbet-allows-use-machine-translation-free-cheaper.html#post14033)
Use time and date formats without letters (http://www.vbenterprisetranslator.com/forum/vbet-announcements/3753-costs-optimization-how-vbet-allows-use-machine-translation-free-cheaper.html#post14034)
Do not translate usernames (http://www.vbenterprisetranslator.com/forum/vbet-announcements/3753-costs-optimization-how-vbet-allows-use-machine-translation-free-cheaper.html#post14035)
Do not translate code html tag (http://www.vbenterprisetranslator.com/forum/vbet-announcements/3753-costs-optimization-how-vbet-allows-use-machine-translation-free-cheaper-2.html#post14036)
Allow more people for manual translations (http://www.vbenterprisetranslator.com/forum/vbet-announcements/3753-costs-optimization-how-vbet-allows-use-machine-translation-free-cheaper-2.html#post14037)
Always use newest vBET version (http://www.vbenterprisetranslator.com/forum/vbet-announcements/3753-costs-optimization-how-vbet-allows-use-machine-translation-free-cheaper-2.html#post14038)
Do not translate pages you do not want to be translated (http://www.vbenterprisetranslator.com/forum/vbet-announcements/3753-costs-optimization-how-vbet-allows-use-machine-translation-free-cheaper-2.html#post14039)
Maybe you do not want to translate old threads (http://www.vbenterprisetranslator.com/forum/vbet-announcements/3753-costs-optimization-how-vbet-allows-use-machine-translation-free-cheaper-2.html#post14040)
Maybe you want to translate only threads form specific time period (http://www.vbenterprisetranslator.com/forum/vbet-announcements/3753-costs-optimization-how-vbet-allows-use-machine-translation-free-cheaper-2.html#post14041)
Block member pages for banned users (http://www.vbenterprisetranslator.com/forum/vbet-announcements/3753-costs-optimization-how-vbet-allows-use-machine-translation-free-cheaper-2.html#post14042)
Consider switching vBulletin 3.x to 4.x (http://www.vbenterprisetranslator.com/forum/vbet-announcements/3753-costs-optimization-how-vbet-allows-use-machine-translation-free-cheaper-2.html#post14043)
Avoid changing texts in GUI (http://www.vbenterprisetranslator.com/forum/vbet-announcements/3753-costs-optimization-how-vbet-allows-use-machine-translation-free-cheaper-2.html#post14044)

vBET
22-10-13, 21:05
For most forums 53 languages are not needed. You can simply disable as many languages as you want in configuration: Admin CP -> vBET -> Translation Options -> Enable Translations (3 parts). Less supported languages means less translations and maybe it will be enough to use only free providers (not all supports all language pairs). Also please note that less languages means potentially less traffic, users and income from advertisements, so you need to made your choice.

Please keep in mind that also good strategy is to enable only few languages at the beginning (best those supported by free providers) and then add more languages from time to time. This will be also better option for your SEO.

NOTE: By default are enabled only those languages which can be translated completely free all to all!

vBET
22-10-13, 22:30
vBET automatically queues translation providers using free and cheaper before more expensive where it is possible. Not all providers supports all languages, but vBET recognizes which pairs are supported by which providers. That is why it is wise to use more that one translation provider. Excluding those who not supports actual language pair the default queue order at this moment is:
Apertium, Yandex, Microsoft, SDL BeGlobal, Google.

The order optimizes costs and it is configurable - for more details please see description of Admin CP -> vBET -> Translation Providers -> Use Google ONLY
Priorities in default order are:
- Cost
- Quotas
So free with no quota APIs are before free with high quota APIs, which are before free with low quota APIs, which are before cheaper paid APIs, which are before more expensive paid APIs. This way when several APIs are OK for language pair vBET is using the one which is not only cheapest but also has biggest quota, so those API with less quota are not used for languages covered by others and can be still used for language pairs not covered by others, becasue the quota was saved.

NOTE: for each used machine translation provider you must set API key in Admin CP -> vBET -> Translation Providers

NOTE FOR SUPPORTED PROVIDERS DESCRIPTIONS BELOW:
- It do not represent any sale offer
- It can be old/not accurate/wrong - but it was OK when it was made. Please let us know if you find any wrong description and we will correct it.
- It do not compares quality of translation nor speed of machine translation at all
- It is only explanation why default order is in this, not the other way

APERTIUM machine translation API

FREE
No information about quota
Supports not too many language pairs, and not all to all
Sometimes ends with timeout errors and internal errors - vBET recognizes it and do not adds to cache such results
Recommended to USE IT AS SUPPORT WITH OTHER PROVIDERS, but not necessary - cannot relay only on this one, but still it is worth to use since it is free


YANDEX machine translation API

FREE
Very high FREE quota - 1 000 000 characters/DAY
Supports over 30 languages but not all to all
Highly recommended to USE IT AS SUPPORT WITH FREE MICROSOFT - because it doesn't support all to all language pairs, it is good to use other APIs too (if you care about multilingual community and posts written in other languages that your forum language).


MICROSOFT machine translation API

FREE or paid
Low FREE quota - 2 000 000 characters/month
Supports over 30 languages - all to all translation (by default vBET has turned on only languages supported by Microsoft - configurable by Admin CP -> vBET -> Translation Options -> Enable Translations (3 parts))
Highly recommended to USE IT AS BASE WITH OTHER FREE APIs - it supports all to all translation, so it can do translations not supported by other free APIs. It has low free monthly quota so you can use it:

Free and wait for new quotas. Some translations will be partial, but vBET will recognize it and translate later when quota will be available again. Meantime other free APIs will translate what they can so low Microsoft quota will be used only when it is really needed.
Like in previous option, but with disabled some of languages enabled by default - this way quota will be used not so fast, so partial translation will be rare or not happen at all.
Paid - Microsoft allows for several paid high quotas. Please see Microsoft Azure (http://datamarket.azure.com/dataset/bing/microsofttranslator) page for details about prices for their machine translation. This is paid by quota not by usage!
Free with other paid APIs (SDL BeGlobal, Google), which are cheaper and paid by usage not by quota.



SDL BeGlobal machine translation API

Paid but 1/4 cheaper than Google and Microsoft
Supports lot of languages but not all to all
If you want to use language pairs not supported by any free API (those not enabled by default), or you are using only default, but Microsoft quota ends too early for you, then using this API can be good option for you

Cost is $15 per 1 000 000 characters (paid by usage)
It is 1/4 cheaper than Google ($20)
It is cheaper than paid option of Microsoft
If you need to pay for translations USE IT WITH FREE MICROSOFT BEFORE YOU GO TO USE GOOGLE and even when you need to use Google, then use it with Google - it will cost less



Google machine translation API

Paid but supports 53 languages all to all
You pay only for what you use not for quota like in Microsoft
More expensive than SDL BeGlobal (but supports more language pairs)
Supports most languages and if you want to use those not enabled by default then you will have to use Google machine translation API
If you need to use languages not enabled by default (not supported by any free API) USE IT WITH ALL OTHER FREE APIs AND PAID SDL BeGlobal. This way you will use what you can for free - even from Microsoft. When it is not possible to translate free, then SDL BeGlobal will be used as cheaper and Google, as more expensive will be used ONLY when no other API cannot be used (no support for language pair or no quota).

vBET
22-10-13, 22:52
vBET supports free language detection API's and you can use detection completely free. You can also disable it by setting Admin CP -> vBET -> Language Detection -> Language Detection to value Always trust user, which is default but it is not recommended, since free quota is really high.
Still it is default, because setting language detection requires not only changing this parameter but also setting API key for provider in Admin CP -> vBET -> Translation Providers.
When API key is set and detection turned on, then you also have to choose which API will be used in option Admin CP -> vBET -> Language Detection -> Detection API

NOTE FOR SUPPORTED PROVIDERS DESCRIPTIONS BELOW:
- It do not represent any sale offer
- It can be old/not accurate/wrong - but it was OK when it was made. Please let us know if you find any wrong description and we will correct it.
- It do not compares quality nor speed of language detection at all

DetectLanguage language detection API

FREE or paid if you want
Very high FREE quota - 5 000 detections/DAY
Detects all languages supported by vBET
RECOMMENDED TO USE and it is default option (still detection have to be turned on and API key set)


Microsoft language detection API

FREE or paid if you want
But it has low quota and uses same quota as for translations
Detects only languages supported by Microsoft (turned on by default)
Because it is using free translation quota, it is NOT RECOMMENDED to use


Google language detection API

PAID same as for translations
Detects all languages supported by vBET
Because it is paid, it is NOT RECOMMENDED to use

vBET
22-10-13, 22:57
Please note that some values for options described here as default could have different default values before. Some APIs where added in time, some TOS where changed in time. Default value is set only when parameter is new, so when default is changing old installations will still have old default value set if it wasn't changed. Also you could change something and do not remember it.

That is why the best option for you is to check each recommendation in this thread to be sure that you are really using best costs optimizations.

vBET
22-10-13, 23:04
Translations are always cost - even when you are using free machine translation providers, then it still costs performance (response time).
That it is wise to use database cache. You do not want to pay several times for same translation, nor wait each time for it.
When database cache is turned on vBET will cache translations locally and next time it will be available immediately. Also vBET will allow to manually update cached translations by forum users witch appropriate rights.

By default database cache is turned on and it should stay that way. Still vBET allows to turn it off for diagnostics. To be sure that your database cache is turned on please check option Admin CP -> vBET Cache -> Database Cache -> Use database cache

Also make sure that not only database cache is turned on, but also it is working for all languages (it can be disabled per language). To do this simply make sure that all 3 parts of option Admin CP -> vBET Cache -> Database Cache -> Cache details have all checkboxes marked. Please note that you can mark even languages for which you have disabled translations (flags) - it will simply be ignored. And it is better to have ignored cache option, than by mistake do not cache all translations for whole language.

vBET
22-10-13, 23:20
Some time ago it was required by TOS of one translation provider to remove old cache (to be more specific it was allowed to cache only for 15 days). Right now it is not necessary and since some translations are paid it has no sense to delete data you paid for.

Now by default clearing database cache is disabled, but it is still allowed to change this configuration in case if TOS of some free API will change, or in case when you are using only free APIs, have small forum (so quota is not a problem), do not use manual translation and want your translation to be refreshed from time to time (maybe machine translation will be better next time).

So it is available to turn on automatic cache cleaning, but it is highly NOT recommended and you are not allowed to use it when you use manual translation (otherwise you will loose your manual translations).

To be sure that your database cache cleaning is disabled please make sure that option Admin CP -> vBET Cache -> Database Cache -> Cache Time to Live (TTL) is set to 0.
It is not necessary but you can additionally make sure that Admin CP -> vBET Cache -> Database Cache -> Cache clearing strategy is set to Disabled.

vBET
23-10-13, 00:02
vBET allows to not translate numbers (configurable and enabled by default). Still it can be disabled, because it have some impact on performance (checking does each text for translation is a number) and perhaps for some languages it can be crucial to translate numbers too.
Still it is highly recommended to not translate numbers, because those can create endless translations. Many GUI parts includes changing numbers like: Views 12, Views 13, and so on. Right now vBET separates numbers (by HTML italic node), what made from the example only one translation Views and changing numbers which will not be translated when the option is on.

To be sure that you are not translating numbers please make sure that you have set option:
Admin CP -> vBET -> Translation Options -> Don't translate numbers
Also make sure that database cache is on (numbers are checked when asking cache for translation):
Admin CP -> vBET Cache -> Database Cache -> Use database cache

Please note that vBET will not translate then not only numbers but also dates and times (if in appropriate formats - please see next hint). Generally all texts which have only numbers and "#-.,: " characters (e.g.: "206", "2,340", "2013-08-04,") will not be translated. And GUI places where numbers, dates and times are included in text are separated as described earlier so still are not translated.

vBET
23-10-13, 00:12
As described in previous hint vBET can separate dates, times and numbers from rest of GUI texts and do not translate those to avoid endless translations.
Still if your date or time will include letters, then it will be translated, because it will not be recognized as number (it can have numbers and "#-.,: " characters - space is included there).

So make sure you are using appropriate date and time formats:
- For only date format, you can have what you want - it is only 12 translations per year
- For only time format, do NOT use 12 hours system with AM/PM, but 24 hours system (format: H:i)
- For date and time, use only format with numbers and mentioned characters

Please note that some plugins can add their own parameters for dates and times.
For vBulletin time format please change Admin CP -> Settings -> Options -> Date and Time Options -> Format For Time to H:i

vBET
23-10-13, 00:19
Usernames mostly have no real meaning so cannot be really translated. Also in many forums there is SPAM issue and even when spammers are banned fast, their accounts still exists (banned, but exists to block email for example).

So there is no sense to translate usernames and vBET has option which allows for that (by default enabled). It can be disabled to avoid small performance impact but it is not recommended.

To be sure that your forum do not translates usernames please check do you have set option Admin CP -> vBET -> Translation Options -> Don't translate usernames

Please note that usernames often are included in other GUI texts creating long translations, but vBET separate those on translated pages and will not translate it.

vBET
23-10-13, 00:33
It is not often used, but some forums can allow to include HTML tags in messages. In such case code HTML tag can be used (HTML no BBCode). Please note that there is no sense to translate code, that is why vBET by default do not translate content of BBCodes: code, php, html. But this do not include code HTML tag.

So if your forum allows to use HTML tags in messages, then it will be wise for you to turn on option:
Admin CP -> vBET -> Translation Options -> Don't translate code html tag

Please note that by default this option is disabled for better performance - as mentioned before for most forums it is not necessary. So do not turn it on if your forum, like most others, do not allow to include HTML tags in messages.

vBET
23-10-13, 00:50
vBET allows for users with appropriate usergroups to use manual translations. This not only will increase quality of your translations (by corrections), but also can decrease number of automatic translations. If your quotas in machine translation providers will be over then pages can be partially translated. In such case vBET will still allow to translate it manually. So maybe it will be translated automatically anyway when quota will be available again, but maybe some of your users will translate it completely manually and this will save you some characters of next quota (and money if you are using paid machine translation).

So it is OK to allow users for manual translations, but still remember that this is potential arena for abuse. That is why vBET allows it only to users with appropriate usergroups. You can set allowed usergroups in:
Admin CP -> vBET Manual Translations -> Options -> Usergroups allowed for manual translation
By default it is allowed only for: Super Moderators, Administrators and Moderators. Please read description of the option for more details.

You can allow more usergroups to use manual translation.

More usergroups allowed for manual translations means potential more corrected and free translations, but also more potential abuse. That is why vBET allows you not only to set who is allowed, but also to track those changes, reverse it and even automatically ban users who abuse by manual translation. You will find more about it here: http://www.vbenterprisetranslator.com/forum/vbet4-general-discussions/3166-manual-translations.html

vBET
23-10-13, 00:56
We are adding new optimizations when we see such possibility.

It includes new places where vBET separates numbers, times, dates, usernames, corrections of bugs, sometimes adding whole new machine translation or language detection providers and other optimizations (including performance).

So please use always newest vBET version to be sure that you have best solution available :)

vBET
23-10-13, 01:00
Some forums can have special areas which shouldn't be translated at all.

If you have such special pages which you do not want to be translated, then simply do not translate it. It is configurable - vBET allows to simply do this by option:
Admin CP -> vBET -> Translation Options -> Ignore URLs

vBET
23-10-13, 01:10
If you are starting using vBET, then there can be lot of content to translate on your forum until cache will be filled. In such case your quotas can be used for old threads and you can have not enough for new ones. There is no sense your new threads to wait for next quota because of old ones.

We have solution for that case - you can simply set how many days for thread without answer makes it old one:
Admin CP -> vBET -> Translation Options -> Don't tanslate old threads
When you set it, old threads will simply not be translated.

Please note that by default it is disabled (set to 0). You can set it if you need it. Also you can switch the setting to higher number from time to time and finally disable, if you see that your cache is filling and you still have free quota (no partial translations visible on pages).

vBET
23-10-13, 01:34
Option from previous hint can make some threads which was already translated not available on translated pages, because those become old after translation. But this is OK, because previous option is for new installations which want to focus on new posts and later fill cache for old ones. So it suppose to be disabled after all. Maybe after several changes to bigger value, but there is no sense to keep it for ever. Otherwise already indexed translated content can disappear from indexes (but not from cache, so when some answer will come then only new message will require new translation).

OK - but what in case when you do not want at all to translate some old threads. Newer - not when new answer will come. Not lose already translated content from index when replies stop coming...
Or when you want to rest from translations. Stop translating anything new, but not uninstall vBET and lose what is already translated and indexed.

For booth those cases your solution will be to configure option: Admin CP -> vBET -> Translation Options -> Translate only threads between date

You simply set here time period (from to) and only threads created in this time period will be translated. You can set:

Normal time range - only threads created in this time range will be translated.
Time range with really high future date as "to" - all threads from specified date will be translated including ne ones (because "to" is in the future - like year 3000)
Time range with very low past date as "from" - all old threads created up to "to" date will be translated (all because "from" is the distant past - like year 1000)

vBET
23-10-13, 01:46
This is more important when you have more spammer accounts created every day. Even when banned, such user can create new translations by his username added to GUI communicates on his member page. vBET separates and do not translates usernames when it is possible - unfortunately not always it is be possible.

So it is good idea to not show member pages for banned users at all - just error message instead (especially that banned users can still show their contact data on member page).
And vBET allows to block such pages by setting one option: Admin CP -> vBET -> Misc -> Block banned users member pages

Please note that by default this option is set, but it is configurable, because some forums can not have issues with spammers and some valuable member pages from old banned members (unlikely, but it is configurable, so make sure nobody disabled it and it is still on).

vBET
23-10-13, 01:58
If you are still using vBulletin 3.x then it can be good idea to change it to 4.x

Please note that vBulletin 4.x is much more sophisticated. Has more hooks and allows for better integration. And in fact vBET is better integrated with vBulletin 4.x than 3.x. Because 3.x simply do not allows for some things.

Booth vBET versions 3.x and 4.x has same functionality (basically - 4.x supports blogs, CMS and friendly URLS which are not included in vBulletin 3.x so cannot be supported in vBET 3.x). But in 4.x in more places vBET separated numbers, dates, times and usernames from GUI texts and that means less translations. In vBulletin 3.x there are places where we simply have no way to separate numbers and additional translations can appear.

Still please remember that switching from 3.x to 4.x is a big step so better think about it a lot. There are other MySQL and PHP limitations for vBulletin 4.x, other versions of all plugins, additional license for vBulletin and so on (no additional license for vBET).

So think hard is it worth to do this. And even when you think it is, first block your forum, do full backup and then start upgrades - there is a chance that the backup will save you in case of troubles.
But it is doable of course - our very forum was first on vBulletin 3.x :)

vBET
23-10-13, 02:10
If you generate on your server side some GUI parts which constantly changes then it can have big impact on translation costs by producing new translations constantly.
Of course users messages will be constantly changing - new ones, corrections, but this is valuable content and we want it translated.
Still translating some changing on server side text can be not the best idea.

So avoid this:

Do not allow for it at all
Use it in Java Script - it will change on client side and it will not be translated, since scripts cannot be translated
Use it on sever side, but put changing text into no translated area. See here (http://www.vbenterprisetranslator.com/forum/vbet4-general-discussions/3168-how-make-some-text-not-translated.html) for more info how to not translate text (last section: Notrasnlate area)
If changing part is username - then separate just username from rest of the text by no translate area from previous point
If changing part is number, date or time - then separate it from rest of the text by any HTML tag (like span, i, u...)

AfrikaansAlbanianArabicBelarusianBulgarianCatalanChineseCroatianCzechDanishDutchEnglishEstonianFilipinoFinnishFrenchGalicianGermanGreekHaitian CreoleHebrewHindiHungarianIcelandicIndonesianIrishItalianJapaneseKoreanLatvianLithuanianMacedonianMalayMalteseNorwegianPersianPolishPortugueseRomanianRussianSerbianSlovakSlovenianSpanishSwahiliSwedishTaiwaneseThaiTurkishUkrainianVietnameseWelshYiddish
Languages translations delivered by vBET 4.9.99