Internationalization is an increasingly important consideration for Magento merchants developers looking to expand market penetration and increase usability. A significant part of this effort is realized in the form of maintaining translations for multiple locales – quite the undertaking, in spite of Magento’s robust localization capabilities.
However, a journey of a thousand miles begins with a single step, and this initial step can be particularly daunting. What must be translated?
Ideally, every string ever used, be it backend or frontend, would be documented so that an exhaustive list is always available of material scheduled for translation. In practice, however, this is rarely the case – maybe the site or module wasn’t initially slated for an international market or the ROI was difficult to justify. Because of this, orphan strings with no record of their existence are very common and a barrier to internationalization.
Wouldn’t it be nice to have a mechanism to retroactively examine a site or module and perform a translation gap analysis?
One approach to ferreting out untranslated strings is to modify the translation tool itself to report untranslated strings as they are encountered. This is often expressed as a quick hack to the translation classes whereby strings are logged, then the changes reverted.
The basic idea is solid, but the execution is essentially a transient hack – requiring repeated discovery and implementation, and is prone to oversights.
After a recent internationalization-intensive project involving several countries and localizations, I attempted to formalize this approach into a robust module.
When calling the familiar __() method (whether from a block, helper, controller, etc), Mage_Core_Model_Translate::translate() is invoked to do the actual heavy lifting.
Mage_Core_Model_Translate::translate()
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 |
/** * Translate * * @param array $args * @return string */ public function translate($args) { $text = array_shift($args); if (is_string($text) & amp; & amp; '' == $text || is_null($text) || is_bool($text) & amp; & amp; false === $text || is_object($text) & amp; & amp; '' == $text - & gt; getText()) { return ''; } if ($text instanceof Mage_Core_Model_Translate_Expr) { $code = $text - & gt; getCode(self::SCOPE_SEPARATOR); $module = $text - & gt; getModule(); $text = $text - & gt; getText(); $translated = $this - & gt; _getTranslatedString($text, $code); } else { if (!empty($_REQUEST['theme'])) { $module = 'frontend/default/'.$_REQUEST['theme']; } else { $module = 'frontend/default/default'; } $code = $module.self::SCOPE_SEPARATOR.$text; $translated = $this - & gt; _getTranslatedString($text, $code); } //array_unshift($args, $translated); //$result = @call_user_func_array('sprintf', $args); $result = @vsprintf($translated, $args); if ($result === false) { $result = $translated; } if ($this - & gt; _translateInline & amp; & amp; $this - & gt; getTranslateInline()) { if (strpos($result, '{{{') === false || strpos($result, '}}}') === false || strpos($result, '}}{{') === false) { $result = '{{{'.$result. '}}{{'.$translated. '}}{{'.$text. '}}{{'.$module. '}}}'; } } return $result; } |
After doing some heuristics to determine exactly what module and code to use for context, Mage_Core_Model_Translate::_getTranslatedString() takes over.
Mage_Core_Model_Translate::_getTranslatedString()
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
/** * Return translated string from text. * * @param string $text * @param string $code * @return string */ protected function _getTranslatedString($text, $code) { $translated = ''; if (array_key_exists($code, $this->getData())) { $translated = $this->_data[$code]; } elseif (array_key_exists($text, $this->getData())) { $translated = $this->_data[$text]; } else { $translated = $text; } return $translated; } |
If a translation by either code or text exists, then it is returned – otherwise, the key (untranslated string) is returned.
The Mage_Core_Model_Translate::_getTranslatedString() method offers a perfect opportunity to detect strings with no translation and record them.
Unfortunately, there is no event present which can be observed to accomplish this detection, so the model must be rewritten. Using the lightest touch possible for such a fundamental model, the string is preprocessed before translation to determine if it is missing translations for interesting locales.
EW_UntranslatedStrings_Model_Core_Translate::_getTranslatedString()
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 |
/** * Evaluate translated text and code and determine * if they are untranslated. * * @param string $text * @param string $code */ protected function _checkTranslatedString($text, $code) { Varien_Profiler::start(__CLASS__ . '::' . __FUNCTION__); Varien_Profiler::start(EW_UntranslatedStrings_Helper_Data::PROFILER_KEY); //loop locale(s) and find gaps $untranslatedPhrases = array(); foreach($this->_getLocalesToCheck() as $locale) { if(!Mage::helper('ew_untranslatedstrings')->isTranslated($text,$code,$locale)) { $untranslatedPhrases[] = array( 'text' => $text, 'code' => $code, 'locale' => $locale ); } } $this->_storeUntranslated($untranslatedPhrases); Varien_Profiler::stop(EW_UntranslatedStrings_Helper_Data::PROFILER_KEY); Varien_Profiler::stop(__CLASS__ . '::' . __FUNCTION__); } /** * Check for translation gap before returning * * @param string $text * @param string $code * @return string */ protected function _getTranslatedString($text, $code) { if(Mage::helper('ew_untranslatedstrings')->isEnabled()) { $this->_checkTranslatedString($text, $code); } return parent::_getTranslatedString($text, $code); } |
This simple change allows the module to collect untranslated strings, which, for performance sake, are batched up and flushed to the database in a single query after the page is rendered. All that is required to perform this collection is to enable the module in the system configuration, and browse the site – a perfect scenario for stage sites during ongoing UAT.
If a merchant or developer is concerned about translation gaps for one locale, he is likely also concerned about several others. This is facilitated by the loop in _checkTranslatedString() which iterates over multiple locales which are ultimately retrieved from the system configuration.
To check the status of translations other than the currently selected locale, EW_UntranslatedStrings_Helper_Data::isTranslated() performs evaluations independent of current store configuration, aided by getTranslator() which provides ready-to-go translator models for any locale and store ID.
EW_UntranslatedStrings_Helper_Data::isTranslated() and getTranslator()
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
/** * Get translator prepared for given locale * * @param $locale * @param bool $allowMatchingKeyValuePairs - matching key / value pairs count as translations * @param null $storeIdContext * @param bool $forceRefresh * @return EW_UntranslatedStrings_Model_Core_Translate */ public function getTranslator($locale, $allowMatchingKeyValuePairs = null, $storeIdContext = null, $forceRefresh = false) { if(!isset($this->_translators[$locale])) { if(is_null($allowMatchingKeyValuePairs)) { // "allow" and "log" are opposite concepts $allowMatchingKeyValuePairs = !$this->logMatchingKeyValuePairs(); } /* @var $translate EW_UntranslatedStrings_Model_Core_Translate */ $translate = Mage::getModel('ew_untranslatedstrings/core_translate'); $translate->setConfig( array( Mage_Core_Model_Translate::CONFIG_KEY_LOCALE => $locale ) ); $translate->setLocale($locale); $translate->setAllowLooseDevModuleMode(true); //prevent native dev mode differences $translate->setAllowMatchingKeyValuePairs($allowMatchingKeyValuePairs); if(!is_null($storeIdContext)) { $translate->setThemeContext($storeIdContext); } $translate->init(Mage_Core_Model_Design_Package::DEFAULT_AREA, $forceRefresh); $this->_translators[$locale] = $translate; } return $this->_translators[$locale]; } /** * Does text/code have translation for given locale? * * @param $text * @param $code * @param $locale * @return bool */ public function isTranslated($text, $code, $locale) { /* @var $translate EW_UntranslatedStrings_Model_Core_Translate */ $translate = $this->getTranslator($locale); return $translate->hasTranslation($text, $code); } |
While it requires a few more rewritten methods on the translator model, this feature allows a store owner or developer to quickly assemble a gap analysis of multiple locales at once.
Although simple in concept, there are several optional tweaks that can make this feature much more useful for efficient string collection.
This module adds a new system configuration section which can be found at Advanced -> Developer -> Untranslated Strings.
These options allow several powerful options for customizability. For example
Even the most robust untranslated string detection is useless if the results are not easily accessible and actionable.
To this end, there are two new reports which are now available at Reports -> Untranslated Strings in the main admin menu.
Additionally, the Untranslated Strings Report allows strings to be exported to CSV file. Combined with its filtering capabilities, this allows a merchant or developer to effectively manage the strings and provide them to translation teams, along with all important context.
Viewing the untranslated strings report as a sort of “to-do list”, the summary view allows merchants or developers to curate the remaining untranslated strings. In particular, the strings associated with a given locale and store can be truncated (individually or en masse), cleaning the slate and allowing a fresh set of strings to be collected.
More powerful, however, is the purge option. Exercising this option on a locale and store (or group of these, via the mass action functionality) causes the module to reevaluate the translation status of associated strings, removing any which are now considered translated according to the system configuration settings.
After adding translations to address gaps, this feature allows merchants and developers to cull strings which are no longer relevant, ensuring that the untranslated string list is always actionable.
While this module goes a long way to formalize a former “hacky” process, there is always room to improve. Some particular things to keep in mind:
If you’d like to get your hands on this open source module and take the first step toward internationalization, you can find it on Github:
https://github.com/ericthehacker/magento-untranslatedstrings
As stated in the readme, it’s easily installed via modman. Use it wisely!
Interested in building an international e-commerce store or looking to upgrade the one you have? Let us know on our contact form!
3 Comments
What’s the strategy for going forward when all the untranslated strings are found. Preferably we’d like to keep the CSV in git and then dump them into some locale dir in the Magento hierarchy. But I guess you’ll need to know where themes, and plugins read their CSV locale files from?
Hi Eric
Fantastic work! However, my admin grid for ‘untranslated strings’ is lacking some layout. Is there a layout xml file missing from the github repository for above?
I have seen a few modules to help with translation, but your code is the code I want, I just want the layout working in admin so I can see some untranslated strings.
If I am right and things are broken, please fix as your module looks superb and I definitely want it!
P J
Glad the module could help!
There is one layout file which should be included: https://github.com/ericthehacker/magento-untranslatedstrings/blob/master/app/design/adminhtml/default/default/layout/ew_untranslatedstrings.xml .
Looks like this post lost its formatting and images during a recent data migration, so I’ve restored the screenshots to let you know what to expect to see. Does your instance differ from the screenshots? If so, can you post a screenshot?