[Zope-dev] Re: Additional locales for zope.i18n.locales.data?

[Nathan Yergler]

[Re-including the list in the reply...]

On 11/27/07, Hanno Schlichting <plone at hannosch.info> wrote:
> Nathan Yergler wrote:
> > On 11/27/07, Philipp von Weitershausen <philipp at weitershausen.de> wrote:
> >> Nathan Yergler wrote:
> >>> I've run into a problem this morning with i18n in Zope3, and I'm not
> >>> exactly sure what the appropriate approach is.  We have a set of South
> >>> African translations, paticularly Zulu (zu) and Sotho (st) that are
> >>> getting picked up correctly.  Upon some inspection it appears that the
> >>> i18n negotiator is skipping over them, with an exception such as:
> >>>
> >>> The desired locale is not available.
> >>> Path: /Users/nathan/p/cc.engine/branches/production/eggs/zope.i18n-3.4.0-py2.4.egg/zope/i18n/locales/data/st.xml
> >>>
> >>> Upon inspection I can confirm that the locales which are not working
> >>> are those without .xml files in the data directory.  So a few
> >>> questions:
> >>>
> >>> * Is there an external source for those .xml files, or do we create
> >>> them ourselves?
> >> The XML files are from the ICU repository
> >> (http://www.unicode.org/cldr/). AFAIK we're using quite an old version
> >> of the files. The problem is we can't simply upgrade to the newest
> >> release because the XML DTD changed and the code in zope.i18n.locales
> >> isn't very robust regarding DTD changes (which is quite hard anyway). I
> >> don't think anybody would object if the files and the code were updated
> >> *wink* ;).
> >
> > Where "wink" is defined as "so why don't you go ahead and do it", I expect :).
>
> If you follow the wink,
> http://www.unicode.org/cldr/repository_access.html has the details to
> the files. Currently the latest is at
> http://unicode.org/Public/cldr/1.5.0/core.zip.
>
> Zope at this point still uses LDML 1.0 whereas the latest version is
> LDML 1.5.
>
> Upon casual inspection of the files it seems their basic structure is
> still the same, though more careful inspection is required.

I've been avoiding even less interesting work this afternoon, taking a
look at this.  I started by dumping the new files into the data
directory and just running the tests.  As expected, things blew up in
a really spectacular manner.

After some wrangling I've discovered that in the newer versions of the
CLDR dataset some of the information previously contained in the
locale files (such as weekend start/end, etc), is now in located in a
"supplemental" file.  While this makes a certain amount of sense (it's
tied to territories, not really languages), it does mean that the
information needed for a Locale is no longer self-contained in a
single XML file.

So unfortunately it's going to require some more work to fix up the
loader; I'll probably create a branch to work on this some...


>
> Hanno
>
> P.S. Updating those files is on my list for some month now, but I don't
> expect to have any time for that for real in the near future :(
>