--On 21. November 2005 15:14:50 +0100 Lennart Regebro <regebro@gmail.com> wrote:

Well, that would be cool. Just a question: How do you plan to keep and verify backwards compatibility? Any database, with any type of DateTime object, must work transparently.

Perfect question. When we considered rewriting DateTime using mx.DateTime in 2001 we could answer this question :-) That's why the rewrite never happened. -aj

--On 21. November 2005 15:37:42 +0100 Jürgen Herrmann <Juergen.Herrmann@XLhost.de> wrote:

btw. you mentioned mxDateTime - is it recommendable to make it a requirement for zope or should we better stick to python's datetime?

Python's datetime module would be the way to go. In 2001/2002 mx.DateTime was the only reasonable alternative. -aj

[ Lennart Regebro wrote:]

On 11/21/05, Jürgen Herrmann <Juergen.Herrmann@xlhost.de> wrote:

...

recently i came up here with the intention to fix DateTime#strftime(). while trying this, i had to dig deeper and deeper into the implementation of DateTime and especially the timezone and daylight saving stuff. to be honest, it's completely hacked together :(

Yup.

...

sorry guys, i won't be able to completely fix this for now. i found a way to monkey patch zope to make it work for my case (2 timezones only). my plan is to completely reimplement DateTime, based on python's datetime in my own freetime (maybe around xmas this year) and give it back to the community.

Well, that would be cool. Just a question: How do you plan to keep and verify backwards compatibility? Any database, with any type of DateTime object, must work transparently.

100% compatibility isn't desireable, because its broken atm ;P _______________________________________________________________________

...

XLhost.de - eXperts in Linux hosting <<

Jürgen Herrmann Bruderwöhrdstraße 15b, DE-93051 Regensburg Fon: +49 (0)700 XLHOSTDE [0700 95467833] Fax: +49 (0)721 151 463027 WEB: http://www.XLhost.de

On 11/23/05, Chris Withers <chris@simplistix.co.uk> wrote:

Hi Jürgen,

Many thanks for even thinking about looking at the DateTime debacle ;)

Jürgen Herrmann wrote:

...

...

Well, that would be cool. Just a question: How do you plan to keep and verify backwards compatibility? Any database, with any type of DateTime object, must work transparently.

100% compatibility isn't desireable, because its broken atm ;P

Well yes, but you don't want to force people to change in an unpleasant way. From what I can see, DateTime _should_ just be a lightweight wrapper around Python's datetime.datetime to add the extra formatting commands, helper methods and zodb persistence.

However, it isn't, and I don't think changing it to be so is the way to go.

What I _would_ like to see is a new package that works with both Zope 3 and Zope 2 and offers exactly this. We could advocate switching to this, with DateTime becoming deprecated and removed in, say, Zope 2.10.

-1 on making DateTime deprecated. I'm sure thousands of zope people use DateTime in thousands of places. Technically it might be correct deprecate it but politically it's just going to put people off from upgrading.

I wonder if Zope 3 has something that meets these needs already? Perhaps you could have a dig and find out on zope3-dev@zope.org?

cheers,

Chris

-- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk

_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )

-- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com

Lennart Regebro wrote:

It depends on what you mena with deprecated. It *is* deprecated from the point of view, that if you start developing something new now, then don't use it.

Really? What should we be using instead then?

It is not deprecated in the way that "this will go away in the future", though. It must stay for backwards compatibility reasons, but if it can be fixed, than this would be a Good Thing.

I honestly think it _should_ go away, just with plenty of notice... cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk

Peter Bengtsson wrote:

-1 on making DateTime deprecated. I'm sure thousands of zope people use DateTime in thousands of places. Technically it might be correct deprecate it but politically it's just going to put people off from upgrading.

I'm sure it would be fairly trivial to write an unpgrader, but really, deprecating it, even if the deprecation period is fairly long, is the only sane way to tackle this... cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk

--On 25. November 2005 12:22:05 -0800 Dennis Allison <allison@shasta.stanford.edu> wrote:

If I have a vote, +1 on deprecation and replacement with something less buggy.

You can only deprecate when you have something better. As long as there is no useful replacemenet there is nothing to deprecate :-) -aj

--On 25. November 2005 12:43:48 -0800 Dennis Allison <allison@shasta.stanford.edu> wrote:

True. But the context of this thread was whether deprecation made sense or not. I was agreeing with Chris that the DateTime problems are better solved by replacement than by patching, but also that there is a large amount of code that has been made to work using the current DataTime and that investment needs to be protected.

I agree that is should be replaced but I also mentioned that we tried to achieve that already some years ago and we gave up....so talking about deprecation really makes sense when someone puts a replacement module on the table. I know that the DateTime module is a piece of crap but crap you can live mostly with (as long as you don't need timezones :-)). -aj

Chris Withers wrote:

Andreas Jung wrote:

...

I agree that is should be replaced but I also mentioned that we tried to achieve that already some years ago and we gave up....so talking about deprecation really makes sense when someone puts a replacement module on the table. I know that the DateTime module is a piece of crap but crap you can live mostly with (as long as you don't need timezones :-)).

Okay, well, I'd advocate replacement with a zope.datetime that subclasses python's datetime, mixes in persistence and provides some of the extra helper methods.

What do people think?

cheers,

Chris

(CC'ing Phillip 'cos I saw him doing some Zope 3'ish DateTime stuff...)

Hey Chris, hey Hermann, hey others, I've secretly being working on some evil plans to make DateTime more understandable so that we can work out a migration strategy. In my strong opinion, Zope should not maintain its own date/time implementation and I don't even see the need for a zope.datetime like Chris suggests. Why does it need to be persistent? Datetimes are dull values, they should just pickle nicely. If someone needs more than datetime.datetime and pytz, I would be very interested in their usecase... Anyway, my long-term plans are roughly this: 1. Create some extensive tests about how DateTime currently works. I'm currently working on this to see whether any further procedure makes sense. 2. If we find it's possible, we rid the current DateTime implementation and recreate the DateTime class by subclassing datetime.datetime. For backwards compatability, we make sure that old pickles can be revived and that the old DateTime API is supported for two more Zope releases. 3. After two releases we get rid of the old DateTime API and will provide a script to migrate old DateTime pickles to datetime.datetime pickles in the ZODB. I have a proposal for this in works, but you guys made me blurt it out prematurely :). By the way, I only skimmed over this thread, but I haven't actually found anywhere explained what Hermann's problem with strftime was and a detailed suggestion on how his "rewrite" would take place... Maybe it'd be a good idea to adopt the proposal process we have for Zope 3. Philipp

Andrew Milton wrote:

+-------[ Philipp von Weitershausen ]----------------------

| 2. If we find it's possible, we rid the current DateTime implementation | and recreate the DateTime class by subclassing datetime.datetime. For | backwards compatability, we make sure that old pickles can be revived | and that the old DateTime API is supported for two more Zope releases. | | 3. After two releases we get rid of the old DateTime API and will | provide a script to migrate old DateTime pickles to datetime.datetime | pickles in the ZODB.

-1 for any scheme that deliberately breaks currently working code / installs.

Agreed. Deprecation is a way of deliberately killing you softly :). Seriously, there's no going forward without getting rid of stuff. Deprecation is a good way of dealing with getting rid of stuff without having currently working code broken instantly.

-1 for any scheme that involves diddling the ZODB to 'fix' pickles, because you just know you're going to corrupt someone's ZODB, and that's just noone's idea of fun.

There are sensible ways of upgrading the ZODB. Zope 3 has had it since 3.0 (called generations) and they've been working reasonably well for these things.

Didn't see any mention of fixing ZClasses (not sure if that's an issue).

I'm not sure it is either, but this would be something to be discussed in the proposal discussion process, I guess. But now that you've mentioned it, I will investigate and possibly include notions in the proposal.

I'm the first in line of the people wanting Zope DateTime to die. However, you need to leave it there. Fix Zope to internally use something different, and provide a new implementation that 'sensible' people can use going forwards.

I don't think that'll work. I've thought about just letting DateTime die and switch to datetime.datetime over completely. However, code that expects Zope to return DateTime-compatible objects would break. I think we need Zope to use some frankendatetime (a datetime.datetime that also supports the old DateTime API) throughout the deprecation period. Whenever people make a call to the old DateTime API on a frankendatetime object, they'll get a deprecation warning telling them to use the datetime.datetime API instead. After the deprecation phase is over, Zope 2 can return pure datetime.datetime objects. By the way, this is really not a thread for zope@zope.org but for zope-dev@zope.org. From the beginning, actually. I suggest we move all further discussion there. Philipp

Chris Withers wrote:

Philipp von Weitershausen wrote:

...

I think we need Zope to use some frankendatetime (a datetime.datetime that also supports the old DateTime API) throughout the deprecation period. Whenever people make a call to the old DateTime API on a frankendatetime object, they'll get a deprecation warning telling them to use the datetime.datetime API instead. After the deprecation phase is over, Zope 2 can return pure datetime.datetime objects.

I don't see any need for diabolical creations ;-) Just the deprecation warnings will suffice. (...which will promptly be added to warn filters, and only fixed when DateTime really goes away *sigh*)

Ok, so take this example: A piece of code calls bobobase_modification_time() which currently returns a DateTime; after that the piece of code calls some DateTime-specific methods on the returned object. Now in Zope 2.12 we make bobobase_modification_time() return a datetime.datetime and subsequently the piece of code would fail. There are a lot of pieces of code like this in third party products... We could deprecate DateTime now to warn people about that. But then their product would only either work with Zope <2.12 or with Zope >2.12 (depending on whether it still expects the DateTime API or the datetime.datetime API); it would not with both versions. That's a harsh upgrade plan and it would probably take ages before people would adopt Zope 2.12. With a frankendatetime in Zope 2.10, however, they could still call DateTime-specific methods on the return value of bobobase_modification_time(). They'd get deprecation warnings telling them to use the datetime.datetime API instead. That would work instantly, not only in Zope 2.12. That way they could smoothly upgrade their products. By the way, I've come to think that touching the current DateTime implementation wouldn't be a good idea, mostly because of the old pickles issue. The frankendatetime should be a separate class that subclasses datetime.datetime, provides old-style DateTime compatability and has datetime.datetime-compatible pickles. Philipp

[ Lennart Regebro wrote:]

On 11/29/05, Philipp von Weitershausen <philipp@weitershausen.de> wrote:

...

We could deprecate DateTime now to warn people about that. But then their product would only either work with Zope <2.12 or with Zope >2.12 (depending on whether it still expects the DateTime API or the datetime.datetime API); it would not with both versions. That's a harsh upgrade plan and it would probably take ages before people would adopt Zope 2.12.

I agree, official deprecation and warnings should only happen when we have a real backwards compatibility and transition-path. Which would include what you call a "frankendatetime". :-)

Which, as I understand it, is what Jürgen Herrman wanted to create...

correct that is ;) regards, juergen herrmann _______________________________________________________________________

...

XLhost.de - eXperts in Linux hosting <<

Jürgen Herrmann Bruderwöhrdstraße 15b, DE-93051 Regensburg Fon: +49 (0)700 XLHOSTDE [0700 95467833] Fax: +49 (0)721 151 463027 WEB: http://www.XLhost.de

A piece of code calls bobobase_modification_time() which currently returns a DateTime; after that the piece of code calls some DateTime-specific methods on the returned object. Now in Zope 2.12 we make bobobase_modification_time() return a datetime.datetime and subsequently the piece of code would fail. There are a lot of pieces of code like this in third party products...

Good point, I don't see a nice way around this though :-S Your frankendatetime is one possibility, but for me it feels just as nasty as the scenario you describe above :-S

We could deprecate DateTime now to warn people about that. But then their product would only either work with Zope <2.12 or with Zope >2.12 (depending on whether it still expects the DateTime API or the datetime.datetime API); it would not with both versions. That's a harsh upgrade plan and it would probably take ages before people would adopt Zope 2.12.

Yup...

With a frankendatetime in Zope 2.10, however, they could still call DateTime-specific methods on the return value of bobobase_modification_time(). They'd get deprecation warnings telling them to use the datetime.datetime API instead. That would work instantly, not only in Zope 2.12. That way they could smoothly upgrade their products.

OK, I'm convinced, _if_ you can make it work ;-)

By the way, I've come to think that touching the current DateTime implementation wouldn't be a good idea, mostly because of the old pickles issue.

My thoughts exactly.

The frankendatetime should be a separate class that subclasses datetime.datetime, provides old-style DateTime compatability and has datetime.datetime-compatible pickles.

sounds good to me :-) cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk

On 11/28/05, Philipp von Weitershausen <philipp@weitershausen.de> wrote:

Datetimes are dull values, they should just pickle nicely.

They do. A module with extra utilities + timezone support would be nice.

1. Create some extensive tests about how DateTime currently works. I'm currently working on this to see whether any further procedure makes sense.

I'm not worried about how it works now. DateTime is buggy and it's behaviour has undergone several subtle changes, sometimes for the worse, without much screaming. :-)

2. If we find it's possible, we rid the current DateTime implementation and recreate the DateTime class by subclassing datetime.datetime. For backwards compatability, we make sure that old pickles can be revived and that the old DateTime API is supported for two more Zope releases.

Yeah, it's that pickling revival that worries me. It's non-trivial. -- Lennart Regebro, Nuxeo http://www.nuxeo.com/ CPS Content Management http://www.cps-project.org/

Philipp von Weitershausen wrote:

Chris suggests. Why does it need to be persistent?

See the use case I sketched out in my other reply...

1. Create some extensive tests about how DateTime currently works. I'm currently working on this to see whether any further procedure makes sense.

yay!

2. If we find it's possible, we rid the current DateTime implementation and recreate the DateTime class by subclassing datetime.datetime. For backwards compatability, we make sure that old pickles can be revived and that the old DateTime API is supported for two more Zope releases.

-1 Too complicated. I prefer the deprecate, provide migration utility, then remove method...

3. After two releases we get rid of the old DateTime API and will provide a script to migrate old DateTime pickles to datetime.datetime pickles in the ZODB.

...yes, this. Just skip step 2 ;-)

By the way, I only skimmed over this thread, but I haven't actually found anywhere explained what Hermann's problem with strftime was and a detailed suggestion on how his "rewrite" would take place... Maybe it'd be a good idea to adopt the proposal process we have for Zope 3.

As I said elsewhere, I'd love to have one datetime and timezone story for both Zope 2 and Zope 3, and I think Zope 3 has better processes for making that happen ;-) cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk

[ Florent Guillaume wrote:]

Jürgen Herrmann wrote:

...

recently i came up here with the intention to fix DateTime#strftime(). while trying this, i had to dig deeper and deeper into the implementation of DateTime and especially the timezone and daylight saving stuff. to be honest, it's completely hacked together :( DateTimeZone.py has one BIG dictionary in it, not a single line of comments. values of this dict are nested lists, among other hackish things (list like usage of a string, with \000 as separator). the methods that use this dict also have no comments/docstrings at all. obviously the guy(s) that originally wrote this, is/are hiding (i know why :) so, there's nobody to ask either...

sorry guys, i won't be able to completely fix this for now. i found a way to monkey patch zope to make it work for my case (2 timezones only). my plan is to completely reimplement DateTime, based on python's datetime in my own freetime (maybe around xmas this year) and give it back to the community.

once again sry, if i raised expectations on the fix of strftime.

Yes replacing DateTime is a laudable but difficult goal.

One thing that could be done meanwhile is just refactor the unit test to be a base class that could then be used to test DateTime or to test another potential implementation. That would go a long way to help actually write a new implementation.

Florent

hi florent! actually that's the best thing to do! this way the implementer knows what to do exactly :) but be aware that some tests got modified to pass with current (broken) behaviour! one more question (to the public!): do we REALLY need dates <1900 / >2036 ? using unix timestamps for storage and as the base for all conversions would make things a lot easier! regards, juergen herrmann _______________________________________________________________________

...

XLhost.de - eXperts in Linux hosting <<

Jürgen Herrmann Bruderwöhrdstraße 15b, DE-93051 Regensburg Fon: +49 (0)700 XLHOSTDE [0700 95467833] Fax: +49 (0)721 151 463027 WEB: http://www.XLhost.de

[ Lennart Regebro wrote:]

On 11/22/05, Jürgen Herrmann <Juergen.Herrmann@xlhost.de> wrote:

...

do we REALLY need dates <1900 / >2036 ?

Yes.

...

using unix timestamps for storage and as the base for all conversions would make things a lot easier!

datetimes are picklable, so if you are going to change how they are stored (which may not be a good idea for backwards compatible reasons), you should use that.

i'll surely change the storage format, when rewriting it! storing year, month, day, hour, minute, second, a unix timestamp and tzinfo is plain odd! so using a datetime instance for storage indeed seems the most obvious thing if we need dates <1900 / >2036. regards, juergen herrmann _______________________________________________________________________

...

XLhost.de - eXperts in Linux hosting <<

Jürgen Herrmann Bruderwöhrdstraße 15b, DE-93051 Regensburg Fon: +49 (0)700 XLHOSTDE [0700 95467833] Fax: +49 (0)721 151 463027 WEB: http://www.XLhost.de

[ Lennart Regebro wrote:]

On 11/22/05, Jürgen Herrmann <Juergen.Herrmann@xlhost.de> wrote:

...

i'll surely change the storage format, when rewriting it!

So you plan on having some version marker, or so, which tells which storage format is used?

//Curious.

basicall i thought about having a dateime instance as "storage" in the new implementation and test for it's existance on DateTime instances. something like that: dt = getattr(self, '_datetime', None) if dt is not None: #new, everything ok else: #migrate! rfc! regards, juergen herrmann _______________________________________________________________________

...

XLhost.de - eXperts in Linux hosting <<

Jürgen Herrmann Bruderwöhrdstraße 15b, DE-93051 Regensburg Fon: +49 (0)700 XLHOSTDE [0700 95467833] Fax: +49 (0)721 151 463027 WEB: http://www.XLhost.de

At Tuesday 22/11/2005 05:50, Jürgen Herrmann wrote:

one more question (to the public!): do we REALLY need dates <1900 / >2036 ? using unix timestamps for storage and as the base for all conversions would make things a lot easier!

Sure. What about birthdays of aged people? Long running forecasts? Gabriel Genellina Softlab SRL