Globally unique id's in Zope in a new way
Zopsters, I use the Catalog for quite a bit of jumping around the ZODB, and have a 30,000 plus objects in a system. I like the idea of not having to worry about id's conflicting with other numbers that might appear (invoice numbers, employee numbers, etc...) so I don't like to use a counter for ids. The ZopeTime() to int to string conversion works unless you are programatically creating multiple items in the same folder, and have a descent processor. I have derived the following bad hack from the code used to generate ZClass ids. Does anyone know off hand how many different combinations this offers? Is it considered unique? It returns a url valid string with no special signs. import md5, base64, time, string from urllib import quote from whrandom import choice def unique_id(self): id=md5.new() id.update(self.absolute_url()) id.update(str(time.time())) id=id.digest() id=string.strip(id) id=string.strip(base64.encodestring(id)) id=quote(id[:-2]) seq = string.hexdigits + string.lowercase + string.uppercase new_id = '' for l in id: if l == '/' or l == '%': new_id = new_id + choice(seq) else: new_id = new_id + l return new_id If anyone has a more elegant solution I would love to set my tired eyes on it. ;-) All my best, -- Jason Spisak 444@hiretechs.com
On Wed, 8 Mar 2000, Jason Spisak wrote:
import md5, base64, time, string from urllib import quote from whrandom import choice def unique_id(self): id=md5.new() id.update(self.absolute_url()) id.update(str(time.time())) id=id.digest() id=string.strip(id) id=string.strip(base64.encodestring(id)) id=quote(id[:-2]) seq = string.hexdigits + string.lowercase + string.uppercase new_id = '' for l in id: if l == '/' or l == '%': new_id = new_id + choice(seq) else: new_id = new_id + l return new_id
If anyone has a more elegant solution I would love to set my tired eyes on it. ;-)
All my best,
Couldn't you use a combination of ZopeTime and a random integer in the range 1-10000000? The whrandom module is available to _ I believe. Pavlos
That sounds like a more elegant descision. I don't know why, but something in me said that a wholelly random sting needed to be generated. Really only the last integer conflicts. Boy was I tired last night 8-| Thanks again, Pavlos Christoforou writes:
On Wed, 8 Mar 2000, Jason Spisak wrote:
import md5, base64, time, string from urllib import quote from whrandom import choice def unique_id(self): id=md5.new() id.update(self.absolute_url()) id.update(str(time.time())) id=id.digest() id=string.strip(id) id=string.strip(base64.encodestring(id)) id=quote(id[:-2]) seq = string.hexdigits + string.lowercase + string.uppercase new_id = '' for l in id: if l == '/' or l == '%': new_id = new_id + choice(seq) else: new_id = new_id + l return new_id
If anyone has a more elegant solution I would love to set my tired eyes on it. ;-)
All my best,
Couldn't you use a combination of ZopeTime and a random integer in the range 1-10000000? The whrandom module is available to _ I believe.
Pavlos
Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.
At 07:49 PM 3/8/00 -0500, Jason Spisak wrote:
Zopsters,
I use the Catalog for quite a bit of jumping around the ZODB, and have a 30,000 plus objects in a system. I like the idea of not having to worry about id's conflicting with other numbers that might appear (invoice numbers, employee numbers, etc...) so I don't like to use a counter for ids. [...]
I have the same problem. I want to automatically create objects and I'm currently using the time method. When I bulk add objects, I would get a collision on the time value, so I simply remembered the last id and added 1 to that value. It works for my current needs, but I wish Zope provided a standard mechanism for generating unique id's. For example, it would actually be nice if Zope provided an implementation of UUID (called GUID by Microsoft). A UUID is a guaranteed unique 64-bit value. It's used in COM, CORBA and other systems which need to have automatically generated guaranteed unique values. The nice thing about UUID's is that they are guaranteed to be unique network wide. In other words, you (supposedly) can't generate a UUID which would ever conflict with a UUID that I generated. It does this by an algorithm which takes into account things like time-of-date, network card address, etc. I don't have the details on the algorithm but I know it is publically available. It would probably best be implemented at the C level. Just an idea. James W. Howe mailto:jwh@allencreek.com Allen Creek Software, Inc. pgpkey: http://ic.net/~jwh/pgpkey.html Ann Arbor, MI 48103
You mean this: print pythoncom.CreateGuid() creates something like {30BD3490-2632-11cf-AD5B-524153480001} It's from the Pythom COM package. I guess all you need is access to that package from inside Zope and an external method would do the trick. Jason James W. Howe writes:
At 07:49 PM 3/8/00 -0500, Jason Spisak wrote:
Zopsters,
I use the Catalog for quite a bit of jumping around the ZODB, and have a 30,000 plus objects in a system. I like the idea of not having to worry about id's conflicting with other numbers that might appear (invoice numbers, employee numbers, etc...) so I don't like to use a counter for ids. [...]
I have the same problem. I want to automatically create objects and I'm currently using the time method. When I bulk add objects, I would get a collision on the time value, so I simply remembered the last id and added 1 to that value. It works for my current needs, but I wish Zope provided a standard mechanism for generating unique id's.
For example, it would actually be nice if Zope provided an implementation of UUID (called GUID by Microsoft). A UUID is a guaranteed unique 64-bit value. It's used in COM, CORBA and other systems which need to have automatically generated guaranteed unique values. The nice thing about UUID's is that they are guaranteed to be unique network wide. In other words, you (supposedly) can't generate a UUID which would ever conflict with a UUID that I generated. It does this by an algorithm which takes into account things like time-of-date, network card address, etc. I don't have the details on the algorithm but I know it is publically available. It would probably best be implemented at the C level.
Just an idea.
James W. Howe mailto:jwh@allencreek.com Allen Creek Software, Inc. pgpkey: http://ic.net/~jwh/pgpkey.html Ann Arbor, MI 48103
Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.
Do you know if this module is available on all Python platforms, or is it only Win32? At 05:29 PM 3/9/00 +0000, Jason Spisak wrote:
You mean this:
print pythoncom.CreateGuid()
creates something like
{30BD3490-2632-11cf-AD5B-524153480001}
It's from the Pythom COM package. I guess all you need is access to that package from inside Zope and an external method would do the trick.
Jason
James W. Howe writes:
[...]
For example, it would actually be nice if Zope provided an implementation of UUID (called GUID by Microsoft).
James W. Howe mailto:jwh@allencreek.com Allen Creek Software, Inc. pgpkey: http://ic.net/~jwh/pgpkey.html Ann Arbor, MI 48103
I've been bulk adding objects today, and have come up with this method: If you use: <dtml-call "REQUEST.set('id',_.str(_.int(1000000*_.DateTime().second())))"> Then you will have an id unique to 1 millionth of a second. If this is still not good enough then you can do 10,000,000, etc. -Ed
This is, however, only unique for a few minutes. So if you're after longer lasting ones, then combine it with the id=ZopeTime way. (I'm not sure how long ids can be - but I reckon you could concatenate them somehow.) -Ed
For example, it would actually be nice if Zope provided an implementation of UUID (called GUID by Microsoft). A UUID is a guaranteed unique 64-bit value. It's used in COM, CORBA and other systems which need to have automatically generated guaranteed unique values. The nice thing about UUID's is that they are guaranteed to be unique network wide. In other words, you (supposedly) can't generate a UUID which would ever conflict with a UUID that I generated. It does this by an algorithm which takes into account things like time-of-date, network card address, etc. I don't have the details on the algorithm but I know it is publically available. It would probably best be implemented at the C level.
I just generated something like this for a product, in python. In order to guarantee uniqueness across a cluster of servers, I use the last octet of the ip address (assuming the cluster is all on the same subnet), the time, a call to randint(1,10000). If you really want to make things interesting, run that through the sha algorithm, and you get a nice 160 bit key rendered in hex. In obfuscated code, it looks like: sha(split(gethostbyname(gethostname()),'.')[3]+str(int(time.time()-900000000))+str(randint(1,1000))).hexdigest() but you would do well to make it multiple lines of code... ;-) --sam -- -------------------------- Sam Gendler CTO, Impossible, Inc. 1222 State St. Suite 250 Santa Barbara, CA. 93101 p: 805-560-0508 f: 805-560-0608 c: 805-689-1191 e: sgendler@impossible.com
Thanks. I like the fact that it is unique across multiple servers. I wonder If their is a way to get a truely unique one generated by using the MAC address with time? Sam Gendler writes:
I just generated something like this for a product, in python. In order to guarantee uniqueness across a cluster of servers, I use the last octet of the ip address (assuming the cluster is all on the same subnet), the time, a call to randint(1,10000). If you really want to make things interesting, run that through the sha algorithm, and you get a nice 160 bit key rendered in hex.
In obfuscated code, it looks like:
sha(split(gethostbyname(gethostname()),'.')[3]+str(int(time.time()-900000000))+str(randint(1,1000))).hexdigest()
but you would do well to make it multiple lines of code... ;-)
--sam
-- -------------------------- Sam Gendler CTO, Impossible, Inc. 1222 State St. Suite 250 Santa Barbara, CA. 93101 p: 805-560-0508 f: 805-560-0608 c: 805-689-1191 e: sgendler@impossible.com
Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.
participants (5)
-
Andrew Edmondson -
James W. Howe -
Jason Spisak -
Pavlos Christoforou -
Sam Gendler