Hi... I've seen here and there several articles about a replacement of the ZODB storage via an SQL backend, to store every Zope objects. Actually, I would like to test with a PostgreSQL storage, but I can't find any link anywhere to these kind of informations. Any link, advise or experimentation reports would be greatly appreciated... Thanks, Thierry -- Linux every day, keeps Dr Watson away... http://gpc.tuxfamily.org -- http://www.ulthar.net
Thierry Florac wrote:
I've seen here and there several articles about a replacement of the ZODB storage via an SQL backend, to store every Zope objects. Actually, I would like to test with a PostgreSQL storage, but I can't find any link anywhere to these kind of informations.
Any link, advise or experimentation reports would be greatly appreciated...
Please be aware of the common misunderstanding that Python objects are in some way converted to sql tables in this way. What happens is that Python objects has their pickled representation stored in a text field in the relational db. So it is not so that you can pull out data from Zope via the db. regards Max M -- "Sorry I would Really Like To Help More On This Project, But Am To Busy Doing Paid Work On A Tight Deadline" Max M
On Thu, 2002-07-04 at 13:31, Max M wrote:
Thierry Florac wrote:
I've seen here and there several articles about a replacement of the ZODB storage via an SQL backend, to store every Zope objects. Actually, I would like to test with a PostgreSQL storage, but I can't find any link anywhere to these kind of informations.
Any link, advise or experimentation reports would be greatly appreciated...
Please be aware of the common misunderstanding that Python objects are in some way converted to sql tables in this way.
What happens is that Python objects has their pickled representation stored in a text field in the relational db. So it is not so that you can pull out data from Zope via the db.
Absolutely... I don't want to execute queries on Zope's database, but what I'd like to be able to do is to use an external storage, which could provide advanced functions such as : - replication (also available with ZEO), - partitionning, - handling of very high volumes, - and so on... In fact, I'm a little bit afraid by the ZODB FileStorage, and by it's potential ability to handle very big data files correctly... We have a lot of big files to publish and index, and our ZODB will probably have a size of 10 GB quite rapidly... Any advise ?? Thanks, Thierry
On Thursday 04 Jul 2002 3:57 pm, Thierry Florac wrote:
I don't want to execute queries on Zope's database, but what I'd like to be able to do is to use an external storage, which could provide advanced functions such as :
It looks like your concerns are the same as what prompted me to start on DirectoryStorage.....
- replication (also available with ZEO),
ZEO doesnt do replication today. It is planned, but for far in the future.
- partitionning,
of what, and why?
- handling of very high volumes,
As in file size, or throughput? FileStorage leads the throughput benchmarks, but even the slowest storage is not sufficiently slow to make a difference (IMO). All of today's production storages have serious problems when packing databases with many objects, because they all use an in-memory index of reachable objects. For high volume you need a non-versioning storage (or DirectoryStorage, when its ready)
In fact, I'm a little bit afraid by the ZODB FileStorage, and by it's potential ability to handle very big data files correctly... We have a lot of big files to publish and index, and our ZODB will probably have a size of 10 GB quite rapidly... Any advise ??
Ive never had a filestorage that big. If memory usage is linear (which I think is a reasonable assumption) then a FileStorage that big will need (roughly) 1G of ram for normal use, and 2-3G during packing. If thats not a problem; do it.
On Friday 12 Jul 2002 11:35 am, Magnus Heino wrote:
ZEO doesnt do replication today. It is planned, but for far in the future.
Really? I though that was the reason for the python spread wrapper developement that happened a while ago?
yes, for StandbyStorage. That will give some level of replication. rsync does too. Neither of these are in ZEO, and are missing some crucial features of the original ZEO replication proposal. Maybe that doesnt matter.
Thierry Florac wrote:
Hi...
I've seen here and there several articles about a replacement of the ZODB storage via an SQL backend, to store every Zope objects. Actually, I would like to test with a PostgreSQL storage, but I can't find any link anywhere to these kind of informations.
Any link, advise or experimentation reports would be greatly appreciated...
Dirstorage sound very interesting: It will store the pickles in several files and diredtories in the filesystem. Haven't tried it: http://dirstorage.sourceforge.net/ thomas
On Friday 12 Jul 2002 8:27 am, Thomas Guettler wrote:
Thierry Florac wrote:
Hi...
I've seen here and there several articles about a replacement of the ZODB storage via an SQL backend, to store every Zope objects. Actually, I would like to test with a PostgreSQL storage, but I can't find any link anywhere to these kind of informations.
Any link, advise or experimentation reports would be greatly appreciated...
Dirstorage sound very interesting: It will store the pickles in several files and diredtories in the filesystem. Haven't tried it:
Its still pre-alpha; Today I would only recommend trying it if you have time to spend fixing it. I am working up to an alpha release hopefully over the next few weeks.
participants (5)
-
Magnus Heino -
Max M -
Thierry Florac -
Thomas Guettler -
Toby Dickenson