[Zope-dev] how bad are per-request-write-transactions
Paul Everitt
paul@zope.com
Wed, 17 Apr 2002 07:54:04 -0400
I don't agree that high write is always forbidden. I think there are
plenty of cases where this can work. It simply becomes unworkable much
sooner than other data systems (e.g. a relational database or FS-based
solution).
For instance, think about bloat for a second. Let's be crazy and say it
takes 100 bytes to store an integer representing a count. Let's say you
write once a second. That's under 7Mb a day (per counter). Combined
with hourly packing, that might be well within limits.
Let's take the next step and say that you can live with a little
volatility in the data. You write an object that caches ten seconds
worth of writes. Whenever a write comes in at the over-ten-second mark,
you write the _v_ attribute to the persistent attribute. There's an
order of magnitude improvement.
Finally, you store all your counters in a non-versioned storage. Now
you have *no* bloat problem. :^)
Regarding performance, maybe his application isn't doing 50
requests/second and he'd be willing to trade the slight performance hit
and bloat for a decrease in system complexity.
All of the above has downsides as well. My point, though, is that we
shouldn't automatically dismiss the zodb as inappropriate for *all*
high-write situations. In fact, with Andreas and Matt Hamilton's
TextIndexNG, you might even be able to write to catalogued applications
at a faster rate than one document per minute. :^)
--Paul
Casey Duncan wrote:
> This will kill performance, especially concurrent use of the site. It
> will also cause large amounts of database bloat. Do you need real time
> numbers, or is a delay (such as 24 hours) acceptable?
>
> If you can stand a delay, another approach would be to write a script
> which scans the z2.log file (or another log that you generate on page
> hits) each night and in a single transaction updates a counter on each
> object hit.
>
> If you use the z2.log, no additional writing is needed to the FS, and
> you get the benefit of easy access to the counts directly from the
> objects, without degrading performance or db bloat.
>
> -Casey
>
> Ivo van der Wijk wrote:
>
>> Hi,
>>
>> How bad are per-request transactions in a non-ZEO environment? I.e.
>> each request on a folder or its subobjects will cause a write transaction
>> (somewhat like a non-fs counter, but worse as it happens for all
>> subobjects)
>>
>> And if this is really bad, are there any workarounds except for writing
>> to the filesystem?
>>
>> Cheers
>>
>> Ivo
>>
>>
>
>
>
>
> _______________________________________________
> Zope-Dev maillist - Zope-Dev@zope.org
> http://lists.zope.org/mailman/listinfo/zope-dev
> ** No cross posts or HTML encoding! **
> (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce
> http://lists.zope.org/mailman/listinfo/zope )