I don't agree that high write is always forbidden. I think there are plenty of cases where this can work. It simply becomes unworkable much sooner than other data systems (e.g. a relational database or FS-based solution). For instance, think about bloat for a second. Let's be crazy and say it takes 100 bytes to store an integer representing a count. Let's say you write once a second. That's under 7Mb a day (per counter). Combined with hourly packing, that might be well within limits. Let's take the next step and say that you can live with a little volatility in the data. You write an object that caches ten seconds worth of writes. Whenever a write comes in at the over-ten-second mark, you write the _v_ attribute to the persistent attribute. There's an order of magnitude improvement. Finally, you store all your counters in a non-versioned storage. Now you have *no* bloat problem. :^) Regarding performance, maybe his application isn't doing 50 requests/second and he'd be willing to trade the slight performance hit and bloat for a decrease in system complexity. All of the above has downsides as well. My point, though, is that we shouldn't automatically dismiss the zodb as inappropriate for *all* high-write situations. In fact, with Andreas and Matt Hamilton's TextIndexNG, you might even be able to write to catalogued applications at a faster rate than one document per minute. :^) --Paul Casey Duncan wrote:
This will kill performance, especially concurrent use of the site. It will also cause large amounts of database bloat. Do you need real time numbers, or is a delay (such as 24 hours) acceptable?
If you can stand a delay, another approach would be to write a script which scans the z2.log file (or another log that you generate on page hits) each night and in a single transaction updates a counter on each object hit.
If you use the z2.log, no additional writing is needed to the FS, and you get the benefit of easy access to the counts directly from the objects, without degrading performance or db bloat.
-Casey
Ivo van der Wijk wrote:
Hi,
How bad are per-request transactions in a non-ZEO environment? I.e. each request on a folder or its subobjects will cause a write transaction (somewhat like a non-fs counter, but worse as it happens for all subobjects)
And if this is really bad, are there any workarounds except for writing to the filesystem?
Cheers
Ivo
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )