[Zope] large images to a database via zope.

[ghaley@mail.venaca.com]

	We are serving a variety of binary formats.  jpeg, gifs, flash
files, etc.  The range of binary file size can be from a few k to over 100
MB.  The decision was made prior to my coming on board to store everything
in a db -- to avoid two problems:  growing the data.fs file too large, and
to conserve inode entries.  

	We also chose MySQL as we do not anticipate a large number of data
writes but a huge number of reads, so the rollback, commit, type options
of other dbs was not as important, and the speed of MySQL retrieval was a
big bonus.  But, MySQL barfs when trying to insert a record with more than
about 15 or 16 meg.  We suspect that the insert happens faster than MySQL
can accommodate.  I wrote a perl script that will feed in a buffer of
about a meg at a time, and we were able to get it to work several times,
but even that was too much, and, again MySQL crashed on large inserts.

	So, the question has turned, in this thread at least, into one of
the efficiency and integrity of file storage.  The file system approach
looks to be the best recommendation, while using the database to store the
metadata and address of the object.  Though, I like the discussion about
the capacity of the data.fs.  

	Thanks for your question.  Sorting through the thread, it does
appear to be rather protean, but it is more of an evolution; and it helps
to clarify my own thinking.

	Thanks, also, to various people who have responded.  

	ciao!
	greg.

	Gregory Haley
	DBA/Web Programmer
	Venaca, LLC.

	p.s., Please note the correct spelling of my name.
> 
> Gary - I'm confused about what you want to do. The subject says large 
> images and your original message sought a method to accommodate images > 
> 16MB. And you need to store a million of them? What's your average image 
> size? 1MB, 5MB? A million row db that is 1,000 to 5,000 GB is going to have 
> problems of its own in addition to the file system that you'll need.
> 
> --
> Dennis Nichols
> nichols@tradingconnections.com