To address comments from several people:
Andreas: Have you thought about storing the data inside the ZODB?
I had thought of it but wasn't sure if it would be appropriate. Is it possible to assign arbitrary metadata to a file in the ZODB?
Tom P: It's certainly feasible to insert data into a relational database using DTML. If the data is truly binary, it's probably not very well suited for web forms, where some characters are not allowed without escaping.
Wouldn't using <type=file> and <enctype="multipart/form-data"> get around the need to escape certain things? I'm looking to build something that is simple to use and preferably wouldn't need any special applications for the client.
It also would depend on what you want to do with the data, and whether there is any metadata that needs to be attached to the data. For example, it's one thing to store a blob in a table, and another to find the right data later on. Of course, you have the same problem when you store files. You may find that it's better to store the files in the file system and the metadata in a database, either the ZODB or in a relational database.
There will be metadata (what equipment was used, what settings were, who did it, etc) which needs to be searchable and is appropriate for a relational database. Now what to do with the data itself? File names are similar and I'd probably have to hash the timestamp or something in order to guarantee that they're not the same in a directory, making it essentially meaningless without the database. Plus they're small and the hierarchy could get ugly in a hurry. So I thought I might go ahead and store them as records in the database. It seemed simple in principal and more intuitive since whoever inherits this system will likely not be a professional IT person. Just as I am not one.
I'm setting up a database to store data file for a scientific research group. I'm thinking of writing the data into a MySQL table rather than storing them externally in the filesystem. The files are generally small but the number of them will grow quickly.
Are there opinions as to whether this is a good or bad idea?
I know there's overhead and a (severe?) throughput penalty, but I was thinking this would be simple to manage (and back up) as well as being easy to search. The load on the server won't be much anyway.
Can collecting binary data in a web form and writing it to MySQL be done totally in DTML? I've found some past threads in the list dealing with this kind of thing (and an old how-to) but there doesn't seem to be a consensus on a "right" way to implement it.