[Zope] Re: audio order

[Nate Aune]

> I am most interested and will check out your efforts.

You are welcome to join as a member on the Plone4Artists site. There is also
a newsletter and mailing list to make it easier to stay in touch with the
developments. http://plone4artists.jazkarta.com
 
> my wiki address (when it is up) is 192.117.110.160:8080/nigunim/thewiki

I checked out the wiki, and it looks like you have a very ambitious project
indeed!

> My problem is that the audio is only one aspect of the picture and I
> need to keep the metadata in sync with, notation and information.
> 
> I am currently leaning towards keeping the metadata outside the actual
> ogg files and having it added when the playlists are created.

Why not keep the metadata in the OGG files themselves. It's in the OGG spec,
so you might as well take advantage of it!

> First people will search the metadata and then reaquest to hear the
> songs.

You can store the metadata in the Zope database as well, so that it is
searchable.

> I unfortuately got all my reponses off list.

And I can see that this message you sent to me was also sent privately
instead of to the list. You should really use "Reply to All" in your email
client, so that BOTH the list and the recipient get a copy of your msg.

> Some suggested a way to use the Zope db for large files.

Yes, Sergey Volobuev mentioned that Zope 2.7.1 will have some improvements
in this area.

> I am still so new to zope that I am still unsure what will provide me
> with the best solution.

You have some pretty special requirements in that you need to keep metadata
in sync with audio files and notation files. I would strongly recommend that
you have a look at Archetypes. It gives you a lot of flexibility in this
area. 

You could for example extend the ATAudio product to be referenceable to a
ATNotation type. In this way, you can upload your Audio files with all
metadata intact, and then "link" this audio file to a corresponding Notation
file that you have also uploaded.

Using WebDAV, you can get these files into your site very quickly by
drag-n-drop, and you won't have to enter all the metadata into your site
because it will already be embedded in the OGG files.

As for the notation, I know that Andrew Kuchling from the docutils project
wrote a directive for the ABC music language.
http://sourceforge.net/mailarchive/message.php?msg_id=7625994

Michael Hauser is working on getting this to work in a Zope/Plone site,
which means you could upload a notation file, and it would be rendered in
the site as a PNG image. It looks like the only dependencies besides
docutils are abcm2ps, Ghostscript and PIL.

Now to reply to your comments...

>>> Have you seen the audio products? In what way will they help me?
>> 
>> CMFAudio, written by Marc Bowery is fine for small MP3 files,
> 
> I have small ogg files but lots of them

If they are small and Zope 2.7.1 is able to stream them sufficiently, you
may not even want to bother with storing them on the file system. There are
some advantages to having all your data in the ZODB.

>> want to workflow, edit and
>> search for them.
>> 
> but not for the general public only a small group of developers

You can define these permissions however you want.
 
>> The cool thing is that we've implemented an ID3Storage layer which acts
>> as an interface for reading/writing the ID3 tags from/to the MP3 files
>> This was done using pyid3lib (http://pyid3lib.sourceforge.net).
>> 
>> This saves you from doing a lot of data-entry, especially if your audio
>> files already have metadata embedded in them.
>> 
> In either case the metadata is yet to be added. The question is whether
> it should reside withing the ogg files or the notation or ?.

Well, that depends on your use cases. You could also make an abstract "Song"
type which contains ONLY metadata but has references to the actual audio and
notation files. 

Or if you don't want to bother with Archetypes, you could make this "Song"
type folderish, so it could be a container for the audio and notation files.
In my opinion, this would be the simplest way to do it.
 
> I also have the language issue where I need to serve metadata in
> multiple languages (hebrew and english)

Ugh.. Then that complicates things a bit. Currently, the only solution that
I know of is I18NLayer, but it gets very messy. If you can wait a month,
there will be a new product called LinguaPlone which makes handling
multilingual content much easier. However, I think it requires that your
content types are Archetypes-based since it relies on the new Archetypes
references engine.

>> Unless you don't even use Zope to serve the file, but simply provide a
>> pointer to the URL where another 3rd party tool (such as Apache,
>> Icecast, Shoutcast, Edna, etc.) actually serves up the file.
>> 
> sounds interesting

But again, if your OGG files are small, you may not want to complicate
matters with 3rd party streaming tools. On the other hand, your solution
will be more scalable if you store your files outside of the ZODB because
then you can use any streaming server and easily change to a better one if
the need arises.

>>> Well I think the largest ogg file will be 9 megs. Certainly not 30- 50
>>> meg.

9 Meg is not what I would consider "small". If you are going to have many of
these large files, I would definitely consider storing them on the file
system instead of the ZODB. Or else you will quickly find yourself with ZODB
bloat.

>> This is how we do it with ATAudio. When you click on the 'Play' icon, it
>> auto-generates an .m3u file which points to the MP3 file being served
>> from your Apache server.

> Also ogg?

I answered this in the last mail. No ogg support yet, but it's coming...

>> Jens Klein (jensens) is working on providing support for OGG files,
>> using the pyOGG libraries. http://www.andrewchatham.com/pyogg/

>>> I am not planning to run apache in addition to zope.  I haven't looked
>>> at ZEO and I will do that soon I guess.
>> 
>> Why aren't you planning to run Apache? It's pretty foolish to run Zope
>> 'in the wild' without Apache in front of it. Also, Apache (esp. with
>> mod_mp3 or Apache::MP3) makes an excellent server for serving up those
>> large MP3/OGG files.
> 
> Well I have seen those who use it in the wild as you say and just use a
> lot of cache.

Well, the Zope caching is not very robust and eats up a lot of RAM. You can
reduce the hit on your Zope server a lot by using Apache or Squid caching in
front of Zope. Also, using Apache is not only for performance reasons, but
also for security reasons.

> I may use apache etc. but for development I am working locally. I am
> trying presently to avoid learning another thing, at least until I
> understand Zope enough to get off the ground.

It's fine while you're developing the site, but once you launch it, you
should really use Apache. It's not very difficult to set up. Just a few
lines in your Apache's httpd.conf file (VirtualHost directive).

>>> If anyone is willing/interested in giving some feedback about how to
>>> best use zope for my project, My Wiki explains the project details and
>>> design issues.
>> 
>> What is the URL to your wiki?  Mine is
>> http://plone4artists.jazkarta.com/development/Audio
>> 
>> The ATAudio product is still a little rough around the edges, but you
>> are welcome to join in the development efforts.
> I will take a look thanks.
>> 
>> The bugs/issues collector for ATAudio describes the known issues and
>> features which are planned. http://tinyurl.com/37u7k
>> 
>> You can also read the TODO.txt if you grab the product from the
>> collective CVS. http://sf.net/projects/collective
>> 
>> I'm looking forward to this talk in a few weeks at the EuroPython
>> conference: "The Railroad Project: Managing Large files Around Zope"
>> http://www.europython.org/conferences/epc2004/info/talks/zope/kitblake01
>> 
>> Nate
> 
> Thanks
> Aaron
> 
>