On 12/04/2011 09:52 AM, Sareesh Sudhakaran wrote:

Thanks Niels. Just to clarify:
Does my particular instance fall under an OODBMS model or a RDBMS model (with ORM if necessary)?

Data modelling is a bit of an art and probably you could tackle your problem with any approach. I think the important is for you to figure out which model suits more your personality. No kidding.

I would personally start with the RDBMS approach considering only the information you provide. Also, you can easily use zope in combination with a RDBMS. When you read the book, beware that zope has been changing from a "through the web" approach, to a typical file system based approach, which is a loss, but it seems to be what suits the needs of the zope developers.

The approach I use is:

zpt page -> (one) Python Script -> (some) ZSQL Methods -> MySQL

The zpt defines the layout of the pages, the ZSQL Methods retrieve the data needed and the Python script massages the data to make it suitable for the ZPT page. Or the other way around, from user input in a form to storage in the database.

The advantage of the relational approach is that it is a very well understood model and although different people will still come to different models there are sufficient objective guidelines out there (e.g., the normalization rules, and then criteria for when to denormalise) to help you along. Furthermore, there are lots of people in db-related forums that can help you.

Also, RDBMS provides you with a "standard" query language, SQL, which plenty of systems and tools can use. In general, RDBMS gives you the safest approach to keep your data and not loose it when you need to migrate either the front-end or the back-end. This language is very powerful and can avoid you a lot of low level programming.

However, plenty of people can not deal well with SQL because it follows a paradigm so different from the classic imperative programming. With SQL, you specify the "what", with the other languages you specify the "how". The advantage of the "what" is that you can do a lot of data processing in a few lines of code. The problem with the "what" is that because you don't know the "how" of it, you feel you don't have control and you are led to say the language is "obscure" or "unreadable".

However, even if you are not comfortable with the "what" (you have to try to know), you can still rely on an library like SQLAlchemy to keep you a bit in the comfort zone of the "how". So instead of learning SQL, you need to learn the API of a specific library. Your choice. I recommend the first by far.

The real main issue with Relational is that it is a highly structured data model. It allows you to keep high quality data but if you don't get it right soon enough in the development cycle, some later changes can have a huge impact in the application, requiring rewrites. Furthermore, it works the best when you have many objects with the same properties. If you have many entities all different from each other (the tools in your case, maybe), then maybe a OODBMS might be better. But here, there is no standard language, or standard whatever. Perosnally, I would avoid as much as possible to put data in something like ZODB (I use it merely to store and manage the application).

The problem with your specific case is that it does not seem to be a typical case of books and authors, which might be a risk for someone without experience. The issue "Tool A might have only three fixed rpms - 100, 200 and 500, but Tool B might have rpms from 20 to 2000", is indeed tricky. I suspect in general the needs of your system will be too specific to be able to rely only on SQL queries. You would need to put a lot of intelligence in the data (really highly structured) and it might become unmanageable or not scalable.

I guess you will need to put a lot of intelligence in the Python Script. So, the ZSQL retrieves the relevant tool data and then makes the tool choice. The knowledge of the meaning of the attributes is maintained in programming.

I should say I am not totally sure the Python Script is the best approach here, in terms of management facilities. But Python is surely a very good language due to its readability. However, you might need to use External methods or a more typical file-system based Python approach.

Or maybe you actually need to create a Domain Specific Language to encode the intelligence needed for your tool selection process. If your python code becomes repetitive, with patterns showing up, then a DSL might be a good approach, but this might be too much engineering for you at this stage. It looks like you are in a typical CIM scenario and I remember handling a similar problem 20 years ago. I vaguely remember at that time to use custom graph structures in C and the the intelligence was coded in Lisp/Scheme. So, there is a big range of solutions to try out :)

If you have time, then the simple approach

zpt page -> (one) Python Script -> (some) ZSQL Methods -> MySQL database

might be a good starting point. You should not need to spend much time to implement a prototype using this approach. In the worse case scenario it helps you understand better your problem and what could be a better approach with little investment. Essentially you try the classical approach and if it does not work well, either you are doing something wrong, or you have a better understanding of your needs and where to go.

Good luck,
Fernando