Hi John and FernandoIntuitively I feel that my project fits in with an Object DB, which is why I have spent a lot of time trying to understand its methodology. But now I'm more confused than ever. To answer your questions - Only an expert can perform data entry for the tools, obviously. Since I have no clue how to enter data into mySQL or ZODB I have no idea what's in store for me. The ultimate goal is for the engine to become an expert system that is capable of taking inputs, learning from mistakes, scraping data from reports and from the internet, and become better and better at its predictions. The logic engine will get better and more complex as more tools are added - but once all the 'current' tools are added, the system must be able to go 'solo', with an accuracy that reaches six sigma levels. As Fernando has suggested, I will have to go with C/C++, or even my own language in the future, but that day is far off. I plan to start with 50 higher-level tools and try to build a prototype. The front-end is really very simple. The choices available for the start parameters and end parameters are pretty basic, and most of the logic happens under the hood - probably at the back end if possible. Once I've built this prototype, I will know whether mqSQL can handle scaling up (in my particular case). If not, I can confidently invest in ZODB. If I can make my position any clearer in order to get a conclusive answer either way - I'll be glad to. Thanks again for your help. I really am indebted to you guys.sareesh
From: zope-request@zope.org Subject: Zope Digest, Vol 91, Issue 2 To: zope@zope.org Date: Mon, 5 Dec 2011 12:00:02 +0000
Send Zope mailing list submissions to zope@zope.org
To subscribe or unsubscribe via the World Wide Web, visit https://mail.zope.org/mailman/listinfo/zope or, via email, send a message with subject or body 'help' to zope-request@zope.org
You can reach the person managing the list at zope-owner@zope.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of Zope digest..."
Today's Topics:
1. Re: Help in deciding approach to Web App (Fernando Martins) 2. Re: Help in deciding approach to Web App (Sareesh Sudhakaran) 3. Re: Help in deciding approach to Web App (Fernando Martins) 4. Re: Help in deciding approach to Web App (John Schinnerer) 5. Re: Help in deciding approach to Web App (Sareesh Sudhakaran) 6. Re: Help in deciding approach to Web App (Fernando Martins) 7. Re: Help in deciding approach to Web App (John Schinnerer) 8. Re: Help in deciding approach to Web App (Niels Dettenbach) 9. Re: Help in deciding approach to Web App (Fernando) 10. Re: Help in deciding approach to Web App (Sareesh Sudhakaran) 11. Re: Help in deciding approach to Web App (Niels Dettenbach)
----------------------------------------------------------------------
Message: 1 Date: Sun, 04 Dec 2011 13:28:36 +0100 From: Fernando Martins <fernando@cmartins.nl> To: Sareesh Sudhakaran <aysand@hotmail.com> Cc: zope@zope.org Subject: Re: [Zope] Help in deciding approach to Web App Message-ID: <4EDB6774.1050603@cmartins.nl> Content-Type: text/plain; charset="iso-8859-1"; Format="flowed"
On 12/04/2011 09:52 AM, Sareesh Sudhakaran wrote:
Thanks Niels. Just to clarify: Does my particular instance fall under an OODBMS model or a RDBMS model (with ORM if necessary)?
Data modelling is a bit of an art and probably you could tackle your problem with any approach. I think the important is for you to figure out which model suits more your personality. No kidding.
I would personally start with the RDBMS approach considering only the information you provide. Also, you can easily use zope in combination with a RDBMS. When you read the book, beware that zope has been changing from a "through the web" approach, to a typical file system based approach, which is a loss, but it seems to be what suits the needs of the zope developers.
The approach I use is:
zpt page -> (one) Python Script -> (some) ZSQL Methods -> MySQL
The zpt defines the layout of the pages, the ZSQL Methods retrieve the data needed and the Python script massages the data to make it suitable for the ZPT page. Or the other way around, from user input in a form to storage in the database.
The advantage of the relational approach is that it is a very well understood model and although different people will still come to different models there are sufficient objective guidelines out there (e.g., the normalization rules, and then criteria for when to denormalise) to help you along. Furthermore, there are lots of people in db-related forums that can help you.
Also, RDBMS provides you with a "standard" query language, SQL, which plenty of systems and tools can use. In general, RDBMS gives you the safest approach to keep your data and not loose it when you need to migrate either the front-end or the back-end. This language is very powerful and can avoid you a lot of low level programming.
However, plenty of people can not deal well with SQL because it follows a paradigm so different from the classic imperative programming. With SQL, you specify the "what", with the other languages you specify the "how". The advantage of the "what" is that you can do a lot of data processing in a few lines of code. The problem with the "what" is that because you don't know the "how" of it, you feel you don't have control and you are led to say the language is "obscure" or "unreadable".
However, even if you are not comfortable with the "what" (you have to try to know), you can still rely on an library like SQLAlchemy to keep you a bit in the comfort zone of the "how". So instead of learning SQL, you need to learn the API of a specific library. Your choice. I recommend the first by far.
The real main issue with Relational is that it is a highly structured data model. It allows you to keep high quality data but if you don't get it right soon enough in the development cycle, some later changes can have a huge impact in the application, requiring rewrites. Furthermore, it works the best when you have many objects with the same properties. If you have many entities all different from each other (the tools in your case, maybe), then maybe a OODBMS might be better. But here, there is no standard language, or standard whatever. Perosnally, I would avoid as much as possible to put data in something like ZODB (I use it merely to store and manage the application).
The problem with your specific case is that it does not seem to be a typical case of books and authors, which might be a risk for someone without experience. The issue "Tool A might have only three fixed rpms - 100, 200 and 500, but Tool B might have rpms from 20 to 2000", is indeed tricky. I suspect in general the needs of your system will be too specific to be able to rely only on SQL queries. You would need to put a lot of intelligence in the data (really highly structured) and it might become unmanageable or not scalable.
I guess you will need to put a lot of intelligence in the Python Script. So, the ZSQL retrieves the relevant tool data and then makes the tool choice. The knowledge of the meaning of the attributes is maintained in programming.
I should say I am not totally sure the Python Script is the best approach here, in terms of management facilities. But Python is surely a very good language due to its readability. However, you might need to use External methods or a more typical file-system based Python approach.
Or maybe you actually need to create a Domain Specific Language to encode the intelligence needed for your tool selection process. If your python code becomes repetitive, with patterns showing up, then a DSL might be a good approach, but this might be too much engineering for you at this stage. It looks like you are in a typical CIM scenario and I remember handling a similar problem 20 years ago. I vaguely remember at that time to use custom graph structures in C and the the intelligence was coded in Lisp/Scheme. So, there is a big range of solutions to try out :)
If you have time, then the simple approach
zpt page -> (one) Python Script -> (some) ZSQL Methods -> MySQL database
might be a good starting point. You should not need to spend much time to implement a prototype using this approach. In the worse case scenario it helps you understand better your problem and what could be a better approach with little investment. Essentially you try the classical approach and if it does not work well, either you are doing something wrong, or you have a better understanding of your needs and where to go.
Good luck, Fernando