Help in deciding approach to Web App
I have a personal project - a web application I wanted to develop - but I'm confused on which route to take. I am not under any time constraint. About the App:The best I can describe it is as a kind of expert system (but not AI) that needs to find the best workflow for a process, given a set of initial and final parameters. E.g. a 'capsule' of data must pass through many 'tools' or 'environments' to reach a desired output - something like a very complicated car wash.Let's say there are many tools that can be used at various stages in the process. I have estimated there are at least 500 tools as of now, and it is bound to grow in the future as newer tools are introduced. Existing tools will also have version updates.Each tool, on average, has at least 100 properties that define the tool. Some of them have as high as 1000 unique properties. Some of these tools are linked to each other - e.g if one tool is selected, there are only n tools that can correspond to it for the next step in the process. I also have the problem of 'matching' the tools for analysis. E.g. Tool A might have only three fixed rpms - 100, 200 and 500, but Tool B might have rpms from 20 to 2000. I'm not sure how I can construct a database without spelling out each number, as in the example above.The total number of tools needed for the process can be defined at the beginning, however, it will change as the application becomes more complex in the future. I plan to address every contingency in the process. The idea is - if the user inputs the initial parameters and the desired outcome (another set of parameters), the app must find the 'best' path - sort of like a decision tree. The best path can be the fastest, cheapest, etc. I would like the user to choose what is best for him/her.Unfortunately, parameters might change, relationships might change (but not regularly) - the 'rules' I will be using might be revised for better accuracy in prediction.I also need to track each user's path and solutions' for future reference (but no personal details except username and email address for logging in). Maybe when the app is up and running, I'd like to make it more democratic, with users contributing to refining the logic/rules involved.If possible, I would also like the app to output a graphical flowchart at the end showing the workflow with all tools grouped in an easy to understand layout. My questions:Will the app be better served with a relational DB like mySQL or an Object database? After a lot of research I've guessed that my particular case might be better served with Python and Zope/ZODB. But I might be wrong? Maybe PHP+mySQL or Django is a better fit?Can anyone provide general advice on how to go about beginning such a project in ZOPE. Which is the best place to start learning for a newbie? Can anyone recommend a good shared hosting provider that supports Zope fully but is not expensive? Is there a module or app that is open source that I can use to output a graphical flowchart based on the results, or will I be better served programming it from scratch with Python?I would appreciate any help in getting started. Thank you in advance. I have tried most online forums but have not good any productive answers. Most of the answers I got were pro-PHP+mySQL. Adam
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - From reading: your database model appears pretty much relational. Where would you take advantages from using Zope as framework? This sounds like a task for Pyramid + RDBMS or a graph DB. - -aj Sareesh Sudhakaran wrote:
I have a personal project - a web application I wanted to develop - but I'm confused on which route to take. I am not under any time constraint.
*About the App:*
The best I can describe it is as a kind of expert system (but not AI) that needs to find the best workflow for a process, given a set of initial and final parameters. E.g. a 'capsule' of data must pass through many 'tools' or 'environments' to reach a desired output - something like a very complicated car wash.
Let's say there are many tools that can be used at various stages in the process. I have estimated there are at least 500 tools as of now, and it is bound to grow in the future as newer tools are introduced. Existing tools will also have version updates.
Each tool, on average, has at least 100 properties that define the tool. Some of them have as high as 1000 unique properties. Some of these tools are linked to each other - e.g if one tool is selected, there are only n tools that can correspond to it for the next step in the process. I also have the problem of 'matching' the tools for analysis. E.g. Tool A might have only three fixed rpms - 100, 200 and 500, but Tool B might have rpms from 20 to 2000. I'm not sure how I can construct a database without spelling out each number, as in the example above.
The total number of tools needed for the process can be defined at the beginning, however, it will change as the application becomes more complex in the future. I plan to address every contingency in the process. The idea is - if the user inputs the initial parameters and the desired outcome (another set of parameters), the app must find the 'best' path - sort of like a decision tree. The best path can be the fastest, cheapest, etc. I would like the user to choose what is best for him/her.
Unfortunately, parameters might change, relationships might change (but not regularly) - the 'rules' I will be using might be revised for better accuracy in prediction.
I also need to track each user's path and solutions' for future reference (but no personal details except username and email address for logging in). Maybe when the app is up and running, I'd like to make it more democratic, with users contributing to refining the logic/rules involved.
If possible, I would also like the app to output a graphical flowchart at the end showing the workflow with all tools grouped in an easy to understand layout.
*My questions:*
1. Will the app be better served with a relational DB like mySQL or an Object database? After a lot of research I've guessed that my particular case might be better served with Python and Zope/ZODB. But I might be wrong? Maybe PHP+mySQL or Django is a better fit? 2. Can anyone provide general advice on how to go about beginning such a project in ZOPE. Which is the best place to start learning for a newbie? 3. Can anyone recommend a good shared hosting provider that supports Zope fully but is not expensive? 4. Is there a module or app that is open source that I can use to output a graphical flowchart based on the results, or will I be better served programming it from scratch with Python?
I would appreciate any help in getting started. Thank you in advance. I have tried most online forums but have not good any productive answers. Most of the answers I got were pro-PHP+mySQL.
Adam
_______________________________________________ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
- -- ZOPYX Limited | zopyx group Charlottenstr. 37/1 | The full-service network for Zope & Plone D-72070 Tübingen | Produce & Publish www.zopyx.com | www.produce-and-publish.com - ------------------------------------------------------------------------ E-Publishing, Python, Zope & Plone development, Consulting -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQGUBAEBAgAGBQJO2yPrAAoJEADcfz7u4AZjnJULviZSrj8nLwzeqjgxGT+d9/aU 1x0bj/E3zuPFeKtWkevp26K4OiCe/yw7PmgVXh1xBbfLhFzUcet9Ubdu0eIjfE/8 dC8hayQf4fR8KD3J06CbkMAEJsJur3gTPoFxzDWd7S9ybwuFjb3E14AhiQGPpdYN DVwE6U3t240Wj7ebNTblASQnvI2sfzr9r2tYVqZc8O+SMyROL1oIs56RNofT4pz5 p9OXiYlnHmn1keCbEKnYk1e2zTU7kFJdGQz/Uy+yV4QTiU7nMwhxWCx1gBKxVgNk XseqPdWKl9epz/h1Pb0qPTvK6PXl46Cj/0Bls/XU6yHDDpB49SqMXai2i6VIAzdL lik6JwSPVxBv24eTngrisx2qvhl1ln63EM+AWwky7jUgNl0cwALNdR3Gj+zDVioc ZWAZfWcuI3TFKYpwJiwzNgT0DSbOLvlM/09xXPsxmE+rfXT8arTxMSEgaQS60LqJ gil8bt+flbzjAJ3GEwza2KmoLpS/zxc= =gVTz -----END PGP SIGNATURE-----
Thanks AJ. I assumed an OODBMS would be the right choice because of the object nature of my 'tools'. Of course, it was an assumption. Instead of Pyramid+RDBMS, can I use PHP+mySQL (my current hosting provider supports this).If ZODB isn't for my project, then would a GraphDB help? I have no idea on where to start with Graph DB - I've read the wikis and it's made me more confused. -ss
Date: Sun, 4 Dec 2011 08:40:28 +0100 From: lists@zopyx.com To: aysand@hotmail.com CC: zope@zope.org Subject: Re: [Zope] Help in deciding approach to Web App
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
- From reading: your database model appears pretty much relational. Where would you take advantages from using Zope as framework? This sounds like a task for Pyramid + RDBMS or a graph DB.
- -aj
Sareesh Sudhakaran wrote:
I have a personal project - a web application I wanted to develop - but I'm confused on which route to take. I am not under any time constraint.
*About the App:*
The best I can describe it is as a kind of expert system (but not AI) that needs to find the best workflow for a process, given a set of initial and final parameters. E.g. a 'capsule' of data must pass through many 'tools' or 'environments' to reach a desired output - something like a very complicated car wash.
Let's say there are many tools that can be used at various stages in the process. I have estimated there are at least 500 tools as of now, and it is bound to grow in the future as newer tools are introduced. Existing tools will also have version updates.
Each tool, on average, has at least 100 properties that define the tool. Some of them have as high as 1000 unique properties. Some of these tools are linked to each other - e.g if one tool is selected, there are only n tools that can correspond to it for the next step in the process. I also have the problem of 'matching' the tools for analysis. E.g. Tool A might have only three fixed rpms - 100, 200 and 500, but Tool B might have rpms from 20 to 2000. I'm not sure how I can construct a database without spelling out each number, as in the example above.
The total number of tools needed for the process can be defined at the beginning, however, it will change as the application becomes more complex in the future. I plan to address every contingency in the process. The idea is - if the user inputs the initial parameters and the desired outcome (another set of parameters), the app must find the 'best' path - sort of like a decision tree. The best path can be the fastest, cheapest, etc. I would like the user to choose what is best for him/her.
Unfortunately, parameters might change, relationships might change (but not regularly) - the 'rules' I will be using might be revised for better accuracy in prediction.
I also need to track each user's path and solutions' for future reference (but no personal details except username and email address for logging in). Maybe when the app is up and running, I'd like to make it more democratic, with users contributing to refining the logic/rules involved.
If possible, I would also like the app to output a graphical flowchart at the end showing the workflow with all tools grouped in an easy to understand layout.
*My questions:*
1. Will the app be better served with a relational DB like mySQL or an Object database? After a lot of research I've guessed that my particular case might be better served with Python and Zope/ZODB. But I might be wrong? Maybe PHP+mySQL or Django is a better fit? 2. Can anyone provide general advice on how to go about beginning such a project in ZOPE. Which is the best place to start learning for a newbie? 3. Can anyone recommend a good shared hosting provider that supports Zope fully but is not expensive? 4. Is there a module or app that is open source that I can use to output a graphical flowchart based on the results, or will I be better served programming it from scratch with Python?
I would appreciate any help in getting started. Thank you in advance. I have tried most online forums but have not good any productive answers. Most of the answers I got were pro-PHP+mySQL.
Adam
_______________________________________________ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
- -- ZOPYX Limited | zopyx group Charlottenstr. 37/1 | The full-service network for Zope & Plone D-72070 Tübingen | Produce & Publish www.zopyx.com | www.produce-and-publish.com - ------------------------------------------------------------------------ E-Publishing, Python, Zope & Plone development, Consulting
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQGUBAEBAgAGBQJO2yPrAAoJEADcfz7u4AZjnJULviZSrj8nLwzeqjgxGT+d9/aU 1x0bj/E3zuPFeKtWkevp26K4OiCe/yw7PmgVXh1xBbfLhFzUcet9Ubdu0eIjfE/8 dC8hayQf4fR8KD3J06CbkMAEJsJur3gTPoFxzDWd7S9ybwuFjb3E14AhiQGPpdYN DVwE6U3t240Wj7ebNTblASQnvI2sfzr9r2tYVqZc8O+SMyROL1oIs56RNofT4pz5 p9OXiYlnHmn1keCbEKnYk1e2zTU7kFJdGQz/Uy+yV4QTiU7nMwhxWCx1gBKxVgNk XseqPdWKl9epz/h1Pb0qPTvK6PXl46Cj/0Bls/XU6yHDDpB49SqMXai2i6VIAzdL lik6JwSPVxBv24eTngrisx2qvhl1ln63EM+AWwky7jUgNl0cwALNdR3Gj+zDVioc ZWAZfWcuI3TFKYpwJiwzNgT0DSbOLvlM/09xXPsxmE+rfXT8arTxMSEgaQS60LqJ gil8bt+flbzjAJ3GEwza2KmoLpS/zxc= =gVTz -----END PGP SIGNATURE-----
Aloha, Very briefly, from what you describe, it looks like you are dealing with large numbers of complex objects (your 'tools') that interact with each other - and with some other elements that are going through this workflow process? And a context for the process? ...not sure of that part... In any case, this looks (to me anyhow) like a very object-oriented system you are modeling so an object oriented approach and language would seem most suitable. Python is one reasonable language option; zope for the web publishing aspect of the project would fit well with that. I haven't worked with other OO languages enough to usefully compare. It also sounds to me like the web publishing is the lesser part of this...? That is, the workflow of interacting tools is the real 'app' here, a process which does not all get shown somehow on a web page...or does it? Or just the outcomes? LAMP platform is so common and cheap that it's all a lot of people know about. It can be used well, and, it is all too easy to make horrible hacks and Frankestein monster apps in PHP/MySQL. Meaning, a highly structured (yet powerful) OO programming language will support you in avoiding that. Also, for development (or just to explore if python+zope might meet your needs) you don't need any hosting, you can install python+zope on your development system and try it out. best wishes... John S. On 12/03/2011 07:12 PM, Sareesh Sudhakaran wrote:
I have a personal project - a web application I wanted to develop - but I'm confused on which route to take. I am not under any time constraint.
*About the App:*
The best I can describe it is as a kind of expert system (but not AI) that needs to find the best workflow for a process, given a set of initial and final parameters. E.g. a 'capsule' of data must pass through many 'tools' or 'environments' to reach a desired output - something like a very complicated car wash.
Let's say there are many tools that can be used at various stages in the process. I have estimated there are at least 500 tools as of now, and it is bound to grow in the future as newer tools are introduced. Existing tools will also have version updates.
Each tool, on average, has at least 100 properties that define the tool. Some of them have as high as 1000 unique properties. Some of these tools are linked to each other - e.g if one tool is selected, there are only n tools that can correspond to it for the next step in the process. I also have the problem of 'matching' the tools for analysis. E.g. Tool A might have only three fixed rpms - 100, 200 and 500, but Tool B might have rpms from 20 to 2000. I'm not sure how I can construct a database without spelling out each number, as in the example above.
The total number of tools needed for the process can be defined at the beginning, however, it will change as the application becomes more complex in the future. I plan to address every contingency in the process. The idea is - if the user inputs the initial parameters and the desired outcome (another set of parameters), the app must find the 'best' path - sort of like a decision tree. The best path can be the fastest, cheapest, etc. I would like the user to choose what is best for him/her.
Unfortunately, parameters might change, relationships might change (but not regularly) - the 'rules' I will be using might be revised for better accuracy in prediction.
I also need to track each user's path and solutions' for future reference (but no personal details except username and email address for logging in). Maybe when the app is up and running, I'd like to make it more democratic, with users contributing to refining the logic/rules involved.
If possible, I would also like the app to output a graphical flowchart at the end showing the workflow with all tools grouped in an easy to understand layout.
*My questions:*
1. Will the app be better served with a relational DB like mySQL or an Object database? After a lot of research I've guessed that my particular case might be better served with Python and Zope/ZODB. But I might be wrong? Maybe PHP+mySQL or Django is a better fit? 2. Can anyone provide general advice on how to go about beginning such a project in ZOPE. Which is the best place to start learning for a newbie? 3. Can anyone recommend a good shared hosting provider that supports Zope fully but is not expensive? 4. Is there a module or app that is open source that I can use to output a graphical flowchart based on the results, or will I be better served programming it from scratch with Python?
I would appreciate any help in getting started. Thank you in advance. I have tried most online forums but have not good any productive answers. Most of the answers I got were pro-PHP+mySQL.
Adam
_______________________________________________ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
-- John Schinnerer - M.A., Whole Systems Design -------------------------------------------- - Eco-Living - Whole Systems Design Services People - Place - Learning - Integration john@eco-living.net http://eco-living.net
On 12/04/2011 09:31 PM, John Schinnerer wrote:
In any case, this looks (to me anyhow) like a very object-oriented system you are modeling so an object oriented approach and language would seem most suitable.
And how would you create (and update) objects in Python for: "at least 500 tools as of now, and it is bound to grow in the future as newer tools are introduced. Existing tools will also have version updates. Each tool, on average, has at least 100 properties that define the tool. Some of them have as high as 1000 unique properties." Regards, Fernando
On 12/04/2011 09:56 PM, Fernando Martins wrote:
On 12/04/2011 09:31 PM, John Schinnerer wrote:
In any case, this looks (to me anyhow) like a very object-oriented system you are modeling so an object oriented approach and language would seem most suitable.
And how would you create (and update) objects in Python for:
"at least 500 tools as of now, and it is bound to grow in the future as newer tools are introduced. Existing tools will also have version updates.
Each tool, on average, has at least 100 properties that define the tool. Some of them have as high as 1000 unique properties."
How familiar are you with OOP? What I mean is, when I read your high-level description of what you want to do, I imagine objects interacting with other. In your "car wash" example I see that, for instance. Or any kind of work-flow, which is what this sounds like to me. A state machine. Each tool is an object; it knows what its properties and abilities and possible states are and can communicate them to other objects and can accept and act on communication from other objects. In that way the objects interact with one another to do whatever it is you build them to do. I'm thinking of your "car wash" metaphor. In a car wash there are a variety of elements (objects) that communicate with one another to move the car through and to wash it as it moves through. Simple example: * mechanism that pulls or pushes the car through * mechanism that sprays it with water and soap * mechanism that scrubs and wipes and rubs it * mechanism that sprays it with rinse water * drying mechanism(s) These all need to signal one another so they do the right thing at the right time. Spray (or scrub or blow hot air) only when the car is in the right place for each. If some tools can be built by adding to/extending other tools, that is an object-friendly situation, since enhancing an existing tool to make another that you need saves you having to create all tools from scratch. In the car wash, maybe the different sized scrubbers used are all made by bolting together two or more of the smallest size scrubber. Maybe in parallel, maybe in series, depends on what is needed. Maybe the same scrubber controller can have different size brushes attached to it. In short, an object - tool - is code and data that interacts with other tools (other entities of code and data). An upgrade to a tool would involve changing the code and/or data that constitutes the tool. If that introduces some new way of interacting that other tools need to also know, then you add that to those tools also. Adding more tools means coding them. If they can be based on existing tools, so much the easier (and more object-appropriate). So that's some high-level information about an object-oriented approach to what I think your project is about. Tthe flexibility you appear to need as the system grows may be problematic for an RDB. And, my bias is OO, so perhaps someone who thinks in RDB-space can describe at high level how this would look in an RDB implmentation. cheers, John S. -- John Schinnerer - M.A., Whole Systems Design -------------------------------------------- - Eco-Living - Whole Systems Design Services People - Place - Learning - Integration john@eco-living.net http://eco-living.net
On Dec 5, 2011 09:48 "John Schinnerer" <john@eco-living.net> wrote:
On 12/04/2011 09:56 PM, Fernando Martins wrote:
On 12/04/2011 09:31 PM, John Schinnerer wrote:
In any case, this looks (to me anyhow) like a very object-oriented system you are modeling so an object oriented approach and language would seem most suitable.
And how would you create (and update) objects in Python for:
"at least 500 tools as of now, and it is bound to grow in the future as newer tools are introduced. Existing tools will also have version updates.
Each tool, on average, has at least 100 properties that define the tool. Some of them have as high as 1000 unique properties."
How familiar are you with OOP?
I think you are confusing me with the OP. And you did not answer my question. Are you recommending that a programmer codes all these objects by hand in Python? The know-how of what constitutes a tool, their properties and even the tool selection criteria is not developer know-how. Therefore, this information should be defined outside the program in way that the tool expert(s) can manage it. Which leads us to some storage solution, an RDB being the most common. Since, as I pointed out before, SQL most likely would not be able to do the tool selection alone, Python would merely load the data from the external source and create objects and expertise on the fly. Anyway, I agree with you that the main issue does not seem to be the web publishing solution, but rather how to represent the tool information and how to do tool selection, which is off-topic. Regards, Fernando
On Sun, Dec 4, 2011 at 06:12, Sareesh Sudhakaran <aysand@hotmail.com> wrote:
I have a personal project - a web application I wanted to develop - but I'm confused on which route to take. I am not under any time constraint.
Your case is complex and the answer is non-obvious. I think you will have to try to see.
The best I can describe it is as a kind of expert system (but not AI) that needs to find the best workflow for a process, given a set of initial and final parameters. E.g. a 'capsule' of data must pass through many 'tools' or 'environments' to reach a desired output - something like a very complicated car wash.
Let's say there are many tools that can be used at various stages in the process. I have estimated there are at least 500 tools as of now, and it is bound to grow in the future as newer tools are introduced. Existing tools will also have version updates.
Each tool, on average, has at least 100 properties that define the tool.
This kind of complexity and flexibility that your case has does lend itself well to the ZODB. But it is by no means impossible to do in a relational database. What you would need to do here if you did this in SQL is to have one table for the tools, and one table for the properties, and one table that for each tool and property has a value. This might be a bit complex to use of you use an ORM, but it should be doable.
problem of 'matching' the tools for analysis. E.g. Tool A might have only three fixed rpms - 100, 200 and 500, but Tool B might have rpms from 20 to 2000. I'm not sure how I can construct a database without spelling out each number, as in the example above.
Well, if you want to search for a specific RPM range it does get complicated, because then you probably want to store the RPM values as integers. And that means that you in the property table needs to have several columns depending on the value type. If the property needs text, you need to fetch it from the text column. If the property is an integer you need to fetch it from the integer column. If the property is a reference to another table, because it is a multi-select property with a limited set, you need to fetch it from a column for that, which in turn refers to another table with the actual values. This *does* get very complex very quickly. Of course, the ZODB has few such problems.
Will the app be better served with a relational DB like mySQL or an Object database? After a lot of research I've guessed that my particular case might be better served with Python and Zope/ZODB. But I might be wrong? Maybe PHP+mySQL or Django is a better fit?
First of all, find yourself a web-framework you like. Then use that. Most likely, that framework is going to limit you to using SQL. There are frameworks that don't specifically Pyramid integrates nicely with ZODB and hence gives you a choice. Then I would simply try to see if you are able to model the data in SQL at all, or if you dig yourself into unholy complexity. That should not take more than a couple of days of work to figure that out, if you concentrate on building a database and filling it with real or realistic data. If you don't dig yourself into a hole, the go with SQL, since that's what you know. If you do, try to build the model with ZODB and see of that works better for you. The ZODB might very well be the right choice here. But note that Zope is not. Zope was a trailblazer in the web framework world that's been around for 15 years now, and has as a result made some choices which turned out to not be the best ones in the long run, and accumulated a lot of cruft. There is work on fixing this, but that will take time, and the framework will be in a great flux during that time. As a result, Zope is not currently a good choice if you start a project. As languages go, Python rules. As web frameworks go, there are more Python frameworks than you can shake a stick at. I'd recommend either Django (because there are so many people using it, you will be able to find help ) or Pyramid (because it's really cool and supports ZODB well). //Lennart
participants (6)
-
Andreas Jung -
Fernando -
Fernando Martins -
John Schinnerer -
Lennart Regebro -
Sareesh Sudhakaran