- Zope - Zope lists

Truncate and pack
by Ken 02 Jul '03

02 Jul '03

Hi friends.. I've been remiss in packing and have hit the 2 gig (Python) limit. Some ugly creation of mine caused the unexpected bloat but I should've been more attentive... I substituted a 2-week old backup, but I'd like to recover as much as possible of those two weeks' work. If I understand correctly, fsrecover.py is not for this case, since only the very last transaction would/might be corrupt. It seems I need to truncate and pack per this howto: http://www.zope.org/Members/andym/2gig I'm using Zope 2.5.1 / Python 2.1.3 Zope binary / FreeBSD My questions: The howto is a couple of years old; is there any updated/improved advice? I'd like to truncate the minimum, but can't find any reference to the exact upper limit. I'm at 2,148,067,614. Anyone have the precise figure? Wish me luck... TIA, Ken

1 0

Re: [Zope] Retrieving zope management password.
by Sukhwinder Singh 02 Jul '03

02 Jul '03

> In a previous post you indicated you were connecting to ZODB > standalone. Is the inituser login failing in standalone mode, or have > you gone back to working through the ZMI? If it's the latter, make > sure you have followed *these* instructions: I am trying to Zope Management Interface. I replaced old database (data.fs) file with new one. And now login is not possible through ZMI. I do have backup copy. Old data.fs does allow login. It seems that passwords are stored in data.fs Can you tell me where username and passwords are stored in case of zope? In data.fs or in inituser? > On a somewhat related note, it sounds like you may be stuck with a > problem you don't have the resources (time, documentation, experience) > to solve. I'd suggest putting some money toward getting help from > someone with more hands-on Zope experience. I have to do it myself and I'll do it. I have already managed to extract data from one of the object (student), in old data.fs, and saved these 4000 records to csv file. I'll tell you later what method I used to extract it. I have found about 80% of attributes of this object. There are 6 main objects. Object which stores most data is student. I'll get data out of other objects as well. Now the problem is, how to login to ZMI because I have to add some external methods. I cannot do it without management username and password. I did restart the zope after creating the user but login failed. What I did is just replaced data.fs. How do I login now? --Sukhwinder Singh

4 3

RE: [Zope] Urgent - Database has consumed all possible space
by jwsacksteder＠ramprecision.com 02 Jul '03

02 Jul '03

Because every change to an object creates a new copy of that object. This is how the undo feature works. By packing the database you choose to throw away those old copies. If you had a webcam application that pushed a 10k file into your Zope database every 60 seconds, your database would grow by 14 Mb a day even if no other changes happened. If you have 200 Mb of data and 10% changes every day, that adds up exponentially. day 1 200 mb day 2 220 mb day 3 242 mb day 4 266 mb day 5 293 mb day 6 322 mb day 7 355 mb 70% overall growth in just 7 days! Either pack the database frequently or use temp folders for frequently-changing objects(and forego undo). Perhaps you need to pack the database in chunks. If the time the server was created is x, you could drop transactions older than x + 1 month, then x + 2 months, etc. It should slowly shrink. This is similar to what happens to a severely fragmented drive that is completely full- you have to make an initial pass to free space so the defragmentewr can work better on the next pass. If you have another machine available, you could move the data.fs onto it, pack and move it back. -----Original Message----- From: Michael Havard [mailto:nhavar@hotmail.com] Sent: Tuesday, July 01, 2003 4:24 PM To: zope(a)zope.org Cc: plattem(a)inetnebr.com Subject: Re: [Zope] Urgent - Database has consumed all possible space If it were only so easy. This is a shared box with many other applications and databases on it. Our database was only supposed to take up 200-300mb NOT 1.8gb big difference. The DBA's have scrounged for extra space and given me all they could. Hard drive space for these machines is expensive and not something I can just ask for. Plus I still need to explain why 200-300mb worth of text files exploded in ZOPE to 1.8gb of information.

2 1

Job opportunity
by kprah＠aldenstaffing.com 02 Jul '03

02 Jul '03

Hi my name is Karin and I am a technical recruiter located in White Plains, NY. I currently am working on a search for an individual with Zope, Plone, and Python. One of my consultants suggested that I list the opportunity with you. I would greatly appreciate your assistance. Here is a more detailed job description: The technologies they are really concentrating on are: Zope Plone (content management system) Python Apache HTML Linux It is an open-source based content-managed website, and they need someone to own it and all of the code modifications required to get things up and running as well as to maintain the site. They are looking for a full-time consultant, for 3-6 months in Midtown. The right candidate must have extensive hands-on expertise with all of the above technologies. He or she should be technically proficient to modify, enhance or add new functionality to Web content, the CMS interface (Plone), or other areas of the Web site. The right person will need strong expertise with HTML, Apache, and the Python. They are very open on dollars. If you're not interested but know anyone with these technologies please feel free to forward this email on to them. Or give me a call with their contact information and I will give them a call. Again, I appreciate your assistance and look forward to hearing from you! Sincerely, Karin Prah Vice President, Recruiting Alden Information Technology Staffing T/ 914-253-6622 (Mon, Tues, Thurs) T/ 203-402-0500 ext. 109 (Wed, Fri) F/ 914-253-8439 kprah(a)aldenstaffing.com www.aldenstaffing.com

1 0

localize zope
by george donnelly 02 Jul '03

02 Jul '03

Can anyone point me to some info on localizing the ZMI to Spanish (on FreeBSD). I appear to have done everything right but tildes and "enye"'s are getting turned into something else. thanks <--> george donnelly - http://www.zettai.net/ - "Quality Zope Hosting" Zope Hosting - Dynamic Website Design - Search Engine Promotion Yahoo, AIM: zettainet - MSN: zettainet(a)hotmail.com - ICQ: 51907738

1 0

No module named BTree when using standalone zodb
by Sukhwinder Singh 01 Jul '03

01 Jul '03

Now I trying to work directly with zodb. I am using python 2.2.3 and have downloaded theses versions of standalone zodb: ZODB3-3.1.win32-py2.1.exe ZODB3-3.1.win32-py2.2.exe ZODB3-3.2b2.win32-py2.2.exe But when I try this script: ////////////////////// from ZODB import FileStorage, DB storage = FileStorage.FileStorage('c:\\database\\var\\data.fs') db = DB(storage) connection = db.open() root = connection.root() employees = root['employees'] for employee in employees: print employee.name ////////////////////////////////// This error message "No module named BTree" is displayed: Traceback (most recent call last): File "zodb.py", line 7, in ? employees = root['employees'] File "D:\APP\PYTHON\lib\UserDict.py", line 14, in __getitem__ def __getitem__(self, key): return self.data[key] File "D:\APP\PYTHON\Lib\site-packages\ZODB\Connection.py", line 566, in setstate self._set_ghost_state(obj, p) File "D:\APP\PYTHON\Lib\site-packages\ZODB\Connection.py", line 605, in _set_ghost_state state = unpickler.load() File "D:\APP\PYTHON\Lib\site-packages\ZODB\Connection.py", line 203, in _persistent_load return self[oid] File "D:\APP\PYTHON\Lib\site-packages\ZODB\Connection.py", line 163, in __getitem__ klass=self._db._classFactory(self, module, name) File "D:\APP\PYTHON\lib\site-packages\ZODB\DB.py", line 126, in _classFactory return getattr(__import__(location, _globals, _globals, _silly),name) ImportError: No module named BTree Which version of standalone zodb I need to install for this BTree module to be available. --Sukhwinder Singh

1 0

RE: [Zope] Urgent - Database has consumed all possible space
by sean.upton＠uniontrib.com 01 Jul '03

01 Jul '03

1 - Hard drive space is cheap, at least for servers you control, and as I understand you do not control this server... I don't envy you if your DBAs consider <2gb to be a lot of space - don't they let you account for temp files, logs, etc??? 2 - Explain that the spaces is taken up by transaction logging that isn't properly rotated. This is the truth (FileStorage is a logging storage), and this is something that RDBMS admins, for example, would understand. Explain that after packing, the storage will take up less space. 3 - Pack regularly, and monitor your storage space; prune and delete unnecessary/old content. > -----Original Message----- > From: Michael Havard [mailto:nhavar@hotmail.com] > Sent: Tuesday, July 01, 2003 1:24 PM > To: zope(a)zope.org > Cc: plattem(a)inetnebr.com > Subject: Re: [Zope] Urgent - Database has consumed all possible space > > > If it were only so easy. This is a shared box with many other > applications > and databases on it. Our database was only supposed to take > up 200-300mb NOT > 1.8gb big difference. The DBA's have scrounged for extra > space and given me > all they could. Hard drive space for these machines is > expensive and not > something I can just ask for. Plus I still need to explain > why 200-300mb > worth of text files exploded in ZOPE to 1.8gb of information. > > > >From: Matthew Platte <plattem(a)inetnebr.com> > >To: Michael Havard <nhavar(a)hotmail.com> > >Subject: Re: [Zope] Urgent - Database has consumed all possible space > >Date: Tue, 01 Jul 2003 13:45:09 -0500 > >MIME-Version: 1.0 > >Received: from duck.inebraska.com ([199.184.119.13]) by > >mc10-f41.bay6.hotmail.com with Microsoft > SMTPSVC(5.0.2195.5600); Tue, 1 Jul > >2003 11:44:19 -0700 > >Received: from lin-dsl-static-206-222-213-206.inetnebr.com > >(lin-dsl-static-206-222-213-206.inetnebr.com [206.222.213.206])by > >duck.inebraska.com (8.12.7/8.12.7) with ESMTP id h61IiHMv021926for > ><nhavar(a)hotmail.com>; Tue, 1 Jul 2003 13:44:17 -0500 (CDT) > >Received: from inetnebr.com > (lin-dsl-static-206-222-213-205.inetnebr.com > >[206.222.213.205])by > lin-dsl-static-206-222-213-206.inetnebr.com (Postfix) > >with ESMTP id C25BA105344for <nhavar(a)hotmail.com>; Tue, 1 Jul 2003 > >13:43:47 -0500 (CDT) > >X-Message-Info: JGTYoYF78jEHjJx36Oi8+Q1OJDRSDidP > >Message-ID: <3F01D6B5.3030602(a)inetnebr.com> > >User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.1) > >Gecko/20020826 > >X-Accept-Language: en-us, en > >References: <Law9-F477yeZNCKWnXp0001219d(a)hotmail.com> > >Return-Path: plattem(a)inetnebr.com > >X-OriginalArrivalTime: 01 Jul 2003 18:44:19.0491 (UTC) > >FILETIME=[C7ED6F30:01C34000] > > > >Michael Havard wrote: > >>We've already tried packing. It keeps failing because of > insufficient > >>space in the DB. When it fails I get the distinct feeling > that we're > >>orphaning records and not really removing them from the DB > which makes the > >>pack an even larger process next time. > > > >>>>We are migrating data from an IIS server - approximately > 200mb via FTP > >>>>today I am told that the ZOPE tables now consume more > than 1.8gb. How is > >>>>it possible for 200mb of data to consume 1.8gb of > database space and > >>>>what can I do to correct it. > > > > > >Get a bigger drive. If 1.8GB is filling up the drive it was way too > >small/full in the first place. > > > >Then, when panic/things have settled down a bit, investigate > and fix the > >bloat problem. > > > >-- > >-------^.^-- > > > > > > _________________________________________________________________ > The new MSN 8: advanced junk mail protection and 2 months FREE* > http://join.msn.com/?page=features/junkmail > > > _______________________________________________ > Zope maillist - Zope(a)zope.org > http://mail.zope.org/mailman/listinfo/zope > ** No cross posts or HTML encoding! ** > (Related lists - > http://mail.zope.org/mailman/listinfo/zope-announce > http://mail.zope.org/mailman/listinfo/zope-dev ) >

1 0

Re: [Zope] Urgent - Database has consumed all possible space
by Michael Havard 01 Jul '03

01 Jul '03

If it were only so easy. This is a shared box with many other applications and databases on it. Our database was only supposed to take up 200-300mb NOT 1.8gb big difference. The DBA's have scrounged for extra space and given me all they could. Hard drive space for these machines is expensive and not something I can just ask for. Plus I still need to explain why 200-300mb worth of text files exploded in ZOPE to 1.8gb of information. >From: Matthew Platte <plattem(a)inetnebr.com> >To: Michael Havard <nhavar(a)hotmail.com> >Subject: Re: [Zope] Urgent - Database has consumed all possible space >Date: Tue, 01 Jul 2003 13:45:09 -0500 >MIME-Version: 1.0 >Received: from duck.inebraska.com ([199.184.119.13]) by >mc10-f41.bay6.hotmail.com with Microsoft SMTPSVC(5.0.2195.5600); Tue, 1 Jul >2003 11:44:19 -0700 >Received: from lin-dsl-static-206-222-213-206.inetnebr.com >(lin-dsl-static-206-222-213-206.inetnebr.com [206.222.213.206])by >duck.inebraska.com (8.12.7/8.12.7) with ESMTP id h61IiHMv021926for ><nhavar(a)hotmail.com>; Tue, 1 Jul 2003 13:44:17 -0500 (CDT) >Received: from inetnebr.com (lin-dsl-static-206-222-213-205.inetnebr.com >[206.222.213.205])by lin-dsl-static-206-222-213-206.inetnebr.com (Postfix) >with ESMTP id C25BA105344for <nhavar(a)hotmail.com>; Tue, 1 Jul 2003 >13:43:47 -0500 (CDT) >X-Message-Info: JGTYoYF78jEHjJx36Oi8+Q1OJDRSDidP >Message-ID: <3F01D6B5.3030602(a)inetnebr.com> >User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.1) >Gecko/20020826 >X-Accept-Language: en-us, en >References: <Law9-F477yeZNCKWnXp0001219d(a)hotmail.com> >Return-Path: plattem(a)inetnebr.com >X-OriginalArrivalTime: 01 Jul 2003 18:44:19.0491 (UTC) >FILETIME=[C7ED6F30:01C34000] > >Michael Havard wrote: >>We've already tried packing. It keeps failing because of insufficient >>space in the DB. When it fails I get the distinct feeling that we're >>orphaning records and not really removing them from the DB which makes the >>pack an even larger process next time. > >>>>We are migrating data from an IIS server - approximately 200mb via FTP >>>>today I am told that the ZOPE tables now consume more than 1.8gb. How is >>>>it possible for 200mb of data to consume 1.8gb of database space and >>>>what can I do to correct it. > > >Get a bigger drive. If 1.8GB is filling up the drive it was way too >small/full in the first place. > >Then, when panic/things have settled down a bit, investigate and fix the >bloat problem. > >-- >-------^.^-- > > _________________________________________________________________ The new MSN 8: advanced junk mail protection and 2 months FREE* http://join.msn.com/?page=features/junkmail

1 0

Re: [Zope] Urgent - Database has consumed all possible space
by Michael Havard 01 Jul '03

01 Jul '03

We've already tried packing. It keeps failing because of insufficient space in the DB. When it fails I get the distinct feeling that we're orphaning records and not really removing them from the DB which makes the pack an even larger process next time. >From: Jens Vagelpohl <jens(a)zope.com> >To: "Michael Havard" <nhavar(a)hotmail.com> >CC: zope(a)zope.org >Subject: Re: [Zope] Urgent - Database has consumed all possible space >Date: Tue, 1 Jul 2003 12:18:26 -0400 >MIME-Version: 1.0 (Apple Message framework v552) >Received: from smtp.zope.com ([63.100.190.10]) by mc4-f30.law16.hotmail.com >with Microsoft SMTPSVC(5.0.2195.5600); Tue, 1 Jul 2003 09:18:36 -0700 >Received: from zope.com (piggy.zope.com [192.168.23.9])(authenticated >bits=0)by smtp.zope.com (8.12.5/8.12.5) with ESMTP id h61GL8v2023629;Tue, 1 >Jul 2003 12:21:08 -0400 >X-Message-Info: JGTYoYF78jEHjJx36Oi8+Q1OJDRSDidP >In-Reply-To: <Law9-F75Px33E1ioWwr0001965f(a)hotmail.com> >Message-Id: <A4FD0B48-ABDF-11D7-9F63-000393D58818(a)zope.com> >X-Mailer: Apple Mail (2.552) >X-MailScanner: Found to be clean >Return-Path: jens(a)zope.com >X-OriginalArrivalTime: 01 Jul 2003 16:18:36.0999 (UTC) >FILETIME=[6CFF0570:01C33FEC] > >you need to pack your database. this is a normal maintenance step. log into >the ZMI and click on Control_Panel and then Database Management. you should >see a form that allows you to pack the database. > >jens > > >On Tuesday, Jul 1, 2003, at 11:13 US/Eastern, Michael Havard wrote: > >>We are migrating data from an IIS server - approximately 200mb via FTP >>today I am told that the ZOPE tables now consume more than 1.8gb. How is >>it possible for 200mb of data to consume 1.8gb of database space and what >>can I do to correct it. >> >>Can anyone help with this issue? >> >>We are running DCOracle2/OracleStorage on ZOPE 2.5.1 with CMF 1.3. >> >>_________________________________________________________________ >>Help STOP SPAM with the new MSN 8 and get 2 months FREE* >>http://join.msn.com/?page=features/junkmail >> >> >>_______________________________________________ >>Zope maillist - Zope(a)zope.org >>http://mail.zope.org/mailman/listinfo/zope >>** No cross posts or HTML encoding! ** >>(Related lists - http://mail.zope.org/mailman/listinfo/zope-announce >>http://mail.zope.org/mailman/listinfo/zope-dev ) > _________________________________________________________________ Tired of spam? Get advanced junk mail protection with MSN 8. http://join.msn.com/?page=features/junkmail

1 0

ParsedXML Documentation / Examples?
by zoper＠disturbance.dhs.org 01 Jul '03

01 Jul '03

Hello, After searching all afternoon, I have found a ton of periphery 'about' type information re: ParsedXML, but I am having trouble locating any substantive documentation, or in particular, examples. I'm interested in DTML examples, PythonScript examples, anything. Any pointers appreciated. -Bryan

3 4