:( I was hoping against hope that this wouldn't be the answer ... I think that the hard limit onSolaris must be 256, because ulimit -n 200 seems to have the appropriate effect of making Zope complain about "too many open files" ... but anything higher than 256 gets a "File not found" ... Let's try this from another angle ... why does Zope need to open so many files just to process a *large-but-not-really-large* POST? Can I change something about the way the form is set up (I use a lot of ":records") to circumvent the problem? Andreas Jung wrote:

There is always a hard limit of filedescriptors (based on the kernel configuration). But usually you can change the number of descriptors using "limit descriptors xxxx" under tcsh/csh. Under Bash I think you must call "ulimit".

Andreas ----- Original Message ----- From: "John Ziniti" <jziniti@speakeasy.org> To: <zope-dev@zope.org> Sent: Tuesday, October 02, 2001 11:46 Subject: [Zope-dev] file descriptors on Solaris

...

I am running into a problem where Zope is trying to open too many file descriptors (256) in order to process a POST.

I'm not sure how to phrase this question, but my reading has suggested that this limit may be set by the FILE struct in /usr/incldue/stdio.h.

Does anyone know if this is used in Python/Zope?

My first impression was that this problem would be easy to solve -- just up the number of FD's, but now, I'm not so sure. Any suggestions?

Here's a traceback -- but I don't think it's much help

Traceback (innermost last): File /u05/ilocal/opt/Zope-2.4.0-src/lib/python/ZPublisher/Publish.py,

line 223, in publish_module

...

File /u05/ilocal/opt/Zope-2.4.0-src/lib/python/ZPublisher/Publish.py,

line 187, in publish

...

File /u05/ilocal/opt/Zope-2.4.0-src/lib/python/Zope/__init__.py, line

226, in zpublisher_exception_hook

...

(Object: ApplicationDefaultPermissions) File /u05/ilocal/opt/Zope-2.4.0-src/lib/python/ZPublisher/Publish.py,

line 136, in publish

...

File

/u05/ilocal/opt/Zope-2.4.0-src/lib/python/ZPublisher/HTTPRequest.py, line 405, in processInputs

...

File /ilocal/lib/python2.1/cgi.py, line 517, in __init__ File /ilocal/lib/python2.1/cgi.py, line 606, in read_multi File /ilocal/lib/python2.1/cgi.py, line 519, in __init__ File /ilocal/lib/python2.1/cgi.py, line 616, in read_single File /ilocal/lib/python2.1/cgi.py, line 636, in read_lines File /ilocal/lib/python2.1/cgi.py, line 723, in make_file File /ilocal/opt/Zope-2.4.0-src/lib/python/tempfile.py, line 155, in

TemporaryFile

...

OSError: [Errno 2] No such file or directory

_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )

Yeah ... something tells me it's a little more complicated than that. Any advice on the other front? If I can reduce the number of files Zope needs to process this request, I'd grumpily agree to do that, is Zope opening a file for every <input name="x.name:records">? Will using <input name="x_names:list"> help? Thanks. Andreas Jung wrote:

----- Original Message ----- From: "John Ziniti" <jziniti@speakeasy.org> To: <zope-dev@zope.org> Sent: Tuesday, October 02, 2001 12:01 Subject: Re: [Zope-dev] file descriptors on Solaris

...

:( I was hoping against hope that this wouldn't be the answer ...

I think that the hard limit onSolaris must be 256, because ulimit -n 200 seems to have the appropriate effect of making Zope complain about "too many open files" ... but anything higher than 256 gets a "File not found" ...

Just ask your administrator to increase the number of file descriptors. Usually this requires *only* a reboot of the machine :-)

Andreas

_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )

----- Original Message ----- From: "John Ziniti" <jziniti@speakeasy.org> To: <zope-dev@zope.org> Sent: Tuesday, October 02, 2001 14:50 Subject: Re: [Zope-dev] file descriptors on Solaris [SUMMARY]

PROGNOSIS: The problem (on Solaris) is not very easy to fix, since it lies in the system-wide definition of a file descriptor, which uses only one byte to store the fd value (i.e., anything higher than 256 is meaningless, and truncated??). Changing this struct is not easy. The problem is not that Zope is exceeding the *allowed* number of FD's (usually policed by the shell), but that Zope is exceeding the *meaningful* number of FD's. This sucks. :-)

This is nonsense. Solaris allows of course to use more than 256 FDs. I don't know how they are stored inside the kernel but I have been using Solaris in projects where we used 1024 FDs and more. Zope does not increase the number of allowed FDs (resource module) but inherits the settings from the environment where Zope gets started. The number of FDs are set in Solaris in /etc/system: set rlim_fd_max = 4096 set rlim_fd_cur = 1024 Solaris 7+ allows up to 65536 FDs. Cheers, Andreas

----- Original Message ----- From: "John Ziniti" <jziniti@speakeasy.org> To: "Andreas Jung" <andreas@zope.com>; <zope-dev@zope.org> Sent: Tuesday, October 02, 2001 15:57 Subject: Re: [Zope-dev] file descriptors on Solaris [SUMMARY]

...

This is nonsense. Solaris allows of course to use more than 256 FDs. I don't know how they are stored inside the kernel but I have been using Solaris in projects where we used 1024 FDs and more. Zope does not increase the number of allowed FDs (resource module) but inherits the settings from the environment where Zope gets started.

The number of FDs are set in Solaris in /etc/system:

set rlim_fd_max = 4096 set rlim_fd_cur = 1024

Solaris 7+ allows up to 65536 FDs.

Solaris is definitely not my area of expertise, but to the best of my UNIX-hunting-around- looking-for-the-answer ability, that's all I can find. If Zope is started in a shell with $ ulimit -n 200 and I POST the offending form, I get a "too many open files" error. But if ulimit -n 512, then I get "No such file or directory" when $ZOPE/lib/python/tempfile.py tries to fdopen a file descriptor higher than 255 (line 155).

my /etc/system doesn't have anything other than some shm settings.

When u dont have the settings then you are using the default settings (maybe 256). Modify the settings according to your needs.

Can you create a form:

<form method="POST" enctype="multipart/form-data"> <input type="submit" value="Click me to test FD's"> <dtml-in "_.range(500)"> <input type=text name="input_<dtml-var sequence-number>" value="<dtml-var sequence-number>"> </dtml-in> </form>

I am running Linux here with 1024 FDs. Andreas

On Tue, 2 Oct 2001, John Ziniti wrote:

Yeah ... something tells me it's a little more complicated than that.

Like recompiling the kernel, quite possibly. On FreeBSD there's a sysctl, although you may still have to recompile the kernel in some cases I think; on Linux you can zap a variable in proc. On solaris....who knows <wry grin>.

Any advice on the other front? If I can reduce the number of files Zope needs to process this request, I'd grumpily agree to do that, is Zope opening a file for every <input name="x.name:records">? Will using <input name="x_names:list"> help?

I don't see why zope should need so many file descriptors. But you may have to start tracing code <frown>. I have previously noticed a related phenomenon: on FreeBSD when I start zope, *something* tries to open every possible port number using a separate file descriptor. They must then be closed, because after startup only the ports requested are still open. I determined this by running lsof and capturing the output. In this case, it appears to be benign; the only negative consequence is the appearance of the "Too many open file descriptors" message in /var/log/messages. --RDM