Brian Fehrle <brianf at consistentstate.com> writes:
> Anyone know what could have caused this? The only think I know of that was going on was adding additional replication sets to the slony cluster at the time. I am thinking it may be more of a linux thing with too many open files rather than postgres/slony, as I haven't found anything related to slony/postgres when googling around for these errors.
>
> Thanks in advance for any feedback,

You might check the output of "ulimit -a" on your system, as that should
list limits.

cbbrowne at cbbrowne [12:23:35] [~/architecture] [master *]
-> % ulimit -a
-t: cpu time (seconds)         unlimited
-f: file size (blocks)         unlimited
-d: data seg size (kbytes)     unlimited
-s: stack size (kbytes)        8192
-c: core file size (blocks)    0
-m: resident set size (kbytes) unlimited
-u: processes                  unlimited
-n: file descriptors           1024
-l: locked-in-memory size (kb) 64
-v: address space (kb)         unlimited
-x: file locks                 unlimited
-i: pending signals            16382
-q: bytes in POSIX msg queues  819200
-e: max nice                   0
-r: max rt priority            0

On my system, the number of file descriptors is limited to 1024, by
default.  (These parms are commonly overridable, with some
OS-specificity.)

1024 isn't exactly a small number of files, so it's a bit of a surprise
to get hit by this.  If you're running database backends and slons under
a single postgres user, it's conceivable that the competition might be
heading towards the limit, though 1024 is a pretty large limit.

- I haven't had call to search for file descriptor usage, so you'll need
  to research how to do that ;-).  Presumably it can be monitored.

- Perhaps it could help to set up a "slony" Linux user, and run slons in
  that context; that gives an extra 1024 descriptors.

- If there's a process that's chewing up file descriptors and not
  returning that, it would be hugely useful to identify that.  Perhaps
  there's a bug in something.  It's conceivable for it to be the slons,
  but other causes can't be ruled out.
-- 
"cbbrowne","@","afilias.info"
Christopher Browne
"Bother,"  said Pooh,  "Eeyore, ready  two photon  torpedoes  and lock
phasers on the Heffalump, Piglet, meet me in transporter room three"