Fri Feb 10 09:10:14 PST 2006
- Previous message: [Slony1-general] New in slony
- Next message: [Slony1-general] missing css in prebuild docs
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Hi, I have a problem since upgrading from 1.1.0 to 1.1.5 and adding 3 more nodes to my set. I am getting too many connections. Bug ID: 1383 (closed) seems to be very similar. I use the altperl scripts to manage everything. All nodes were set up w/parent in conf file. If I use slon_kill script all connections will drop in about 10 seconds. Replication speed seems to be fine. I can restart slons with script and will receive several lines of "NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=9984" for the master. My original setup was running for about 4-6 months no real problems. It was only in a test environment. Cluster is master + 5 nodes, different sets for various nodes. Master- SuSe Enterprise Server 9 , PG 7.4.8 distro install, slony1-1.1.0 source Nodes- SuSe 9.3 Prof, PG 7.4.9 from source, slony1-1.1.0 source Then I upgraded to 1.1.5 release (not rc versions) and added 3 nodes in the same day. Since then after a few hours of replicating ps ax shows way too many connections. In postgresql.conf, max connections is set to 100. I am also running pg_autovacuum that ends up stopping after the build up of connections. Referencing the question in the Bug ID 1383 concenring the network setup, nothing has changed and all nodes are connected on VPN, the subnetmasks are the same except one. Thanks for any recommendations. 14419 ? S 0:04 /usr/bin/postmaster -D /var/lib/pgsql/data 14423 ? S 0:12 postgres: stats buffer process 14425 ? S 1:28 postgres: stats collector process 15131 ? S 0:00 /usr/bin//slon -c 0 -s 1000 -d2 cpc_attest_replic host=10.10.10.15 dbname=attest user=slony 15141 ? S 0:03 /usr/bin/perl /usr/local/bin/slon_watchdog --config=/usr/local/etc/cpc_attest_slon_tools.co 15142 ? Sl 2:17 /usr/bin//slon -c 0 -s 1000 -d2 cpc_attest_replic host=10.10.10.15 dbname=attest user=slony 15152 ? S 35:45 postgres: slony attest 10.10.10.15 async_notify waiting 15163 ? S 29:03 postgres: slony attest 10.10.10.15 COMMIT waiting 15176 ? S 29:15 postgres: slony attest 10.10.10.15 COMMIT waiting 15177 ? S 30:14 postgres: slony attest 10.10.10.15 COMMIT waiting 15178 ? S 29:00 postgres: slony attest 10.10.10.15 COMMIT waiting 15179 ? S 30:34 postgres: slony attest 10.10.10.15 COMMIT waiting 15180 ? S 29:20 postgres: slony attest 10.10.10.15 COMMIT waiting 15181 ? S 30:30 postgres: slony attest 10.10.10.15 COMMIT waiting 15182 ? S 30:52 postgres: slony attest 10.10.10.15 COMMIT waiting 15183 ? S 29:26 postgres: slony attest 10.10.10.15 COMMIT waiting 15184 ? S 0:46 postgres: slony attest 10.10.10.15 idle 15185 ? S 9:47 postgres: slony attest 10.10.10.15 idle 15197 ? S 0:00 /usr/bin//slon -c 0 -s 1000 -d2 cpc_attest_replic host=10.200.1.175 dbname=attest user=slon 15206 ? S 0:03 /usr/bin/perl /usr/local/bin/slon_watchdog --config=/usr/local/etc/cpc_attest_slon_tools.co 15216 ? Sl 0:24 /usr/bin//slon -c 0 -s 1000 -d2 cpc_attest_replic host=10.200.1.175 dbname=attest user=slon 15231 ? S 60:19 postgres: slony attest 10.10.10.15 async_notify waiting 15233 ? S 0:09 postgres: slony attest 10.10.10.15 idle 15245 ? S 0:00 /usr/bin//slon -c 0 -s 1000 -d2 cpc_attest_replic host=10.200.3.175 dbname=attest user=slon 15254 ? S 0:03 /usr/bin/perl /usr/local/bin/slon_watchdog --config=/usr/local/etc/cpc_attest_slon_tools.co 15264 ? Sl 0:24 /usr/bin//slon -c 0 -s 1000 -d2 cpc_attest_replic host=10.200.3.175 dbname=attest user=slon 15279 ? S 60:39 postgres: slony attest 10.10.10.15 async_notify waiting 15292 ? S 0:00 /usr/bin//slon -c 0 -s 1000 -d2 cpc_attest_replic host=10.200.4.175 dbname=attest user=slon 15301 ? S 0:03 /usr/bin/perl /usr/local/bin/slon_watchdog --config=/usr/local/etc/cpc_attest_slon_tools.co 15311 ? S 0:10 postgres: slony attest 10.10.10.15 idle 15321 ? Sl 0:24 /usr/bin//slon -c 0 -s 1000 -d2 cpc_attest_replic host=10.200.4.175 dbname=attest user=slon 15343 ? S 0:03 /usr/bin/perl /usr/local/bin/slon_watchdog --config=/usr/local/etc/cpc_attest_slon_tools.co 15412 ? S 0:00 /usr/bin//slon -c 0 -s 1000 -d2 cpc_attest_replic host=10.200.12.175 dbname=attest user=slo 15421 ? S 0:03 /usr/bin/perl /usr/local/bin/slon_watchdog --config=/usr/local/etc/cpc_attest_slon_tools.co 15431 ? Sl 0:25 /usr/bin//slon -c 0 -s 1000 -d2 cpc_attest_replic host=10.200.12.175 dbname=attest user=slo 15433 ? S 61:05 postgres: slony attest 10.10.10.15 async_notify waiting 15455 ? S 0:03 /usr/bin/perl /usr/local/bin/slon_watchdog --config=/usr/local/etc/cpc_attest_slon_tools.co 15541 ? S 0:00 /usr/bin//slon -c 0 -s 1000 -d2 cpc_attest_replic host=10.200.29.175 dbname=attest user=slo 15550 ? S 0:03 /usr/bin/perl /usr/local/bin/slon_watchdog --config=/usr/local/etc/cpc_attest_slon_tools.co 15561 ? S 60:26 postgres: slony attest 10.10.10.15 async_notify waiting 15571 ? Sl 0:25 /usr/bin//slon -c 0 -s 1000 -d2 cpc_attest_replic host=10.200.29.175 dbname=attest user=slo 15573 ? S 0:10 postgres: slony attest 10.10.10.15 idle 15574 ? S 60:13 postgres: slony attest 10.10.10.15 async_notify waiting 15586 ? S 0:00 /usr/bin//slon -c 0 -s 1000 -d2 cpc_attest_replic host=10.200.30.175 dbname=attest user=slo 15595 ? S 0:03 /usr/bin/perl /usr/local/bin/slon_watchdog --config=/usr/local/etc/cpc_attest_slon_tools.co 15634 ? Sl 0:25 /usr/bin//slon -c 0 -s 1000 -d2 cpc_attest_replic host=10.200.30.175 dbname=attest user=slo 15666 ? S 0:00 /usr/bin//slon -c 0 -s 1000 -d2 cpc_attest_replic host=192.168.1.175 dbname=attest user=slo 15675 ? S 0:03 /usr/bin/perl /usr/local/bin/slon_watchdog --config=/usr/local/etc/cpc_attest_slon_tools.co 15698 ? Sl 0:24 /usr/bin//slon -c 0 -s 1000 -d2 cpc_attest_replic host=192.168.1.175 dbname=attest user=slo 15736 ? S 60:43 postgres: slony attest 10.10.10.15 async_notify waiting 15747 ? S 0:11 postgres: slony attest 10.10.10.15 idle 15785 ? S 60:55 postgres: slony attest 10.10.10.15 async_notify waiting 15796 ? S 0:10 postgres: slony attest 10.10.10.15 idle 15863 ? S 0:09 postgres: slony attest 10.10.10.15 idle 514 ? S 4:37 postgres: slony attest 10.10.10.15 async_notify waiting 991 ? S 4:27 postgres: slony attest 10.10.10.15 async_notify waiting 1398 ? S 4:19 postgres: slony attest 10.10.10.15 async_notify waiting 1426 ? S 4:18 postgres: slony attest 10.10.10.15 async_notify waiting 1427 ? S 4:14 postgres: slony attest 10.10.10.15 async_notify waiting 1491 ? S 4:15 postgres: slony attest 10.10.10.15 async_notify waiting 1510 ? S 4:16 postgres: slony attest 10.10.10.15 async_notify waiting 1774 ? S 4:09 postgres: slony attest 10.10.10.15 async_notify waiting 1985 ? S 4:05 postgres: slony attest 10.10.10.15 async_notify waiting 1995 ? S 4:06 postgres: slony attest 10.10.10.15 async_notify waiting 2104 ? S 4:04 postgres: slony attest 10.10.10.15 async_notify waiting 2448 ? S 4:04 postgres: slony attest 10.10.10.15 async_notify waiting 2449 ? S 4:01 postgres: slony attest 10.10.10.15 async_notify waiting 2502 ? S 4:00 postgres: slony attest 10.10.10.15 async_notify waiting 2512 ? S 3:59 postgres: slony attest 10.10.10.15 async_notify waiting 2810 ? S 3:54 postgres: slony attest 10.10.10.15 async_notify waiting 2982 ? S 3:53 postgres: slony attest 10.10.10.15 async_notify waiting 2983 ? S 3:57 postgres: slony attest 10.10.10.15 async_notify waiting 2984 ? S 3:52 postgres: slony attest 10.10.10.15 async_notify waiting 3157 ? S 3:51 postgres: slony attest 10.10.10.15 async_notify waiting 3484 ? S 3:46 postgres: slony attest 10.10.10.15 async_notify waiting 3512 ? S 3:53 postgres: slony attest 10.10.10.15 async_notify waiting 3540 ? S 3:49 postgres: slony attest 10.10.10.15 async_notify waiting 3550 ? S 3:50 postgres: slony attest 10.10.10.15 async_notify waiting 3859 ? S 3:45 postgres: slony attest 10.10.10.15 async_notify waiting 4022 ? S 3:43 postgres: slony attest 10.10.10.15 async_notify waiting 4023 ? S 3:44 postgres: slony attest 10.10.10.15 async_notify waiting 4024 ? S 3:46 postgres: slony attest 10.10.10.15 async_notify waiting 4188 ? S 3:43 postgres: slony attest 10.10.10.15 async_notify waiting 4486 ? S 3:42 postgres: slony attest 10.10.10.15 async_notify waiting 4524 ? S 3:43 postgres: slony attest 10.10.10.15 async_notify waiting 4543 ? S 3:40 postgres: slony attest 10.10.10.15 async_notify waiting 4553 ? S 3:39 postgres: slony attest 10.10.10.15 async_notify waiting 4842 ? S 3:36 postgres: slony attest 10.10.10.15 async_notify waiting 5077 ? S 3:37 postgres: slony attest 10.10.10.15 async_notify waiting 5087 ? S 3:34 postgres: slony attest 10.10.10.15 async_notify waiting 5088 ? S 3:37 postgres: slony attest 10.10.10.15 async_notify waiting 5252 ? S 3:40 postgres: slony attest 10.10.10.15 async_notify waiting 5592 ? S 3:36 postgres: slony attest 10.10.10.15 async_notify waiting 5620 ? S 3:36 postgres: slony attest 10.10.10.15 async_notify waiting 5621 ? S 3:35 postgres: slony attest 10.10.10.15 async_notify waiting 5631 ? S 3:34 postgres: slony attest 10.10.10.15 async_notify waiting 5987 ? S 3:34 postgres: slony attest 10.10.10.15 async_notify waiting 6162 ? S 3:27 postgres: slony attest 10.10.10.15 async_notify waiting 6163 ? S 3:29 postgres: slony attest 10.10.10.15 async_notify waiting 6164 ? S 3:30 postgres: slony attest 10.10.10.15 async_notify waiting 6336 ? S 3:31 postgres: slony attest 10.10.10.15 async_notify waiting 6680 ? S 3:30 postgres: slony attest 10.10.10.15 async_notify waiting 6681 ? S 3:25 postgres: slony attest 10.10.10.15 async_notify waiting 6682 ? S 3:29 postgres: slony attest 10.10.10.15 async_notify waiting 6720 ? S 3:26 postgres: slony attest 10.10.10.15 async_notify waiting 7121 ? S 3:27 postgres: slony attest 10.10.10.15 async_notify waiting 7308 ? S 3:27 postgres: slony attest 10.10.10.15 async_notify waiting 7309 ? S 3:24 postgres: slony attest 10.10.10.15 async_notify waiting 7310 ? S 3:26 postgres: slony attest 10.10.10.15 async_notify waiting 7464 ? S 3:22 postgres: slony attest 10.10.10.15 async_notify waiting 7808 ? S 3:24 postgres: slony attest 10.10.10.15 async_notify waiting 7809 ? S 3:19 postgres: slony attest 10.10.10.15 async_notify waiting 7810 ? S 3:24 postgres: slony attest 10.10.10.15 async_notify waiting 7856 ? S 3:21 postgres: slony attest 10.10.10.15 async_notify waiting 8198 ? S 3:21 postgres: slony attest 10.10.10.15 async_notify waiting 8397 ? S 3:21 postgres: slony attest 10.10.10.15 async_notify waiting 8398 ? S 3:22 postgres: slony attest 10.10.10.15 async_notify waiting 8399 ? S 3:20 postgres: slony attest 10.10.10.15 async_notify waiting 8571 ? S 3:18 postgres: slony attest 10.10.10.15 async_notify waiting 8932 ? S 3:20 postgres: slony attest 10.10.10.15 async_notify waiting 8933 ? S 3:19 postgres: slony attest 10.10.10.15 async_notify waiting 8943 ? S 3:16 postgres: slony attest 10.10.10.15 async_notify waiting 8980 ? S 3:21 postgres: slony attest 10.10.10.15 async_notify waiting 9334 ? R 3:15 postgres: slony attest 10.10.10.15 async_notify 9516 ? S 3:15 postgres: slony attest 10.10.10.15 async_notify waiting 9517 ? S 3:17 postgres: slony attest 10.10.10.15 async_notify waiting 9527 ? S 3:16 postgres: slony attest 10.10.10.15 async_notify waiting 13569 ? S 3:03 postgres: slony attest 10.10.10.15 async_notify waiting 13966 ? S 3:02 postgres: slony attest 10.10.10.15 async_notify waiting 28086 ? S 0:00 trivial-rewrite -n rewrite -t unix -u 22888 ? S 0:00 /usr/bin//slon -c 0 -s 1000 -d2 cpc_attest_replic host=10.200.18.175 dbname=attest user=slo 22896 ? Sl 0:00 /usr/bin//slon -c 0 -s 1000 -d2 cpc_attest_replic host=10.200.18.175 dbname=attest user=slo 23501 ? S 0:00 /usr/bin//slon -c 0 -s 1000 -d2 cpc_attest_replic host=10.200.6.175 dbname=attest user=slon 23708 ? Sl 0:00 /usr/bin//slon -c 0 -s 1000 -d2 cpc_attest_replic host=10.200.6.175 dbname=attest user=slon [2006-02-09 02:30:11 AM] Performing: ANALYZE "_cpc_attest_replic"."sl_confirm" [2006-02-09 02:40:11 AM] Performing: ANALYZE "_cpc_attest_replic"."sl_event" [2006-02-09 02:50:11 AM] Performing: ANALYZE "_cpc_attest_replic"."sl_confirm" [2006-02-09 03:00:11 AM] Performing: ANALYZE "_cpc_attest_replic"."sl_event" [2006-02-09 03:00:11 AM] Performing: VACUUM ANALYZE "_cpc_attest_replic"."sl_seqlog" [2006-02-09 03:10:11 AM] Performing: ANALYZE "_cpc_attest_replic"."sl_confirm" [2006-02-09 03:20:11 AM] Performing: VACUUM ANALYZE "_cpc_attest_replic"."sl_event" [2006-02-09 03:20:11 AM] Performing: ANALYZE "_cpc_attest_replic"."sl_seqlog" [2006-02-09 03:30:11 AM] Performing: ANALYZE "_cpc_attest_replic"."sl_confirm" [2006-02-09 03:40:11 AM] Failed connection to database template1 with error: FATAL: sorry, too many clients already . [2006-02-09 03:40:11 AM] Error: Cannot connect to template1, exiting. Waiting for data... (interrupt to abort) CUSTDB:/home/cmount # NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=9983 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=9984 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=9985 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=10499 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=10613 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=10969 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=11024 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=11025 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=11035 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=11054 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=11055 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=11056 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=11507 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=11626 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=12017 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=12072 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=12073 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=12074 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=12165 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=12166 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=12167 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=12592 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=12712 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=13066 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=13112 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=13113 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=13114 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=13249 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=13250 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=13251 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=13731 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=13877 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=14193 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=14309 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=14310 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=14320 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=14443 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=14444 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=14445 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=14880 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=15052 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=15387 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=15487 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=15488 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=15489 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=15607 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=15608 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=15609 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=16020 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=6207 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=6296 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=6297 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=6578 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=6579 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=6596 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=6580 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=6627 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=6582 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=6584 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=6644 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=6656 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=6595 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=6675 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=7182 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=7192 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=7302 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=7339 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=7646 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=7710 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=7711 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=7721 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=7722 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=7723 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=7724 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=8168 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=8322 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=8773 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=8844 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=8845 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=8848 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=8862 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=8863 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=8865 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=9371 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=9534 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=9843 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=9917 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=9918 NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=9919
- Previous message: [Slony1-general] New in slony
- Next message: [Slony1-general] missing css in prebuild docs
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
More information about the Slony1-general mailing list