Henry henry at zen.co.za
Fri Apr 4 05:32:00 PDT 2008
Hello,

We're busy with an initial replication of a large DB to multiple slaves. 
The first slave has completed (after more than a week), but the others are
having what I imagine is a hiccup.

They're repeatedly restarting and copying the same tables - and never
completing.

The relevant section (during a funny restart) from the slony log (on a
slave):

>>> here's the first sign of something amiss (?):
...
01:53:16 FATAL localListenThread: cannot start transaction -
...
01:53:16 DEBUG2 slon_retry() from pid=7750
01:53:17 DEBUG1 slon: retry requested
01:53:17 DEBUG2 slon: notify worker process to shutdown
01:53:17 INFO remoteListenThread_4: disconnecting from 'dbname=xxx
host=zzz3 user=hhh password=000 port=5431'
01:53:17 INFO remoteListenThread_2: disconnecting from 'dbname=xxx
host=zzz1 user=hhh password=000 port=5431'
01:53:17 DEBUG1 syncThread: thread done
01:53:17 INFO remoteListenThread_1: disconnecting from 'dbname=xxx
host=zzz0 user=hhh password=000'
01:53:17 INFO remoteListenThread_3: disconnecting from 'dbname=xxx
host=zzz2 user=hhh password=000 port=5431'
01:53:18 DEBUG1 main: scheduler mainloop returned
01:53:18 DEBUG2 main: wait for remote threads
01:53:18 DEBUG2 sched_wakeup_node(): no_id=1 (0 threads + worker signaled)
01:53:20 DEBUG1 remoteListenThread_3: thread done
01:53:21 DEBUG1 remoteListenThread_1: thread done
01:53:21 DEBUG1 remoteListenThread_2: thread done
01:53:21 DEBUG1 remoteListenThread_4: thread done
01:53:37 DEBUG1 slon: child termination timeout - kill child
01:53:37 DEBUG2 slon: child terminated status: 9; pid: 7750, current
worker pid: 7750
01:53:37 DEBUG1 slon: restart of worker in 10 seconds
...
>>> slony starts again, and proceeds to copy the same tables...
...
01:53:47 CONFIG  main: slon version 1.2.12 starting up
01:53:47 DEBUG2 slon: watchdog process started
01:53:47 DEBUG2 slon: watchdog ready - pid = 7749
01:53:47 DEBUG2 slon: worker process created - pid = 19510
01:53:49 CONFIG  main: local node id = 5
01:53:49 DEBUG2 main: main process started
...

I've also checked the PG logs for any obvious errors.

Any idea where else to look and what to look for?

Any comments welcome.

Thanks
Henry



More information about the Slony1-general mailing list