On 7/2/2007 11:38 AM, David Fetter wrote:
> On Thu, Jun 28, 2007 at 06:17:25PM -0400, Christopher Browne wrote:
>> The v2.0 branch has already taken on the following fairly significant
>> changes (listed below).  Jan and I had a chat this afternoon about
>> some items still to do:
> 
> [snip]
> 
>> - Clone Node - to allow using pg_dump/PITR to populate a new
>>   subscriber node.
>> 
>> Are there more items we should try to add?
> 
> It would be really great to be able to promote a PITR node to a
> subscriber, or better still, to be able to use pg_dump + some kind of
> ancillary information to do same.  This would help a lot with the
> initial sync problem that makes replicating large databases so
> painful.

You are aware that only PITR will avoid a long running transaction?

The stuff I am currently (very slowly) working on is that very problem. 
Any long running transaction causes that the minxid in the SYNC's is 
stuck at that very xid during the entire runtime of the LRT. The problem 
with that is that the log selection in the slon worker uses an index 
scan who's only index scankey candidates are the minxid of one and the 
maxxid of another snapshot. That is the range of rows returned by the 
scan itself. Since the minxid is stuck, it will select larger and larger 
groups of log tuples only to filter out most of them on a higher level 
in the query via xxid_le_snapshot().

While the LRT is in progress, we don't have ANY chance of doing 
something at all because nobody knows if that transaction does any 
changes to the database that the next SYNC after it committed has to 
pick up. But in case it aborts, we know it did not change anything. So 
my idea right now is to trim that snapshot information by removing 
aborted transactions from it. That way slon will only suffer from an 
increasing index scankey range while the LRT is in progress, but not any 
more after it aborted. This will at least take care of pg_dump and slony 
copy_set(), because both never commit. LRT's that really do updates will 
continue to cause that problem.

pg_dump can now be done against a subscriber, and all long running 
reporting and stuff people tend to move off to slaves anyway. So all 
that is really destined to create trouble are long running housekeeping 
transactions.


Jan

-- 
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck at Yahoo.com #