On Wed, May 31, 2006 at 10:01:26AM -0400, Rod Taylor wrote:
> I've been thinking of the initial COPY process.
> 
> The problem is that with a large amount of data you end up with a very
> large transaction on the data provider. The transaction on the
> subscriber isn't as important since it will normally be an otherwise
> idle database.
> 
> COPY in is one part, but building indexes on the subscriber is the
> painful part and during much of this process the data provider has an
> idle connection.
 
Pardon my ignorance, but is the provider actually sitting in a
transaction while the subscriber is building indexes, and if so, why?
ISTM there's no reason you'd need indexes (or RI for that matter) while
loading data into a subscriber.
 
> I haven't looked at the log shipping support, but I suspect it holds
> part of the answer. The data provider does not need to be active while
> the new subscriber is catching up to the transaction log when log
> shipping is enabled.
> 
> Is there any reason you could not convert between a shipped log and the
> standard "online" mode?
> 
> If possible, that would reduce the initial transaction size on the data
> provider to the time required to dump out the initial copy of the
> tables. Still a significant amount of time, one transaction per table
> would be better, but much shorter than today.
> 
> 
> A first approximation of a transition between log shipping and standard
> modes would be to write the data to a temporary area on the data
> provider machine (avoid network overhead) and later on send it across
> the wire without being connected to the data provider database.
> 
> -- 
> 
> _______________________________________________
> Slony1-general mailing list
> Slony1-general at gborg.postgresql.org
> http://gborg.postgresql.org/mailman/listinfo/slony1-general
> 

-- 
Jim C. Nasby, Sr. Engineering Consultant      jnasby at pervasive.com
Pervasive Software      http://pervasive.com    work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf       cell: 512-569-9461