On Tue, 2007-07-03 at 12:33 -0400, Christopher Browne wrote:
> I'll see about doing an experiment on this to see if, for the DELETE
> case, it seems to actually help.  It may be that the performance
> effects are small to none, so that the added code complication isn't
> worthwhile.
> 

In a simple test I ran, DELETE of the entire 5M record table using
sequential scan was MUCH faster (9.41s) than 5M individual DELETE
statements in a single transaction (552.49s).

5M records is small enough to fit into memory. I expect the difference
would be even greater when the index and table can't both fit into
memory and the deletes are distributed randomly over the table.

I think it is worth exploring ways of solving this problem. Right now
slony is great for small inserts, updates, and deletes. But any large
update/delete on the origin can cause the subscribers to fall way
behind.

> Remember, the thought we started with was:
>    "What if we could do something that would make mass operations less
>     expensive?"
> 
> I don't want to introduce anything that can materially increase
> processing costs.
> 
> The more intelligent we try to get, the more expensive the
> logtrigger() function gets, and if the price is high enough, then we
> gain nothing.
> 
> The only "win" I see is if we can opportunistically join some
> statements together.  If we have to make the log trigger function
> universally *WAY* more expensive, well, that's a performance loss :-(.

Is there any way that we could detect (even without 100% confidence)
that a transaction is "big" and we should spend more effort trying to
optimize it?

I know that's a big project (in the general case), but there might be
some simple things that would work.

Regards,
	Jeff Davis