Christopher Browne cbbrowne at ca.afilias.info
Mon Jan 24 09:56:49 PST 2011
Mark Mielke <mark at mark.mielke.cc> writes:
> Hi all:
>
> I am trying to use Slony for the first time - sorry if this is a newbie 
> question...
>
> I am trying to replicating two of our JIRA databases into a data 
> warehouse using Slony. Each instance has it's own "jira" database, and 
> stores all tables and sequences in the "public" schema.
>
> In the data warehouse, I would like to replicate each instance to a 
> schema in the same database.
>
> So something like:
>
>      jira_host1:"jira":"public" -> dw_host1:"dw":"jira1"
>      jira_host2:"jira":"public" -> dw_host2:"dw":"jira2"
>
> Can Slony do this and I am missing it in the documentation? Or is Slony 
> not really designed to do this?
>
> If not designed to do this - does anybody have any suggestions on how to 
> accomplish the above?
>
> Thanks for the help, all!

Sadly, no, this isn't something that Slony can do today.

There has occasionally been talk about this sort of feature; the
challenge comes fundamentally in how to manage the configuration.  That
is, how do you control the names of table and schema?  How should Slony
capture that configuration, and maintain it?  And what is supposed to
happen if you reconfigure the cluster via MOVE SET or FAILOVER?

There's a chance that it has a destabilizing effect (e.g. - if you do
MOVE SET/FAILOVER, that might break the cluster), and if that be the
case, it's a dangerous thing to do.  I don't know that this is the case;
the analysis hasn't been done to determine such.

The current model of sl_table and sl_log_[12] are somewhat unamenable to
this in that the shape of configuration tends to be drawn from the
provider node; there's not a particularly easy place to go to redirect
the names.  

One of the particularly interesting ideas for the future is the change
to use "copy protocol", where we'd use COPY where today the slon
generates INSERT/UPDATE/DELETE statements.  Replicated data would get
transferred via COPY into the sl_log tables on the subscriber, and a
trigger on sl_log_[12] would expand it into I/U/D within the subscriber
database.  'Twould be nifty from performance standpoint, as:
  - It eliminates a bunch of parsing
  - It eliminates some present slon memory bloat issues because COPY is
    a streaming protocol
  - COPYing data into sl_log_* is *way* faster than INSERTing the same,
tho I digress there.

This protocol *IS* relevant in that having the expansion take place on
the subscriber lends itself to doing further transformations (e.g. -
such as changing table/schema name) as a part of that expansion.

In any case, the most troublesome part is thinking about how to
configure this, and managing that configuration.  
-- 
output = ("cbbrowne" "@" "afilias.info")
Christopher Browne
"Bother,"  said Pooh,  "Eeyore, ready  two photon  torpedoes  and lock
phasers on the Heffalump, Piglet, meet me in transporter room three"


More information about the Slony1-general mailing list