Cyril SCETBON <scetbon at echo.fr> writes:
> When I try to create a lot of set slonik dies with segmentation fault
> message. The only way to succeed is to split the file containing the
> create set statement and to try them one by one or two by two. When I
> use strace on slonik it displays :
>
> read(3, "#--\n# Slony-I organizes tables i"..., 8192) = 8192
> read(3, "d table (set id=5, origin=201, i"..., 8192) = 8192
> read(3, ");\nset add table (set id=9, orig"..., 8192) = 8192
> mmap(NULL, 18446744073709547520, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
>
> Is there a maximum number of lines ? a memory parameter to use with slonik ?
>
> thanks

We're using Bison to generate the parser, and it normally doesn't
present any particular fixed limits on how much data it can handle.

I just ran a test run where I created 500 tables and then added them,
in one fell swoop, to replication, using the sample script...
Attached is a patch that may be applied to the sample script in the
Slony-I source tree to get the same results.

With 500 tables to set up and copy, even though they were empty, it
took 452 seconds to apply the subscription.  (Cheap desktop PC :-).)

I encountered no problems.  Might there be something odd about your
platform where your OS defines stack size for programs to be rather
small?  That doesn't make *much* sense; if such an effect were
breaking slonik, then it ought to be breaking other things, like maybe
your PostgreSQL instance!

Here's an excerpt of the set creation slonik script:

include </tmp/slonytest-temp.A24149/preamble.slonik>;
create set (id=1, origin=1, comment='SLONTEST82 Tables and Sequences');
set add table (id=1, set id=1, origin=1, fully qualified name='public.t1', comment='SLONTEST82 table public.t1');
set add table (id=2, set id=1, origin=1, fully qualified name='public.t2', comment='SLONTEST82 table public.t2');
set add table (id=3, set id=1, origin=1, fully qualified name='public.t3', comment='SLONTEST82 table public.t3');
set add table (id=4, set id=1, origin=1, fully qualified name='public.t4', comment='SLONTEST82 table public.t4');
set add table (id=5, set id=1, origin=1, fully qualified name='public.t5', comment='SLONTEST82 table public.t5');
...
set add table (id=494, set id=1, origin=1, fully qualified name='public.t494', comment='SLONTEST82 table public.t494');
set add table (id=495, set id=1, origin=1, fully qualified name='public.t495', comment='SLONTEST82 table public.t495');
set add table (id=496, set id=1, origin=1, fully qualified name='public.t496', comment='SLONTEST82 table public.t496');
set add table (id=497, set id=1, origin=1, fully qualified name='public.t497', comment='SLONTEST82 table public.t497');
set add table (id=498, set id=1, origin=1, fully qualified name='public.t498', comment='SLONTEST82 table public.t498');
set add table (id=499, set id=1, origin=1, fully qualified name='public.t499', comment='SLONTEST82 table public.t499');
set add table (id=500, set id=1, origin=1, fully qualified name='public.t500', comment='SLONTEST82 table public.t500');
set add sequence (id=1, set id=1, origin=1, fully qualified name='public.t1_id_seq', comment='SLONTEST82 sequence public.t1_id_seq');
set add sequence (id=2, set id=1, origin=1, fully qualified name='public.t2_id_seq', comment='SLONTEST82 sequence public.t2_id_seq');
set add sequence (id=3, set id=1, origin=1, fully qualified name='public.t3_id_seq', comment='SLONTEST82 sequence public.t3_id_seq');
set add sequence (id=4, set id=1, origin=1, fully qualified name='public.t4_id_seq', comment='SLONTEST82 sequence public.t4_id_seq');
set add sequence (id=5, set id=1, origin=1, fully qualified name='public.t5_id_seq', comment='SLONTEST82 sequence public.t5_id_seq');
set add sequence (id=6, set id=1, origin=1, fully qualified name='public.t6_id_seq', comment='SLONTEST82 sequence public.t6_id_seq');
set add sequence (id=7, set id=1, origin=1, fully qualified name='public.t7_id_seq', comment='SLONTEST82 sequence public.t7_id_seq');
...
set add sequence (id=496, set id=1, origin=1, fully qualified name='public.t496_id_seq', comment='SLONTEST82 sequence public.t496_id_seq');
set add sequence (id=497, set id=1, origin=1, fully qualified name='public.t497_id_seq', comment='SLONTEST82 sequence public.t497_id_seq');
set add sequence (id=498, set id=1, origin=1, fully qualified name='public.t498_id_seq', comment='SLONTEST82 sequence public.t498_id_seq');
set add sequence (id=499, set id=1, origin=1, fully qualified name='public.t499_id_seq', comment='SLONTEST82 sequence public.t499_id_seq');
set add sequence (id=500, set id=1, origin=1, fully qualified name='public.t500_id_seq', comment='SLONTEST82 sequence public.t500_id_seq');

Here's the CVS diff:

chris at dba2:Slony-I/CMD/slony1-1.2/tools> cvs diff -u configure-replication.sh
Index: configure-replication.sh
===================================================================
RCS file: /home/cvsd/slony1/slony1-engine/tools/configure-replication.sh,v
retrieving revision 1.1.2.4
diff -c -u -r1.1.2.4 configure-replication.sh
cvs diff: conflicting specifications of output style
--- configure-replication.sh	6 Mar 2008 18:49:03 -0000	1.1.2.4
+++ configure-replication.sh	6 Jun 2008 15:43:33 -0000
@@ -6,6 +6,13 @@
 NUMNODES=${NUMNODES:-"2"}
 
 # Defaults - origin node
+DB1=db1
+DB2=db2
+HOST1=localhost
+HOST2=localhost
+PORT1=5882
+PORT2=5882
+
 DB1=${DB1:-${PGDATABASE:-"slonytest"}}
 HOST1=${HOST1:-`hostname`}
 USER1=${USER1:-${PGUSER:-"slony"}}
@@ -117,15 +124,18 @@
  recurringemail recurringprint session shipto sic status tax
  transactions translation vendor vendortax warehouse yearend"
 
+ALTTABLES1=""
+ALTSEQUENCES1=""
+for i in `seq 500`; do
+   ALTTABLES1="$ALTTABLES1 t${i}"
+   ALTSEQUENCES1="$ALTSEQUENCES1 t${i}_id_seq"
+   psql -h $HOST1 -p $PORT1 -d $DB1 -c "create table t${i} (id serial primary key, data text not null unique);"
+   psql -h $HOST2 -p $PORT2 -d $DB2 -c "create table t${i} (id serial primary key, data text not null unique);"
+done
+
 for t in `echo $ALTTABLES1`; do
   ALTTABLES="$ALTTABLES public.${t}"
 done
-  
-ALTSEQUENCES1="acc_trans_entry_id_seq audittrail_entry_id_seq
- custom_field_catalog_field_id_seq custom_table_catalog_table_id_seq
- id inventory_entry_id_seq invoiceid jcitemsid orderitemsid
- partscustomer_entry_id_seq partsvendor_entry_id_seq
- session_session_id_seq shipto_entry_id_seq"
 
 for s in `echo $ALTSEQUENCES1`; do
   ALTSEQUENCES="$ALTSEQUENCES public.${s}"

-- 
(format nil "~S@~S" "cbbrowne" "linuxdatabases.info")
http://linuxfinances.info/info/spiritual.html
"The program is manufactured by Quantel, a Silicon Valley company
located in Clearwater, Florida."