Hi

ABOUT
======

Please be informed that we encountered a issue with slonik parser when submitting a script with more than 9990 to 10000 commands.

I mean

/path/to/slonilk << EOF
# include <preamble.sk>;
command 1;
command 2;
....
command 9997;

command is something like echo, set add sequence (....), set add table (....) and so on.


slonik crash with a memory error.

EXAMPLE FOR SLONIK CRASH

=========================

As a example from Slony sources - once the sources have been compiled -

Generate a stupid script

echo "include <preamble.sk>;" > test.slonik
seq 1 10000 | xargs -i echo "echo '{}';" >> test.slonik

Submit the script to slonik
cat test.slonik | src/slonik/slonik
<stdin>:9995: ERROR: memory exhausted at or near ;


We've got more than 20000 objects (tables + sequences) to replicate.

FIXING ISSUE FOR THE PARSER LIMITATION/WORKAROUND
==================================================

While investigating, I found that the problem came from the default value for YYMAXDEPTH in src/slonik/parser.c
----------

/* YYMAXDEPTH -- maximum size the stacks can grow to (effective only
   if the built-in stack extension method is used).

   Do not make this value too large; the results are undefined if
   YYSTACK_ALLOC_MAXIMUM < YYSTACK_BYTES (YYMAXDEPTH)
   evaluated with infinite-precision integer arithmetic.  */

#ifndef YYMAXDEPTH
# define YYMAXDEPTH 10000
#endif

-----


Taking into account the notice in the file, I came to the conclusoion that 10000 is a arbitrary value. We decided to put a a value like 100000 instead on a Linux x86_64 and it works

Since I do not use slony_logshipping I did not investigate on it.

The first workaround is to split the script in several scripts but it is not a pleasant workaround.

We've tested a value like 100000 for YYMAXDEPTH and we did not have any problem.


At Office we work with EnterpriseDB and ask EnterpriseDB for fixing the required value for Slony 1.2.22


As costumer, EnterpriseDB  was ok to fix this issue.

I did not investigate for Slony 2.X but my concern is that the issue may be the same.


I hope this information could help someone.

Kind regards.

David.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.slony.info/pipermail/slony1-hackers/attachments/20120119/c6353f02/attachment.htm