6.7. Clustertest Test Framework

6.7.1. Introduction and Overview

The clustertest framework is implemented in Java, where tests are implemented in the interpreted JavaScript language. The use of Java made it much easier to implement tests involving concurrent activities, both in terms of inducing test load, and in, concurrently changing configuration of the replication cluster.

It consists of two physical portions:

• A framework, implemented in Java

  This software is available at clustertest-framework @ GitHub .

  This framework makes use of libraries from several other open source projects:

  • js.jar

    This is for org.mozilla.javascript, the Mozilla JavaScript interpreter

  • junit-4.8.1.jar

    JUnit, a unit test framework.

  • log4j-1.2.15.jar

    Log4J is a popular Java-based framework for generating event logs.

  • postgresql-8.4-701.jdbc3.jar

    This is the PostgreSQL JDBC driver.

  To build the framework, it is necessary to have a Java compiler and the build tool, Ant, installed. To build all the .jar files used by the framework, one will run the command, with output similar to the following:

  % ant jar
  Buildfile: /var/lib/postgresql/PostgreSQL/clustertest-framework/build.xml
  
  compile-common:
      [mkdir] Created dir: /var/lib/postgresql/PostgreSQL/clustertest-framework/build/classes
      [javac] /var/lib/postgresql/PostgreSQL/clustertest-framework/build.xml:23: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
  
  compile-testcoordinator:
      [javac] /var/lib/postgresql/PostgreSQL/clustertest-framework/build.xml:44: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
      [javac] Compiling 25 source files to /var/lib/postgresql/PostgreSQL/clustertest-framework/build/classes
      [javac] Note: /var/lib/postgresql/PostgreSQL/clustertest-framework/src/info/slony/clustertest/testcoordinator/script/ClientScript.java uses or overrides a deprecated API.
      [javac] Note: Recompile with -Xlint:deprecation for details.
       [copy] Copying 1 file to /var/lib/postgresql/PostgreSQL/clustertest-framework/build/classes/info/slony/clustertest/testcoordinator
  
  jar-common:
      [mkdir] Created dir: /var/lib/postgresql/PostgreSQL/clustertest-framework/build/jar
        [jar] Building MANIFEST-only jar: /var/lib/postgresql/PostgreSQL/clustertest-framework/build/jar/clustertest-common.jar
  
  compile-client:
      [javac] /var/lib/postgresql/PostgreSQL/clustertest-framework/build.xml:30: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
      [javac] Compiling 1 source file to /var/lib/postgresql/PostgreSQL/clustertest-framework/build/classes
       [copy] Copying 2 files to /var/lib/postgresql/PostgreSQL/clustertest-framework/build/classes/info/slony/clustertest/client
  
  jar-client:
        [jar] Building jar: /var/lib/postgresql/PostgreSQL/clustertest-framework/build/jar/clustertest-client.jar
  
  jar-coordinator:
        [jar] Building jar: /var/lib/postgresql/PostgreSQL/clustertest-framework/build/jar/clustertest-coordinator.jar
  
  jar:
  
  BUILD SUCCESSFUL
  Total time: 2 seconds

  At this time, there is no "regression test" for the test framework; to validate that it works requires running tests that use it.

  It includes classes supporting PostgreSQL- and Slony-I-specific functionality such as:

  • CreateDbScript

    Creates a database

  • DropDbScript

    Drops a database

  • LogShippingDaemon

    Starts up Slony-I logshipping daemon

  • LogShippingDumpScript

    Dumps and loads logshipping-based schema

  • PgCommand

    Run a PostgreSQL shell command (such as psql, createdb, and such)

  • PgDumpCommand

    Dump a PostgreSQL database

  • PsqlCommandExec

    Run SQL

  • ShellExecScript

    Run a shell script/command

  • SlonLauncher

    Start up a slon process

  • SlonikScript

    Run a slonik script

• Tests integrated into the Slony-I software distribution, that consist of a combination of shell scripts, JavaScript, and SQL scripts.

  See the directory clustertest in the Slony-I software distribution, which has two sets of tests:

  • Section 6.7.2

  • Section 6.7.3

6.7.2. DISORDER - DIStributed ORDER test

The DISORDER or DIStributed ORDER test is intended to provide concurrency tests involving a reasonably sophisticated schema to validate various aspects of Slony-I behavior under concurrent load.

It consists of:

• A schema for an inventory management application.

  Major objects include customers, inventory items, orders, order lines, and shipments.

  There are foreign key relationships between the various items, as well as triggers that maintain inventory and customer balances. Some of these relationships involve ON DELETE CASCADE, and so some actions may induce large numbers of cascaded updates.

• Stored procedures to induce creation of the various sorts of objects, purchases, shipments, and additions and removals of customers and products.

• Some tests are intended to be run against replicas, validating that balances add up. We believe that PostgreSQL applies changes in a transactional fashion such that they will always COMMIT leaving the visible state consistent; certain of the tests look for inconsistencies.

• There are JavaScript test scripts that induce all sorts of manipulations of replication clusters to validate that replication configuration changes succeed and fail as expected.

6.7.3. Regression Tests

These tests represent a re-coding of the tests previously implemented as shell scripts using the clustertest framework.

These tests have gradually been enhanced to provide coverage of scenarios with which Slony-I has had problems; it is to be expected that new bugs may lead to the addition of further tests.