Documentation
 
 
 

21.2. EnterpriseDB to EnterpriseDB Replication

21.2.1. Introduction to EnterpriseDB Replication

EnterpriseDB Replication is a "master to multiple slaves" replication system with cascading and slave promotion that includes all features and capabilities needed to replicate large databases to a reasonably limited number of slave systems.

EnterpriseDB Replication allows databases to be started and stopped on an existing database with out the need for a dump/reload cycle.

21.2.2. Limitations of EnterpriseDB Replication

EnterpriseDB Replication does not automatically propagate schema (table definitions) changes, nor does it have any ability to replicate large objects.

  1. Please note that EnterpriseDB Replication does not allow multiple masters.

  2. Keep in mind that EnterpriseDB Replication does not have any functionality within it to detect a node failure, or automatically promote a node to a master or other data origin.

21.2.3. EnterpriseDB Replication Concepts

Before we go into further details about EnterpriseDB Replication it is important to come to terms with some basic EnterpriseDB Replication jargon.

So let us discuss some of those key concepts:

  • Cluster

  • Node

  • Replication Set

  • Master, Slaves and Subscribers

  • edb-replication process

  • slonik configuration processor

21.2.3.1. Cluster

An EnterpriseDB Replication cluster is a set of EnterpriseDB database instances between which replication is to take place.

Each database instance in which replication is to take place is identified by a node number.

The cluster name is specified in each and every Slonik script via the directive:

cluster name = 'test';
  

If the Cluster name is test, then EnterpriseDB Replication will create, in each database instance in the cluster, a schema with the same name as the cluster prefixed with an underscore character. So for a cluster named "test" we would have a corresponding schema named "_test" created by EnterpriseDB Replication.

21.2.3.2. Node

An EnterpriseDB Replication Node is a named EnterpriseDB database that will be participating in replication. Thus, an EnterpriseDB Replication cluster consists of:

  • A cluster name

  • A set of EnterpriseDB Replication nodes, each of which has a namespace based on that cluster name

21.2.3.3. Replication Set

A replication set is defined as the set of tables and sequences that are to be replicated between the subscribed nodes within an EnterpriseDB Replication cluster.

A replication set can replicate the following:

  • Keys on tables that are to be replicated that have no suitable primary key

  • Tables to be replicated

  • Sequences to be replicated

It is possible for multiple sets to exist within the same cluster.

21.2.3.4. Master, Slaves and Subscribers

Origin indicates the originating node and is also referred to as being the "master" node. The master node is the only node where all the database transactions as in inserts, updates, deletes are made. This is the only node in which the required applications modify the data that is to be replicated across all the other nodes.

All the other nodes that are part of a cluster excluding the origin node are referred to as the "slave" nodes. These "slave" nodes in the cluster subscribe to the replication set, indicating that they want to receive data.

Hence the origin node can never be a "subscriber" because it is the node to which all the slave nodes will be subscribing to.

EnterpriseDB Replication also supports the concept of cascaded subscriptions where a subscribed node itself can act as a "provider" to other nodes in the cluster for a particular replication set.

The concept of master, slaves, and subscribers can be further clarified by discussing the example in the screenshot above. In the particular example, Node 1 is the origin or master node, and Nodes 2-6 are all slave nodes. From these Nodes 2-4 have subscribed to Node 1, whereas Nodes 5-6 have subscribed to Node 3 which in turn is subscribed to Node 1, hence illustrating the concept of cascaded subscriptions.

21.2.3.5. The edb-replication process

The "edb-replication" is the process that manages all replication events on each node be it the master or slave. The EnterpriseDB Replication daemons are the actual programs that perform replication.

Currently these events can be either:

  1. Configuration events

  2. SYNC events

Configuration events normally occur when a slonik script is run, which in ends up updating the configuration of the cluster.

SYNC events are composed of a group of transactions on the origin node that are applied together on all the subscriber nodes. For example a bunch of inserts or updates on the origin node will compose a SYNC event.

21.2.3.6. Slonik Configuration Processor

Slonik is a command line utility that is used to set up and modify configurations of EnterpriseDB Replication clusters.

It reads a set of Slonik statements, which are written in a scripting language with syntax similar to that of SQL, and performs the set of configuration changes on the EnterpriseDB Replication nodes specified in the script.

The format of the Slonik language is very similar to that of SQL and the slonik command processor processes scripts that are used to submit events to update the configuration of an EnterpriseDB Replication cluster. This includes such things as adding and removing nodes, modifying communications paths, adding or removing subscriptions.

21.2.3.7. Switchover

Switchover is a planned reversal of master node and slave node. All committed transactions on the master are guaranteed to be flushed to the slave during the switch. When the "old" master is ready to come back online, you can either leave it as the slave or you can perform another switchover and it will become the master again. This is all accomplished with a simple command line script that will usually run in 10-seconds or less.

Suppose we have 2 nodes up and running and in sync with each other as shown in the following screenshot:

Suppose node1 is the current "master" node and node2 is the "slave" and only subscriber node, and a web application is accessing the database on node1. The following script will do a controlled switchover from node1 to node2:

lock set (id = 1, origin = 1);
wait for event (origin = 1, confirmed = 2);
move set (id = 1, old origin = 1, new origin = 2);
wait for event (origin = 1, confirmed = 2);
    

21.2.3.8. Failover

Failover is necessary when an unplanned event takes the master node offline. Once this occurs, our slave node will always be in an inconsistent state where no partially committed transactions are visible. Because of the Asynchronous nature of the transfer, it is possible that a small number of transactions may not have been flushed to the slave before the unplanned failure event. To cater for such a scenario we run a simple command line script that will take less than 10-seconds or so.

Taking along our previous example,suppose we have the following scenario once a failover occurs:

Now in order to promote the slave node as the new master, run the following slonik command:

failover (id = 1, backup node = 2);
    

EnterpriseDB Replication does not provide automatic detection for failed systems out of the box.

21.2.4. EnterpriseDB Replication Setup

In order to have replication up and running within a database cluster, we need to have an instance of EnterpriseDB and the EnterpriseDB Replication daemon running on all nodes that belong to that cluster.

The installation of EnterpriseDB needs to be done prior to setting up EnterpriseDB Replication. You don't need to perform any explicit installation for "EnterpriseDB Replication", as it is an integral part of the EnterpriseDB Database Server distribution.

21.2.4.1. System Requirements

Any platform that can run EnterpriseDB should be able to run EnterpriseDB Replication. EnterpriseDB Replication is successfully tested on Linux/Unix and Windows OS flavors.

21.2.4.2. EnterpriseDB Replication Software Dependencies

21.2.4.2.1. Time Synchronization

All the servers used within the replication cluster need to have their Real Time Clocks in sync. This is to ensure that EnterpriseDB Replication doesn't generate errors with messages indicating that a subscriber is already ahead of its provider during replication. We recommend you have ntpd running on all nodes, where subscriber nodes using the "master" provider host as their time server.

It is possible for EnterpriseDB Replication itself to function even in the face of there being some time discrepancies, but having systems "in sync" is usually pretty important for distributed applications.

In any case, what commonly seems to be the "best practice" with EnterpriseDB Replication is for the edb-postmaster user and/or the user under which "EnterpriseDB Replication" runs to use TZ=UTC or TZ=GMT. Those timezones are sure to be supported on any platform, and have the merit over "local" timezones that times never wind up leaping around due to Daylight Savings Time.

21.2.4.2.2. Network Connectivity

It is necessary that the hosts that are to replicate between one another have bidirectional network communications between the EnterpriseDB instances. That is, if node B is replicating data from node A, it is necessary that there be a path from A to B and from B to A. It is recommended that, as much as possible, all nodes in an EnterpriseDB Replication cluster allow this sort of bidirectional communications from any node in the cluster to any other node in the cluster.

21.2.4.2.3. Setting up EnterpriseDB Replication for replication

Let us start out with a sample database in order to see replication in place. As an example we will be replicating the sample "edb" database that is shipped with EnterpriseDB.

  1. You have uncommented the listen_addresses line in your postgresql.conf to enable TCP/IP socket

  2. You have enabled access to all nodes in your cluster(s) via pg_hba.conf

The REPLICATIONUSER needs to be an EnterpriseDB database superuser. The default superuser for EnterpriseDB is "enterprisedb" although it would be a good idea to define a separate superuser for using EnterpriseDB Replication to distinguish between the roles.

You should also set some shell variables. It is not compulsory to do so, but it is recommended as we will be using these shell variables in most of the commands and/or shell scripts later on in this example. These shell variables are as follows:

  1. CLUSTERNAME=edb_asyncreplic_example

  2. MASTERDBNAME=edb_master

  3. SLAVEDBNAME=edb_slave

  4. MASTERDBPORT=5444

  5. SLAVEDBPORT=5488

  6. MASTERHOST=localhost

  7. SLAVEHOST=localhost

  8. REPLICATIONUSER= edb_asyncreplic_user

Here are a couple of examples for setting variables in common shells:

  1. bash, sh, ksh export CLUSTERNAME= edb_asyncreplic_example

  2. setenv CLUSTERNAME edb_asyncreplic_example

21.2.4.3. Creating a database user for replication

Before proceeding any further you need to create the database superuser on each of the cluster nodes, that we earlier assigned the shell REPLICATIONUSER variable. Please make sure that EnterpriseDB is up and running on both the master and slave(s).

createuser -A -D $REPLICATIONUSER

21.2.4.3.1. Preparing the databases

After creating this database superuser you must proceed with creating the database that you will be replicating the data for on both the master and slave node. First you create the database on the master node and then proceed with creating the same database on the slave node. Please note that this just creates a sample database cloned from the template1 database by default without any tables or data apart from the system tables.

createdb -O $ REPLICATIONUSER -h $MASTERHOST -p $MASTERDBPORT $MASTERDBNAME
createdb -O $ REPLICATIONUSER -h $SLAVEHOST  -p $SLAVEDBPORT $SLAVEDBNAME

Once we are done with creating the database we can now populate it with our sample database tables with data as well as a couple of stored procedures. For this purpose we will store all this in the following database script and name it "edb-sample.sql".

--
--  Script that creates the 'sample' tables, views, procedures,
--  functions, triggers, etc.
--
--  Start new transaction - commit all or nothing
--
BEGIN;
--
--  Create and load tables used in the documentation examples.
--
--  Create the 'dept' table
--
CREATE TABLE dept (
    deptno          NUMBER(2) NOT NULL CONSTRAINT dept_pk PRIMARY KEY,
    dname           VARCHAR2(14) CONSTRAINT dept_dname_uq UNIQUE,
    loc             VARCHAR2(13)
);
--
--  Create the 'emp' table
--
CREATE TABLE emp (
    empno           NUMBER(4) NOT NULL CONSTRAINT emp_pk PRIMARY KEY,
    ename           VARCHAR2(10),
    job             VARCHAR2(9),
    mgr             NUMBER(4),
    hiredate        DATE,
    sal             NUMBER(7,2) CONSTRAINT emp_sal_ck CHECK (sal > 0),
    comm            NUMBER(7,2),
    deptno          NUMBER(2) CONSTRAINT emp_ref_dept_fk
                        REFERENCES dept(deptno)
);
--
--  Create the 'jobhist' table
--
CREATE TABLE jobhist (
    empno           NUMBER(4) NOT NULL,
    startdate       DATE NOT NULL,
    enddate         DATE,
    job             VARCHAR2(9),
    sal             NUMBER(7,2),
    comm            NUMBER(7,2),
    deptno          NUMBER(2),
    chgdesc         VARCHAR2(80),
    CONSTRAINT jobhist_pk PRIMARY KEY (empno, startdate),
    CONSTRAINT jobhist_ref_emp_fk FOREIGN KEY (empno)
        REFERENCES emp(empno) ON DELETE CASCADE,
    CONSTRAINT jobhist_ref_dept_fk FOREIGN KEY (deptno)
        REFERENCES dept (deptno) ON DELETE SET NULL,
    CONSTRAINT jobhist_date_chk CHECK (startdate <= enddate)
);
--
--  Create the 'salesemp' view
--
CREATE OR REPLACE VIEW salesemp AS
    SELECT empno, ename, hiredate, sal, comm FROM emp WHERE job = 'SALESMAN';
--
--  Sequence to generate values for function 'new_empno'.
--
CREATE SEQUENCE next_empno START WITH 8000 INCREMENT BY 1;
--
--  Issue PUBLIC grants
--
GRANT ALL ON emp TO PUBLIC;
GRANT ALL ON dept TO PUBLIC;
GRANT ALL ON jobhist TO PUBLIC;
GRANT ALL ON salesemp TO PUBLIC;
GRANT ALL ON next_empno TO PUBLIC;
--
--  Load the 'dept' table
--
INSERT INTO dept VALUES (10,'ACCOUNTING','NEW YORK');
INSERT INTO dept VALUES (20,'RESEARCH','DALLAS');
INSERT INTO dept VALUES (30,'SALES','CHICAGO');
INSERT INTO dept VALUES (40,'OPERATIONS','BOSTON');
--
--  Load the 'emp' table
--
INSERT INTO emp VALUES (7369,'SMITH','CLERK',7902,'17-DEC-80',800,NULL,20);
INSERT INTO emp VALUES (7499,'ALLEN','SALESMAN',7698,'20-FEB-81',1600,300,30);
INSERT INTO emp VALUES (7521,'WARD','SALESMAN',7698,'22-FEB-81',1250,500,30);
INSERT INTO emp VALUES (7566,'JONES','MANAGER',7839,'02-APR-81',2975,NULL,20);
INSERT INTO emp VALUES (7654,'MARTIN','SALESMAN',7698,'28-SEP-81',1250,1400,30);
INSERT INTO emp VALUES (7698,'BLAKE','MANAGER',7839,'01-MAY-81',2850,NULL,30);
INSERT INTO emp VALUES (7782,'CLARK','MANAGER',7839,'09-JUN-81',2450,NULL,10);
INSERT INTO emp VALUES (7788,'SCOTT','ANALYST',7566,'19-APR-87',3000,NULL,20);
INSERT INTO emp VALUES (7839,'KING','PRESIDENT',NULL,'17-NOV-81',5000,NULL,10);
INSERT INTO emp VALUES (7844,'TURNER','SALESMAN',7698,'08-SEP-81',1500,0,30);
INSERT INTO emp VALUES (7876,'ADAMS','CLERK',7788,'23-MAY-87',1100,NULL,20);
INSERT INTO emp VALUES (7900,'JAMES','CLERK',7698,'03-DEC-81',950,NULL,30);
INSERT INTO emp VALUES (7902,'FORD','ANALYST',7566,'03-DEC-81',3000,NULL,20);
INSERT INTO emp VALUES (7934,'MILLER','CLERK',7782,'23-JAN-82',1300,NULL,10);
--
--  Load the 'jobhist' table
--
INSERT INTO jobhist VALUES (7369,'17-DEC-80',NULL,'CLERK',800,NULL,20,'New Hire');
INSERT INTO jobhist VALUES (7499,'20-FEB-81',NULL,'SALESMAN',1600,300,30,'New Hire');
INSERT INTO jobhist VALUES (7521,'22-FEB-81',NULL,'SALESMAN',1250,500,30,'New Hire');
INSERT INTO jobhist VALUES (7566,'02-APR-81',NULL,'MANAGER',2975,NULL,20,'New Hire');
INSERT INTO jobhist VALUES (7654,'28-SEP-81',NULL,'SALESMAN',1250,1400,30,'New Hire');
INSERT INTO jobhist VALUES (7698,'01-MAY-81',NULL,'MANAGER',2850,NULL,30,'New Hire');
INSERT INTO jobhist VALUES (7782,'09-JUN-81',NULL,'MANAGER',2450,NULL,10,'New Hire');
INSERT INTO jobhist VALUES (7788,'19-APR-87','12-APR-88','CLERK',1000,NULL,20,'New Hire');
INSERT INTO jobhist VALUES (7788,'13-APR-88','04-MAY-89','CLERK',1040,NULL,20,'Raise');
INSERT INTO jobhist VALUES (7788,'05-MAY-90',NULL,'ANALYST',3000,NULL,20,'Promoted to Analyst');
INSERT INTO jobhist VALUES (7839,'17-NOV-81',NULL,'PRESIDENT',5000,NULL,10,'New Hire');
INSERT INTO jobhist VALUES (7844,'08-SEP-81',NULL,'SALESMAN',1500,0,30,'New Hire');
INSERT INTO jobhist VALUES (7876,'23-MAY-87',NULL,'CLERK',1100,NULL,20,'New Hire');
INSERT INTO jobhist VALUES (7900,'03-DEC-81','14-JAN-83','CLERK',950,NULL,10,'New Hire');
INSERT INTO jobhist VALUES (7900,'15-JAN-83',NULL,'CLERK',950,NULL,30,'Changed to Dept 30');
INSERT INTO jobhist VALUES (7902,'03-DEC-81',NULL,'ANALYST',3000,NULL,20,'New Hire');
INSERT INTO jobhist VALUES (7934,'23-JAN-82',NULL,'CLERK',1300,NULL,10,'New Hire');
--
--  Populate statistics table and view (pg_statistic/pg_stats)
--
ANALYZE dept;
ANALYZE emp;
ANALYZE jobhist;
--
--  Procedure that lists all employees' numbers and names
--  from the 'emp' table using a cursor.
--
CREATE OR REPLACE PROCEDURE list_emp
IS
    v_empno         NUMBER(4);
    v_ename         VARCHAR2(10);
    CURSOR emp_cur IS
        SELECT empno, ename FROM emp ORDER BY empno;
BEGIN
    OPEN emp_cur;
    DBMS_OUTPUT.PUT_LINE('EMPNO    ENAME');
    DBMS_OUTPUT.PUT_LINE('-----    -------');
    LOOP
        FETCH emp_cur INTO v_empno, v_ename;
        EXIT WHEN emp_cur%NOTFOUND;
        DBMS_OUTPUT.PUT_LINE(v_empno || '     ' || v_ename);
    END LOOP;
    CLOSE emp_cur;
END;
--
--  Procedure that selects an employee row given the employee
--  number and displays certain columns.
--
CREATE OR REPLACE PROCEDURE select_emp (
    p_empno         IN  NUMBER
)
IS
    v_ename         emp.ename%TYPE;
    v_hiredate      emp.hiredate%TYPE;
    v_sal           emp.sal%TYPE;
    v_comm          emp.comm%TYPE;
    v_dname         dept.dname%TYPE;
    v_disp_date     VARCHAR2(10);
BEGIN
    SELECT ename, hiredate, sal, NVL(comm, 0), dname
        INTO v_ename, v_hiredate, v_sal, v_comm, v_dname
        FROM emp e, dept d
        WHERE empno = p_empno
          AND e.deptno = d.deptno;
    v_disp_date := TO_CHAR(v_hiredate, 'MM/DD/YYYY');
    DBMS_OUTPUT.PUT_LINE('Number    : ' || p_empno);
    DBMS_OUTPUT.PUT_LINE('Name      : ' || v_ename);
    DBMS_OUTPUT.PUT_LINE('Hire Date : ' || v_disp_date);
    DBMS_OUTPUT.PUT_LINE('Salary    : ' || v_sal);
    DBMS_OUTPUT.PUT_LINE('Commission: ' || v_comm);
    DBMS_OUTPUT.PUT_LINE('Department: ' || v_dname);
EXCEPTION
    WHEN NO_DATA_FOUND THEN
        DBMS_OUTPUT.PUT_LINE('Employee ' || p_empno || ' not found');
    WHEN OTHERS THEN
        DBMS_OUTPUT.PUT_LINE('The following is SQLERRM:');
        DBMS_OUTPUT.PUT_LINE(SQLERRM);
        DBMS_OUTPUT.PUT_LINE('The following is SQLCODE:');
        DBMS_OUTPUT.PUT_LINE(SQLCODE);
END;
--
--  Procedure that queries the 'emp' table based on
--  department number and employee number or name.  Returns
--  employee number and name as IN OUT parameters and job,
--  hire date, and salary as OUT parameters.
--
CREATE OR REPLACE PROCEDURE emp_query (
    p_deptno        IN     NUMBER,
    p_empno         IN OUT NUMBER,
    p_ename         IN OUT VARCHAR2,
    p_job           OUT    VARCHAR2,
    p_hiredate      OUT    DATE,
    p_sal           OUT    NUMBER
)
IS
BEGIN
    SELECT empno, ename, job, hiredate, sal
        INTO p_empno, p_ename, p_job, p_hiredate, p_sal
        FROM emp
        WHERE deptno = p_deptno
          AND (empno = p_empno
           OR  ename = UPPER(p_ename));
END;
--
--  Procedure to call 'emp_query_caller' with IN and IN OUT
--  parameters.  Displays the results received from IN OUT and
--  OUT parameters.
--
CREATE OR REPLACE PROCEDURE emp_query_caller
IS
    v_deptno        NUMBER(2);
    v_empno         NUMBER(4);
    v_ename         VARCHAR2(10);
    v_job           VARCHAR2(9);
    v_hiredate      DATE;
    v_sal           NUMBER;
BEGIN
    v_deptno := 30;
    v_empno  := 0;
    v_ename  := 'Martin';
    emp_query(v_deptno, v_empno, v_ename, v_job, v_hiredate, v_sal);
    DBMS_OUTPUT.PUT_LINE('Department : ' || v_deptno);
    DBMS_OUTPUT.PUT_LINE('Employee No: ' || v_empno);
    DBMS_OUTPUT.PUT_LINE('Name       : ' || v_ename);
    DBMS_OUTPUT.PUT_LINE('Job        : ' || v_job);
    DBMS_OUTPUT.PUT_LINE('Hire Date  : ' || v_hiredate);
    DBMS_OUTPUT.PUT_LINE('Salary     : ' || v_sal);
EXCEPTION
    WHEN TOO_MANY_ROWS THEN
        DBMS_OUTPUT.PUT_LINE('More than one employee was selected');
    WHEN NO_DATA_FOUND THEN
        DBMS_OUTPUT.PUT_LINE('No employees were selected');
END;
--
--  Function to compute yearly compensation based on semimonthly
--  salary.
--
CREATE OR REPLACE FUNCTION emp_comp (
    p_sal           NUMBER,
    p_comm          NUMBER
) RETURN NUMBER
IS
BEGIN
    RETURN (p_sal + NVL(p_comm, 0)) * 24;
END;
--
--  Function that gets the next number from sequence, 'next_empno',
--  and ensures it is not already in use as an employee number.
--
CREATE OR REPLACE FUNCTION new_empno RETURN NUMBER
IS
    v_cnt           INTEGER := 1;
    v_new_empno     NUMBER;
BEGIN
    WHILE v_cnt > 0 LOOP
        SELECT next_empno.nextval INTO v_new_empno FROM dual;
        SELECT COUNT(*) INTO v_cnt FROM emp WHERE empno = v_new_empno;
    END LOOP;
    RETURN v_new_empno;
END;
--
--  SPL function that adds a new clerk to table 'emp'.  This function
--  uses package 'emp_admin'.
--
CREATE OR REPLACE FUNCTION hire_clerk (
    p_ename         VARCHAR2,
    p_deptno        NUMBER
) RETURN NUMBER
IS
    v_empno         NUMBER(4);
    v_ename         VARCHAR2(10);
    v_job           VARCHAR2(9);
    v_mgr           NUMBER(4);
    v_hiredate      DATE;
    v_sal           NUMBER(7,2);
    v_comm          NUMBER(7,2);
    v_deptno        NUMBER(2);
BEGIN
    v_empno := new_empno;
    INSERT INTO emp VALUES (v_empno, p_ename, 'CLERK', 7782,
        TRUNC(SYSDATE), 950.00, NULL, p_deptno);
    SELECT empno, ename, job, mgr, hiredate, sal, comm, deptno INTO
        v_empno, v_ename, v_job, v_mgr, v_hiredate, v_sal, v_comm, v_deptno
        FROM emp WHERE empno = v_empno;
    DBMS_OUTPUT.PUT_LINE('Department : ' || v_deptno);
    DBMS_OUTPUT.PUT_LINE('Employee No: ' || v_empno);
    DBMS_OUTPUT.PUT_LINE('Name       : ' || v_ename);
    DBMS_OUTPUT.PUT_LINE('Job        : ' || v_job);
    DBMS_OUTPUT.PUT_LINE('Manager    : ' || v_mgr);
    DBMS_OUTPUT.PUT_LINE('Hire Date  : ' || v_hiredate);
    DBMS_OUTPUT.PUT_LINE('Salary     : ' || v_sal);
    DBMS_OUTPUT.PUT_LINE('Commission : ' || v_comm);
    RETURN v_empno;
EXCEPTION
    WHEN OTHERS THEN
        DBMS_OUTPUT.PUT_LINE('The following is SQLERRM:');
        DBMS_OUTPUT.PUT_LINE(SQLERRM);
        DBMS_OUTPUT.PUT_LINE('The following is SQLCODE:');
        DBMS_OUTPUT.PUT_LINE(SQLCODE);
        RETURN -1;
END;
--
--  PostgreSQL PL/pgSQL function that adds a new salesman
--  to table 'emp'.
--
CREATE OR REPLACE FUNCTION hire_salesman (
    p_ename         VARCHAR,
    p_sal           NUMERIC,
    p_comm          NUMERIC
) RETURNS NUMERIC
AS $$
DECLARE
    v_empno         NUMERIC(4);
    v_ename         VARCHAR(10);
    v_job           VARCHAR(9);
    v_mgr           NUMERIC(4);
    v_hiredate      DATE;
    v_sal           NUMERIC(7,2);
    v_comm          NUMERIC(7,2);
    v_deptno        NUMERIC(2);
BEGIN
    v_empno := new_empno();
    INSERT INTO emp VALUES (v_empno, p_ename, 'SALESMAN', 7698,
        CURRENT_DATE, p_sal, p_comm, 30);
    SELECT INTO
        v_empno, v_ename, v_job, v_mgr, v_hiredate, v_sal, v_comm, v_deptno
        empno, ename, job, mgr, hiredate, sal, comm, deptno
        FROM emp WHERE empno = v_empno;
    RAISE INFO 'Department : %', v_deptno;
    RAISE INFO 'Employee No: %', v_empno;
    RAISE INFO 'Name       : %', v_ename;
    RAISE INFO 'Job        : %', v_job;
    RAISE INFO 'Manager    : %', v_mgr;
    RAISE INFO 'Hire Date  : %', v_hiredate;
    RAISE INFO 'Salary     : %', v_sal;
    RAISE INFO 'Commission : %', v_comm;
    RETURN v_empno;
EXCEPTION
    WHEN OTHERS THEN
        RAISE INFO 'The following is SQLERRM:';
        RAISE INFO '%', SQLERRM;
        RAISE INFO 'The following is SQLSTATE:';
        RAISE INFO '%', SQLSTATE;
        RETURN -1;
END;
$$ LANGUAGE 'plpgsql';
--
--  Rule to INSERT into view 'salesemp'
--
CREATE OR REPLACE RULE salesemp_i AS ON INSERT TO salesemp
DO INSTEAD
    INSERT INTO emp VALUES (NEW.empno, NEW.ename, 'SALESMAN', 7698,
        NEW.hiredate, NEW.sal, NEW.comm, 30);
--
--  Rule to UPDATE view 'salesemp'
--
CREATE OR REPLACE RULE salesemp_u AS ON UPDATE TO salesemp
DO INSTEAD
    UPDATE emp SET empno    = NEW.empno,
                   ename    = NEW.ename,
                   hiredate = NEW.hiredate,
                   sal      = NEW.sal,
                   comm     = NEW.comm
        WHERE empno = OLD.empno;
--
--  Rule to DELETE from view 'salesemp'
--
CREATE OR REPLACE RULE salesemp_d AS ON DELETE TO salesemp
DO INSTEAD
    DELETE FROM emp WHERE empno = OLD.empno;
--
--  After statement-level trigger that displays a message after
--  an insert, update, or deletion to the 'emp' table.  One message
--  per SQL command is displayed.
--
CREATE OR REPLACE TRIGGER user_audit_trig
    AFTER INSERT OR UPDATE OR DELETE ON emp
DECLARE
    v_action        VARCHAR2(24);
BEGIN
    IF INSERTING THEN
        v_action := ' added employee(s) on ';
    ELSIF UPDATING THEN
        v_action := ' updated employee(s) on ';
    ELSIF DELETING THEN
        v_action := ' deleted employee(s) on ';
    END IF;
    DBMS_OUTPUT.PUT_LINE('User ' || USER || v_action || TO_CHAR(SYSDATE,'YYYY-MM-DD'));
END;
/
--
--  Before row-level trigger that displays employee number and
--  salary of an employee that is about to be added, updated,
--  or deleted in the 'emp' table.
--
CREATE OR REPLACE TRIGGER emp_sal_trig
    BEFORE DELETE OR INSERT OR UPDATE ON emp
    FOR EACH ROW
DECLARE
    sal_diff       NUMBER;
BEGIN
    IF INSERTING THEN
        DBMS_OUTPUT.PUT_LINE('Inserting employee ' || :NEW.empno);
        DBMS_OUTPUT.PUT_LINE('..New salary: ' || :NEW.sal);
    END IF;
    IF UPDATING THEN
        sal_diff := :NEW.sal - :OLD.sal;
        DBMS_OUTPUT.PUT_LINE('Updating employee ' || :OLD.empno);
        DBMS_OUTPUT.PUT_LINE('..Old salary: ' || :OLD.sal);
        DBMS_OUTPUT.PUT_LINE('..New salary: ' || :NEW.sal);
        DBMS_OUTPUT.PUT_LINE('..Raise     : ' || sal_diff);
    END IF;
    IF DELETING THEN
        DBMS_OUTPUT.PUT_LINE('Deleting employee ' || :OLD.empno);
        DBMS_OUTPUT.PUT_LINE('..Old salary: ' || :OLD.sal);
    END IF;
END;
/
--
--  Package specification for the 'emp_admin' package.
--
CREATE OR REPLACE PACKAGE emp_admin
IS
    FUNCTION get_dept_name (
        p_deptno        NUMBER
    ) RETURN VARCHAR2;
    FUNCTION update_emp_sal (
        p_empno         NUMBER,
        p_raise         NUMBER
    ) RETURN NUMBER;
    PROCEDURE hire_emp (
        p_empno         NUMBER,
        p_ename         VARCHAR2,
        p_job           VARCHAR2,
        p_sal           NUMBER,
        p_hiredate      DATE,
        p_comm          NUMBER,
        p_mgr           NUMBER,
        p_deptno        NUMBER
    );
    PROCEDURE fire_emp (
        p_empno         NUMBER
    );
END emp_admin;
--
--  Package body for the 'emp_admin' package.
--
CREATE OR REPLACE PACKAGE BODY emp_admin
IS
    --
    --  Function that queries the 'dept' table based on the department
    --  number and returns the corresponding department name.
    --
    FUNCTION get_dept_name (
        p_deptno        IN NUMBER
    ) RETURN VARCHAR2
    IS
        v_dname         VARCHAR2(14);
    BEGIN
        SELECT dname INTO v_dname FROM dept WHERE deptno = p_deptno;
        RETURN v_dname;
    EXCEPTION
        WHEN NO_DATA_FOUND THEN
            DBMS_OUTPUT.PUT_LINE('Invalid department number ' || p_deptno);
            RETURN '';
    END;
    --
    --  Function that updates an employee's salary based on the
    --  employee number and salary increment/decrement passed
    --  as IN parameters.  Upon successful completion the function
    --  returns the new updated salary.
    --
    FUNCTION update_emp_sal (
        p_empno         IN NUMBER,
        p_raise         IN NUMBER
    ) RETURN NUMBER
    IS
        v_sal           NUMBER := 0;
    BEGIN
        SELECT sal INTO v_sal FROM emp WHERE empno = p_empno;
        v_sal := v_sal + p_raise;
        UPDATE emp SET sal = v_sal WHERE empno = p_empno;
        RETURN v_sal;
    EXCEPTION
        WHEN NO_DATA_FOUND THEN
            DBMS_OUTPUT.PUT_LINE('Employee ' || p_empno || ' not found');
            RETURN -1;
        WHEN OTHERS THEN
            DBMS_OUTPUT.PUT_LINE('The following is SQLERRM:');
            DBMS_OUTPUT.PUT_LINE(SQLERRM);
            DBMS_OUTPUT.PUT_LINE('The following is SQLCODE:');
            DBMS_OUTPUT.PUT_LINE(SQLCODE);
            RETURN -1;
    END;
    --
    --  Procedure that inserts a new employee record into the 'emp' table.
    --
    PROCEDURE hire_emp (
        p_empno         NUMBER,
        p_ename         VARCHAR2,
        p_job           VARCHAR2,
        p_sal           NUMBER,
        p_hiredate      DATE,
        p_comm          NUMBER,
        p_mgr           NUMBER,
        p_deptno        NUMBER
    )
    AS
    BEGIN
        INSERT INTO emp(empno, ename, job, sal, hiredate, comm, mgr, deptno)
            VALUES(p_empno, p_ename, p_job, p_sal,
                   p_hiredate, p_comm, p_mgr, p_deptno);
    END;
    --
    --  Procedure that deletes an employee record from the 'emp' table based
    --  on the employee number.
    --
    PROCEDURE fire_emp (
        p_empno         NUMBER
    )
    AS
    BEGIN
        DELETE FROM emp WHERE empno = p_empno;
    END;
END;
COMMIT;

In the following step you run the above sql script ONLY on the master database. We will run this script only on the master and not the slave as we want to populate the sample tables with replication and not with this sql script.

edb-psql -f /edb-sample.sql -h $MASTERHOST -p $MASTERDBPORT -U $REPLICATIONUSER 
$MASTERDBNAME

EnterpriseDB Replication does not automatically copy table definitions from a master when a slave subscribes to it, so we need to import this database schema. We do this with pg_dump. Please note that we only import the database schema and not the data in the database tables.

pg_dump -s -U $REPLICATIONUSER -h $MASTERHOST -p $MASTERDBPORT $MASTERDBNAME | 
edb-psql -U $REPLICATIONUSER -h $SLAVEHOST -p $SLAVEDBPORT $SLAVEDBNAME

To ensure that only the sample tables in the master node are populated with data and not the slave node(s) view the data first on the master and then on the slave.

21.2.4.4. Configuring the Database for Replication

Creating the configuration tables, stored procedures, triggers and configuration is all done through the slonik tool. It is a specialized scripting aid that mostly calls stored procedures in the master/slave (node) databases. The script to create the initial configuration for the simple master-slave setup of our "edb" database looks like this:

  
#!/bin/sh

	#--
	# define the namespace the replication system uses in our example it is
	# slony_example
	#--
	cluster name = $CLUSTERNAME;

	#--
	# admin conninfo's are used by slonik to connect to the nodes one for each
	# node on each side of the cluster, the syntax is that of PQconnectdb in
	# the C-API
	# --
	node 1 admin conninfo = 'dbname=$MASTERDBNAME host=$MASTERHOST port=$MASTERDBPORT 
	user=$REPLICATIONUSER';
	node 2 admin conninfo = 'dbname=$SLAVEDBNAME host=$SLAVEHOST port=$SLAVEDBPORT 
	user=$REPLICATIONUSER';

	#--
	# init the first node.  Its id MUST be 1.  This creates the schema
	# _$CLUSTERNAME containing all replication system specific database
	# objects.

	#--
	init cluster ( id=1, comment = 'Master Node');
 
	#--
	# Slony-I organizes tables into sets.  The smallest unit a node can
	# subscribe is a set.  The following commands create one set containing
	# all 4 pgbench tables.  The master or origin of the set is node 1.
	#--
	create set (id=1, origin=1, comment='All edb sample tables');
	set add table (set id=1, origin=1, id=1, fully qualified name = 'public.dept', 
	comment='dept table');
	set add table (set id=1, origin=1, id=2, fully qualified name = 'public.emp', 
	comment='emp table');
	set add table (set id=1, origin=1, id=3, fully qualified name = 'public.jobhist', 
	comment='jobhist table');

	#--
	# Create the second node (the slave) tell the 2 nodes how to connect to
	# each other and how they should listen for events.
	#--

	store node (id=2, comment = 'Slave node');
	store path (server = 1, client = 2, conninfo='dbname=$MASTERDBNAME host=$MASTERHOST 
	port=$MASTERDBPORT user=$REPLICATIONUSER');
	store path (server = 2, client = 1, conninfo='dbname=$SLAVEDBNAME host=$SLAVEHOST 
	port=$SLAVEDBPORT  user=$REPLICATIONUSER');
	store listen (origin=1, provider = 1, receiver =2);
	store listen (origin=2, provider = 2, receiver =1);

Save the above script as "edb_cluster.slonik" file. The above script is to be executed on the master node. You can run this script by using the following command:

     Slonik edb_cluster.sh
     

By running the script you are doing the following:

  1. initializing a cluster for replication

  2. identifying the nodes that are to be part of the cluster

  3. creating a sample set for replication

  4. adding the emp, dept and jobhist tables to this sample set

  5. storing the path and connectivity information for the nodes in the cluster

  6. enabling the listening of events on both the slave and master nodes

At this point we have 2 databases that are fully prepared for replication. It's now time to start the replication daemons. On the $MASTERHOST the command to start the replication engine is:

edb-replication edb_asyncreplic_example "dbname=edb_master user=enterprisedb host=localhost 
port=5444"     
     

After running this command you should see something like the following (on the master node):

  
CONFIG main: edb-replication version 1.1.0 starting up
CONFIG main: local node id = 1
CONFIG main: loading current cluster configuration
CONFIG storeNode: no_id=2 no_comment='Slave node'
CONFIG storePath: pa_server=2 pa_client=1 pa_conninfo="dbname=edb_slave host=localhost 
port=5488  user=edb_slony" pa_connretry=10
CONFIG storeListen: li_origin=2 li_receiver=1 li_provider=2
CONFIG storeSet: set_id=1 set_origin=1 set_comment='All edb sample tables'
CONFIG main: configuration complete - starting threads
CONFIG enableNode: no_id=2
   

Likewise we start the replication system on node 2 (the slave):

edb-replication edb_asyncreplic_example "dbname=edb_slave user=enterprisedb host=localhost 
port=5488" 
 

Similarly you should see something like the following upon the execution of this command:

  
CONFIG main: edb-replication version 1.1.0 starting up
CONFIG main: local node id = 2
CONFIG main: loading current cluster configuration
CONFIG storeNode: no_id=1 no_comment='Master Node'
CONFIG storePath: pa_server=1 pa_client=2 pa_conninfo="dbname=edb_master 
host=localhost port=5444 user=edb_slony" pa_connretry=10
CONFIG storeListen: li_origin=1 li_receiver=2 li_provider=1
CONFIG storeSet: set_id=1 set_origin=1 set_comment='All edb sample tables'
WARN   remoteWorker_wakeup: node 1 - no worker thread
CONFIG main: configuration complete - starting threads
CONFIG enableNode: no_id=1

21.2.4.4.1. Running EnterpriseDB Replication as Windows Service

To run this version of EnterpriseDB Replication in your Windows environment, proceed with the following steps:

  • Update PATH environment variable so that it points to EnterpriseDB Replication "bin" folder e.g.

           C:\EnterpriseDB\8.2.1.2\dbserver\bin.
    	

  • Register EnterpriseDB Replication as a Windows service by issuing the following command. It will create a service named as "EnterpriseDB Replication".

           edb-replication.exe -regservice
    	

Now, to configure the replication cluster. Do the following:

  • Create a separate conf file for the Master and all Slave nodes that defines the connectivity information for the target database. Save the following sample as "master.conf.sample" under the "sample" sub-folder, and make the changes as per your database server configuration.

    	# Sets how many cleanup cycles to run before a vacuum is done.
    # Range: [0,100], default: 3
    #vac_frequency=3
    
    # Debug log level (higher value ==> more output).  Range: [0,4], default 0
    #log_level=0
    
    # Check for updates at least this often in milliseconds.
    # Range: [10-60000], default 100
    #sync_interval=100
    
    # Maximum amount of time in milliseconds before issuing a SYNC event, 
    # This prevents a possible race condition in which the action sequence 
    # is bumped by the trigger while inserting the log row, which makes 
    # this bump is immediately visible to the sync thread, but 
    # the resulting log rows are not visible yet.  If the sync is picked 
    # up by the subscriber, processed and finished before the transaction 
    # commits, this transaction's changes will not be replicated until the 
    # next SYNC.  But if all application activity suddenly stops, 
    # there will be no more sequence bumps, so the high frequent -s check 
    # won't detect that.  Thus, the need for sync_interval_timeout.
    # Range: [0-120000], default 1000
    #sync_interval_timeout=1000
    
    # Maximum number of SYNC events to group together when/if a subscriber
    # falls behind.  SYNCs are batched only if there are that many available 
    # and if they are contiguous. Every other event type in between leads to 
    # a smaller batch.  And if there is only one SYNC available, even -g60 
    # will apply just that one. As soon as a subscriber catches up, it will 
    # apply every single SYNC by itself.
    # Range:  [0,100], default: 6
    #sync_group_maxsize=6
    
    # If this parameter is 1, messages go both to syslog and the standard 
    # output. A value of 2 sends output only to syslog (some messages will 
    # still go to the standard output/error).  The default is 0, which means 
    # syslog is off.  
    # Range:  [0-2], default: 0
    #syslog=0
    
    # If true, include the process ID on each log line.  Default is false.
    #log_pid=false
    
    # If true, include the timestamp on each log line.  Default is true.
    #log_timestamp=true
    
    # A strftime()-conformant format string for use with log timestamps.
    # Default is '%Y-%m-%d %H:%M:%S %Z'
    #log_timestamp_format='%Y-%m-%d %H:%M:%S %Z'
    
    # Where to write the pid file.  Default:  no pid file
    #pid_file='/path/to/your/pidfile'
    
    # Sets the syslog "facility" to be used when syslog enabled.  Valid 
    # values are LOCAL0, LOCAL1, LOCAL2, LOCAL3, LOCAL4, LOCAL5, LOCAL6, LOCAL7.
    #syslog_facility=LOCAL0
    
    # Set the clsuter name that this instance of slon is running against
    # default is to read it off the command line
    cluster_name='slony_sample'
    
    # Set slon's connection info, default is to read it off the command line
    conn_info='dbname=edb host=localhost port=5444 user=enterprisedb'
    
    # maximum time planned for grouped SYNCs
    # If replication is behind, slon will try to increase numbers of
    # syncs done targetting that they should take this quantity of
    # time to process. in ms
    # Range [10000,600000], default 60000. 
    #desired_sync_time=60000
    
    # Execute the following SQL on each node at slon connect time
    # useful to set logging levels, or to tune the planner/memory
    # settings.  You can specify multiple statements by seperating
    # them with a ;
    #sql_on_connection="SET log_min_duration_statement TO '1000';" 
    	

  • Register the master replication engine against EnterpriseDB Replication service by running the following command. Change the target path for your conf file accordingly. Repeat the command to register all slave engines by changing the specific conf file at the command-line.

          edb-replication.exe -addengine C:\EDB-Replication\master.conf
    	

  • Create slonik scripts for cluster initialization and set subscription.Make sure to empty out the "edb" tables on each of the slaves to verify if the data is replicated successfully.

21.2.4.5. EnterpriseDB Replication Subscribtion

Even though we have the edb-replication process running on both the master and slave, and they are both spitting out diagnostics and other messages, we aren't replicating any data yet. The notices you are seeing is the synchronization of cluster configurations between the 2 EnterpriseDB Replication processes (you can see these messages in the window(s) running the edb-replication process.

To start replicating the data in the 3 sample edb tables (defined under set 1) from the master (node id 1) to the slave (node id 2), save the following script as "subscribe_set.slonik" and run it.

#!/bin/sh

	 # ----
	 # This defines which namespace the replication system uses
	 # ----
	 cluster name = $CLUSTERNAME;

	 # ----
	 # Admin conninfo's are used by the slonik program to connect
	 # to the node databases.  So these are the PQconnectdb arguments
	 # that connect from the administrators workstation (where
	 # slonik is executed).
	 # ----
	 node 1 admin conninfo = 'dbname=$MASTERDBNAME host=$MASTERHOST 
	 user=$REPLICATIONUSER';
	 node 2 admin conninfo = 'dbname=$SLAVEDBNAME host=$SLAVEHOST 
	 user=$REPLICATIONUSER';

	 # ----
	 # Node 2 subscribes set 1
	 # ----
	 subscribe set ( id = 1, provider = 1, receiver = 2, forward = no);

Any second now, the replication daemon on $SLAVEHOST will start to copy the current content of all 3 replicated tables. And you should see the following output appended on the window running the the edb-replication process on the slave node(s) indicating that the slave node(s) has successfully subscribed to the master.

CONFIG storeSubscribe: sub_set=1 sub_provider=1 sub_forward='f'
CONFIG enableSubscription: sub_set=1

You have now successfully set up your first basic master/slave replication system, and the 2 databases should, once the slave has caught up, contain identical data.

21.2.4.6. Manual Step required for Post-Restore Operation

You can backup and restore any of the nodes in your replication cluster, in which event you need to execute a pl/pgsql function named "updateRepRelOID" against the cluster database on the specific node.

We need to execute this function in order to update the old OID's values stored in "sl_table" corresponding to tables (in the replication cluster schema) with new OID's after a replicated database is restored from a backup. The OID's are represented by the "reloid" column in the sl_table table. If these entries are incorrect, replication will fail to work, so make sure you run the specific function on each of the cluster nodes where the database is restored via backup file.

You can run the "updateRepRelOID" function via edb-psql. If the name of your replication cluster is _edbrep_example you would run the following command:

SELECT _edbrep_example.updateRepRelOID();
	

This will update the "reloid" column entries in sl_table table with new OID values by querying "pg_class" table.

 
 ©2004-2007 EnterpriseDB All Rights Reserved