Shorewall-perl

Tom Eastep

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover, and with no Back-Cover Texts. A copy of the license is included in the section entitled “GNU Free Documentation License”.

2008/05/29



Table of Contents

Shorewall-perl - What is it?
Shorewall-perl - The down side
Incompatibilities
Dependence on Perl
Shorewall-perl - Prerequisites
Shorewall-perl - Installation
Using Shorewall-perl
The Shorewall Perl Modules
/usr/share/shorewall-perl/compiler.pl
Shorewall::Compiler
Shorewall 4.0
Shorewall 4.1 and Later
Shorewall::Chains
Shorewall::Config

Shorewall-perl - What is it?

Shorewall-perl is a companion product to Shorewall.

Shorewall-perl contains a re-implementation of the Shorewall compiler written in Perl. The advantages of using Shorewall-perl over Shorewall-shell (the shell-based compiler included in earlier Shorewall 3.x releases) are:

  • The Shorewall-perl compiler is much faster.

  • The script generated by the compiler uses iptables-restore to instantiate the Netfilter configuration. So it runs much faster than the script generated by the Shorewall-shell compiler.

  • The Shorewall-perl compiler does more thorough checking of the configuration than the Shorewall-shell compiler does.

  • The error messages produced by the compiler are better, more consistent and always include the file name and line number where the error was detected.

  • Going forward, the Shorewall-perl compiler will get all enhancements; the Shorewall-shell compiler will only get those enhancements that are easy to retrofit.

Shorewall-perl - The down side

While there are advantages to using Shorewall-perl, there are also disadvantages.

Incompatibilities

There are a number of incompatibilities between the Shorewall-perl compiler and the earlier one.

  1. The Perl-based compiler requires the following capabilities in your kernel and iptables.

    • addrtype match (Restriction relaxed in Shorewall-perl 4.0.1)

    • multiport match (will not be relaxed)

    These capabilities are in current distributions.

  2. Now that Netfilter has features to deal reasonably with port lists, I see no reason to duplicate those features in Shorewall. The Shorewall-shell compiler goes to great pain (in some cases) to break very long port lists ( > 15 where port ranges in lists count as two ports) into individual rules. In the new compiler, I'm avoiding the ugliness required to do that. The new compiler just generates an error if your list is too long. It will also produce an error if you insert a port range into a port list and you don't have extended multiport support.

  3. BRIDGING=Yes is not supported. The kernel code necessary to support this option was removed in Linux kernel 2.6.20. Alternative bridge support is provided by Shorewall-perl.

  4. The BROADCAST column in the interfaces file is essentially unused if your kernel/iptables has Address Type Match support. If that support is present and you enter anything in this column but '-' or 'detect', you will receive a warning.

  5. The 'refresh' command is now similar to restart with the exceptios that:

    • The command fails if Shorewall is not running.

    • A directory name cannot be specified in the command.

    • The refresh command does not alter the Netfilter configuration except for the static blacklist.

  6. With the shell-based compiler, extension scripts were copied into the compiled script and executed at run-time. In many cases, this approach doesn't work with Shorewall Perl because (almost) the entire ruleset is built by the compiler. As a result, Shorewall-perl runs some extension scripts at compile-time rather than at run-time. Because the compiler is written in Perl, your extension scripts from earlier versions will no longer work.

    The following table summarizes when the various extension scripts are run:

    Compile-time (Must be written in Perl)Run-timeEliminated
    initdoneclearcontinue
    macloginitdone 
    Per-chain (including those associated with actions)start 
     started 
     stop 
     stopped 
     tcclear 

    Compile-time extension scripts are executed using the Perl 'eval `cat <file>`' mechanism. Be sure that each script returns a 'true' value; otherwise, the compiler will assume that the script failed and will abort the compilation.

    When a script is invoked, the $chainref scalar variable will usually hold a reference to a chain table entry.

    $chainref->{name} contains the name of the chain
    $chainref->{table} holds the table name

    To add a rule to the chain:

    add_rule $chainref, the-rule

    Where

    the rule is a scalar argument holding the rule text. Do not include "-A chain-name"

    Example:

    add_rule $chainref, '-j ACCEPT';

    To insert a rule into the chain:

    insert_rule $chainref, rulenum, the-rule

    The log_rule_limit function works like it does in the shell compiler with three exceptions:

    • You pass the chain reference rather than the name of the chain.

    • The commands are 'add' and 'insert' rather than '-A' and '-I'.

    • There is only a single "pass as-is to iptables" argument (so you must quote that part

    Example:

        log_rule_limit
                  'info' , 
                  $chainref , 
                  $chainref->{name},
                  'DROP' , 
                  '',    #Limit
                  '' ,   #Log tag
                  'add'
                  '-p tcp ';         

    Here is an example of an actual initdone script used with Shorewall 3.4:

    run_iptables -t mangle -I PREROUTING -p esp -j MARK --set-mark 0x50
    run_iptables -t filter -I INPUT -p udp --dport 1701 -m mark --mark 0x50 -j ACCEPT
    run_iptables -t filter -I OUTPUT -p udp --sport 1701 -j ACCEPT
    

    Here is the corresponding script used with Shorewall-perl:

    use Shorewall::Chains;
    
    insert_rule $mangle_table->{PREROUTING}, 1, "-p esp -j MARK --set-mark 0x50";
    insert_rule $filter_table->{INPUT},      1, "-p udp --dport 1701 -m mark --mark 0x50 -j ACCEPT";
    insert_rule $filter_table->{OUTPUT},     1, "-p udp --sport 1701 -j ACCEPT";
    
    1;

    The initdone script is unique because the $chainref variable is not set before the script is called. The above script illustrates how the $mangle_table, $filter_table, and $nat_table references can be used to add or insert rules in arbitrary chains.

  7. The /etc/shorewall/tos file now has zone-independent SOURCE and DEST columns as do all other files except the rules and policy files.

    The SOURCE column may be one of the following:

    [all:]<address>[,...]
    [all:]<interface>[:<address>[,...]]
    $FW[:<address>[,...]]

    The DEST column may be one of the following:

    [all:]<address>[,...]
    [all:]<interface>[:<address>[,...]]

    This is a permanent change. The old zone-based rules have never worked right and this is a good time to replace them. I've tried to make the new syntax cover the most common cases without requiring change to existing files. In particular, it will handle the tos file released with Shorewall 1.4 and earlier.

  8. Shorewall-perl insists that ipset names begin with a letter and be composed of alphanumeric characters and underscores (_). When used in a Shorewall configuration file, the name must be preceded by a plus sign (+) as with the shell-based compiler.

    Shorewall is now out of the ipset load/reload business. With scripts generated by the Perl-based Compiler, the Netfilter ruleset is never cleared. That means that there is no opportunity for Shorewall to load/reload your ipsets since that cannot be done while there are any current rules using ipsets.

    So:

    1. Your ipsets must be loaded before Shorewall starts. You are free to try to do that with the following code in /etc/shorewall/start (it works for me; your milage may vary):

      if [ "$COMMAND" = start ]; then
          ipset -U :all: :all:
          ipset -U :all: :default:
          ipset -F
          ipset -X
          ipset -R < /etc/shorewall/ipsets
      fi

      The file /etc/shorewall/ipsets will normally be produced using the ipset -S command.

      The above will work most of the time but will fail in a shorewall stop - shorewall start sequence if you use ipsets in your routestopped file (see below).

    2. Your ipsets may not be reloaded until Shorewall is stopped or cleared.

    3. If you specify ipsets in your routestopped file then Shorewall must be cleared in order to reload your ipsets.

    As a consequence, scripts generated by the Perl-based compiler will ignore /etc/shorewall/ipsets and will issue a warning if you set SAVE_IPSETS=Yes in shorewall.conf.

  9. Because the configuration files (with the exception of /etc/shorewall/params) are now processed by the Shorewall-perl compiler rather than by the shell, only the basic forms of Shell expansion ($variable and ${variable}) are supported. The more exotic forms such as ${variable:=default} are not supported. Both variables defined in /etc/shorewall/params and environmental variables (exported by the shell) can be used in configuration files.

  10. USE_ACTIONS=No is not supported. That option is intended to minimize Shorewall's footprint in embedded applications. As a consequence, Default Macros are not supported.

  11. DELAYBLACKLISTLOAD=Yes is not supported. The entire ruleset is atomically loaded with one execution of iptables-restore.

  12. MAPOLDACTIONS=Yes is not supported. People should have converted to using macros by now.

  13. The pre Shorewall-3.0 format of the zones file is not supported (IPSECFILE=ipsec); neither is the /etc/shorewall/ipsec file.

  14. BLACKLISTNEWONLY=No is not permitted with FASTACCEPT=Yes. This combination doesn't work in previous versions of Shorewall so the Perl-based compiler simply rejects it.

  15. Shorewall-perl has a single rule generator that is used for all rule-oriented files. So it is important that the syntax is consistent between files.

    With shorewall-shell, there is a special syntax in the SOURCE column of /etc/shorewall/masq to designate "all traffic entering the firewall on this interface except...".

    Example:

    #INTERFACE       SOURCE                  ADDRESSES
    eth0             eth1!192.168.4.9        ...

    Shorewall-perl uses syntax that is consistent with the rest of Shorewall:

    #INTERFACE       SOURCE                  ADDRESSES
    eth0             eth1:!192.168.4.9       ...
  16. The 'allowoutUPnP' built-in action is no longer supported. In kernel 2.6.14, the Netfilter team have removed support for '-m owner --owner-cmd' which that action depended on.

  17. The PKTTYPE option is ignored by Shorewall-perl. Shorewall-perl 4.0.0 requires Address type match. Shorewall-perl versions 4.0.1 and later will use Address type match if it is available; otherwise, they will behave as if PKTTYPE=No had been specified.

  18. Shorewall-perl detects dead policy file entries that result when an entry is masked by an earlier more general entry.

    Example:

    #SOURCE DEST     POLICY    LOG LEVEL
    all     all      REJECT    info
    loc     net      ACCEPT
  19. In the SOURCE column of the rules file, when an interface name is followed by a list of IP addresses, the behavior of Shorewall-perl differs from that of Shorewall-shell.

    Example:

    #ACTION SOURCE                                 DEST PROTO DEST
    #                                                         PORT(S)
    ACCEPT  loc:eth0:192.168.1.3,192.168.1.5       $FW  tcp   22

    With Shorewall-shell, this rule accepts SSH connection to the firewall from 192.168.1.3 through eth0 or from 192.168.1.5 through any interface. With Shorewall-perl, the rule accepts SSH connections through eth0 from 192.168.1.3 and through eth0 from 192.168.1.5. Shorewall-shell supports this syntax that gives the same result as Shorewall-perl.

    #ACTION SOURCE                                 DEST PROTO DEST
    #                                                         PORT(S)
    ACCEPT loc:eth0:192.168.1.3,eth0:192.168.1.5   $fw  tcp   22

    Shorewall-perl does not support this alternative syntax.

Dependence on Perl

Shorewall-perl is dependent on Perl (see the next section) which has a large disk footprint. This makes Shorewall-perl less desirable in an embedded environment.

Shorewall-perl - Prerequisites

  • Perl (I use Perl 5.8.8 but other 5.8 or later versions should work fine)

  • Perl Cwd Module

  • Perl File::Basename Module

  • Perl File::Temp Module

  • Perl Getopt::Long Module

  • Perl Carp Module

  • Perl FindBin Module (Shorewall 4.0.3 and later)

  • Perl Scalar::Util Module (Shorewall 4.0.6 and later)

Shorewall-perl - Installation

Either

tar -jxf shorewall-perl-4.0.x.tar.bz2
cd shorewall-perl-4.0.x
./install.sh

or

rpm -ivh shorewall-perl-4.0.x.noarch.rpm

Using Shorewall-perl

If you only install one compiler, then that compiler will be used.

If you install both compilers, then the compiler actually used depends on the SHOREWALL_COMPILER setting in shorewall.conf. The value of this option can be either 'perl' or 'shell'.

If you add 'SHOREWALL_COMPILER=perl' to /etc/shorewall/shorewall.conf then by default, the new compiler will be used on the system. If you add it to shorewall.conf in a separate directory (such as a Shorewall-lite export directory) then the new compiler will only be used when you compile from that directory.

If you only install one compiler, it is suggested that you do not set SHOREWALL_COMPILER.

You may also select the compiler to use on the command line using the 'C option:

'-C shell' means use the shell compiler
'-C perl' means use the perl compiler

The -C option overrides the setting in shorewall.conf.

Example:

shorewall restart -C perl

When the Shorewall-perl compiler has been selected, the params file is processed twice, the second time using the -a option which causes all variables set within the file to be exported automatically by the shell. The Shorewall-perl compiler uses the current environmental variables to perform variable expansion within the other Shorewall configuration files.

The Shorewall Perl Modules

Shorewall's Perl modules are installed in /usr/share/shorewall-perl/Shorewall and the names of the packages are of the form Shorewall::name. So by using this directive

use lib '/usr/share/shorewall-perl';

You can then load the modules via normal Perl use statements.

/usr/share/shorewall-perl/compiler.pl

While the compiler is normally run indirectly using /sbin/shorewall, it can be run directly as well.

compiler.pl [ option ... ] [ filename ]

If a filename is given, then the configuration will be compiled and the output placed in the named file. If filename is not given, then the configuration will simply be syntax checked.

Options are:

-v<verbosity>
--verbosity=<verbosity>

The <verbosity> is a number between 0 and 2 and corresponds to the VERBOSITY setting in shorewall.conf. This setting controls the verbosity of the compiler itself.

Note

The VERBOSITY setting in the shorewall.conf file read by the compiler will determine the default verbosity for the compiled program.

-e
--export

If given, the configuration will be compiled for export to another system.

-d <directory>
--directory=<directory>

If this option is omitted, the configuration in /etc/shorewall is compiled/checked. Otherwise, the configuration in the named directory will be compiled/checked.

-t
--timestamp

If given, each progress message issued by the compiler and by the compiled program will be timestamped.

--debug

If given, when a warning or error message is issued, it is supplimented with a stack trace. Requires the Carp Perl module.

--refresh=<chainlist>

If given, the compiled script's 'refresh' command will refresh the chains in the comma-separated <chainlist> rather than 'blacklst'.

--log=<logfile>

Added in Shorewall 4.1. If given, compiler will log to this file provider that --log_verbosity is > -1.

--log_verbosity=-1|0|1|2

Added in Shorewall 4.1. If given, controls the verbosity of logging to the log specified by the --log parameter.

Example (compiles the configuration in the current directory generating a script named 'firewall' and using VERBOSITY 2).

/usr/share/shorewall-perl/compiler.pl -v 2 -d . firewall

Note

The Perl-based compiler does not process /etc/shorewall/params. To include definitions in that file, you would need to do something like the following:

. /usr/share/shorewall/lib.base  # In case /etc/shorewall/params does INCLUDE
set -a                           # Export all variables set in /etc/shorewall/params
. /etc/shorewall/params
set +a
/usr/share/shorewall-perl/compiler.pl ...

Shorewall::Compiler

Shorewall 4.0

 use lib '/usr/share/shorewall-perl';
 use Shorewall::Compiler;
        
 compiler $filename, $directory, $verbose, $options $chains

Arguments to the compiler are:

$filename

Name of the compiled script to be created. If the arguments evaluates to false, the configuration is syntax checked.

$directory

The directory containing the configuration. If passed as '', then /etc/shorewall/ is assumed.

$verbose

The verbosity level that the compiler will run with (0-2).

Note

The VERBOSITY setting in the shorewall.conf file read by the compiler will determine the default verbosity for the compiled program.

$options

A bitmap of options. Shorewall::Compiler exports three constants to help building this argument:

EXPORT = 0x01
TIMESTAMP = 0x02
DEBUG = 0x04
$chains

A comma-separated list of chains that the generated script's 'refresh' command will reload. If passed as an empty string, then 'blacklist' is assumed.

The compiler raises an exception with 'die' if it encounters an error; $@ contains the 'ERROR' messages describing the problem. The compiler function can be called repeatedly with different inputs.

Shorewall 4.1 and Later

To avoid a proliferation of parameters to Shorewall::Compiler::compile(), that function has been changed to use named parameters. Parameter names are:

object

Object file. If omitted or '', the configuration is syntax checked.

directory

Directory. If omitted or '', configuration files are located using CONFIG_PATH. Otherwise, the directory named by this parameter is searched first.

verbosity

Verbosity; range -1 to 2

timestamp

0|1 -- timestamp messages.

debug

0|1 -- include stack trace in warning/error messages.

export

0|1 -- compile for export.

chains

List of chains to be reloaded by 'refresh'

log

File to log compiler messages to.

log_verbosity

Log Verbosity; range -1 to 2.

Those parameters that are supplied must have defined values. Defaults are:

object '' ('check' command)
directory ''
verbosity 1
timestamp 0
debug 0
export 0
chains ''
log ''
log_verbosity -1

Example:

use lib '/usr/share/shorewall-perl/';
use Shorewall::Compiler;

compiler( object => '/root/firewall', log => '/root/compile.log', log_verbosity => 2 ); 

Shorewall::Chains

use lib '/usr/share/shorewall-perl';
use Shorewall::Chains;

my $chainref1 = chain_new $table, $name1;
add_rule $chainref1, $rule;
insert_rule $chainref1, $ordinal, $rule;
my $chainref2 = new_manual_chain $name3;
my $chainref3 = ensure_manual_chain $name;
log_rule_limit $level, $chainref3, $name, $disposition, $limit, $tag, $command, $predicates;


my $chainref4 = $chain_table{$table}{$name};
my $chainref5 = $nat_table{$name};
my $chainref6 = $mangle_table{$name};
my $chainref7 = $filter_table{$name};

Shorewall::Chains is Shorewall-perl's interface to iptables/netfilter. It creates a chain table (%chain_table) which is populated as the various tables are processed. The table (actually a hash) is two-dimensional with the first dimension being the Netfilter table name (raw, mangle, nat and filter) and the second dimension being the chain name. Each table is a hash reference -- the hash defines the attributes of the chain. See the large comment at the beginning of the module (/usr/share/shorewall-perl/Shorewall/Chains.pm).

The module export the chain table along with three hash references into the table:

$nat_table

Reference to the 'nat' portion of the table ($chain_table{nat}). This is a hash whose key is the chain name.

$mangle_table

Reference to the 'mangle' portion of the table ($chain_table{mangle}). This is a hash whose key is the chain name.

$filter

Reference to the 'filter' portion of the table ($chain_table{filter}). This is a hash whose key is the chain name.

You can create a new chain in any of the tables using new_chain(). Arguments to the function are:

$table

'nat', 'mangle', or 'filter'.

$name

Name of the chain to create.

The function creates a hash at $chain_table{$table}{$name} and populates the hash with default values. A reference to the hash is returned.

Each chain table entry includes a list of rules to be added to the chain. These rules are written to the iptables-restore input file when the resulting script is executed. To append a rule to that list, call add_rule(). Arguments are:

$chainref

A reference to the chain table entry.

$rule

The rule to add. Do not include the leading '-A ' in this argument -- it will be supplied by the function.

To insert a rule into that list, call insert_rule(). Arguments are:

$chainref

A reference to the chain table entry.

$ordinal

The position of the inserted rule in the list. A value of 1 inserts the rule at the head of the list, a value of 2 places the rule second in the list, and so on.

$rule

The rule to add. Do not include the leading '-I' in this argument -- it will be supplied by the function.

To create a manual chain, use the new_manual_chain() function. The function accepts a single argument which is the name of the chain. The function returns a reference to the resulting chain table entry.

A companion function, ensure_manual_chain(), can be called when a manual chain of the desired name may have alread been created. If a manual chain table entry with the passed name already exists, a reference to the chain table entry is returned. Otherwise, the function calls new_manual_chain() and returns the result.

To create a logging rule, call log_rule_limit(). Arguments are:

$level

The log level. May be specified as a name or as a number.

$chainref

Chain table reference for the chain to which the rule is to be added.

$name

The chain name to be reported in the log message (see LOGFORMAT in shorewall.conf(5)).

$disposition

The disposition to be reported in the log message (see LOGFORMAT in shorewall.conf(5)).

$limit

Rate limiting match. If an empty string is passed, the LOGRATE/LOGBURST (shorewall.conf(5)) is used.

$tag

Log tag.

$command

If 'add', append the log rule to the chain. If 'insert', then insert the rule at the beginning of the chain.

$predicates

Any additional matches that are to be applied to the rule.

Shorewall::Config

use lib '/usr/share/shorewall-perl';
use Shorewall::Config;

warning message "This entry is bogus";
fatal_error "You have made an error";

progress_message  "This will only be seen if VERBOSITY >= 2";
progress_message2 "This will only be seen if VERBOSITY >= 1";
progress_message3 "This will be seen unless VERBOSITY < 0";

The shorewall() function may be optionally included.

use lib '/usr/share/shorewall-perl';
use Shorewall::Config qw/shorewall/;

shorewall $config_file_entry;

The Shorewall::Config module provides basic services to Shorewall-perl. By default, it exports the functions that produce progress messages and warning/error messages.

To issue a warning message, call warning_message(). The single argument describes the warning.

To raise a fatal error, call fatal_error(). Again, the single argument described the error.

In both cases, the function will augment the warning/error with the current configuration file and line number, if any. fatal_error() raised an exception via either confess() or die(), depending on whether the debugging stack trace is enabled or not..

The three 'progress message' functions conditionally produce output depending on the current verbosity setting.

The shorewall() function is used by embedded Perl scripts to generate entries to be included in the current configuration file.