Upgrade Issues

Tom Eastep

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover, and with no Back-Cover Texts. A copy of the license is included in the section entitled “GNU Free Documentation License”.

2007/07/19

Table of Contents

Important
Versions >= 4.0.0-Beta7
Versions >= 3.4.0-Beta1
Version >= 3.2.0
Version >= 3.0.0
Version >= 2.4.0

Important

It is important that you read all of the sections on this page where the version number mentioned in the section title is later than what you are currently running.

In the descriptions that follows, the term group refers to a particular network or subnetwork (which may be 0.0.0.0/0 or it may be a host address) accessed through a particular interface.

Examples:

eth0:0.0.0.0/0

eth2:192.168.1.0/24

eth3:192.0.2.123

You can use the shorewall check command to see the groups associated with each of your zones.

Versions >= 4.0.0-Beta7

Beginning with Shorewall 4.0.0, there is no single 'shorewall' package. Rather there are two compiler packages (shorewall-shell and shorewall-perl) and a set of base files (shorewall-common) required by either compiler package.
Although the names of the packages are changing, you can upgrade without having to uninstall/reinstall.
To repeat: You do not need to uninstall any existing package.
If you attempt to upgrade using the shorewall-common RPM, you get this result:
```
gateway:~ # rpm -Uvh shorewall-common-4.0.0.noarch.rpm 
error: Failed dependencies:
shorewall_compiler is needed by shorewall-common-4.0.0-1.noarch
gateway:~ #
```
You must either:
```
rpm -Uvh shorewall-shell-4.0.0.noarch.rpm shorewall-common-4.0.0.noarch.rpm
```
or
```
rpm -Uvh shorewall-shell-4.0.0.noarch.rpm shorewall-perl-4.0.0.noarch.rpm shorewall-common-4.0.0.noarch.rpm
```
If you don't want shorewall-shell, you must use the second command (installing both shorewall-shell and shorewall-perl) then remove shorewall-shell using this command:
```
rpm -e shorewall-shell
```
If you are upgrading using the tarball, you must install shorewall-shell and/or shorewall-perl before you upgrade using shorewall-common. Otherwise, the install.sh script fails with:
ERROR: No Shorewall compiler is installed
The shorewall-shell and shorewall-perl packages are installed from the tarball in the expected way; untar the package, and run the install.sh script.
Example 1: You have 'shorewall' installed and you want to continue to use the shorewall-shell compiler.
```
tar -jxf shorewall-common-4.0.0.tar.bz2
tar -jxf shorewall-shell-4.0.0.tar.bz2

pushd shorewall-shell-4.0.0
./install.sh
popd
pushd shorewall-common-4.0.0
./install.sh
shorewall check
shorewall restart
```
Example 2: You have shorewall 3.4.4 and shorewall-perl 4.0.0-Beta7 installed and you want to upgrade to 4.0. You do not need the shell-based compiler.
```
tar -jxf shorewall-common-4.0.0.tar.bz2
tar -jxf shorewall-perl-4.0.0.tar.bz2

pushd shorewall-perl-4.0.0
./install.sh
popd
pushd /shorewall-common-4.0.0
./install.sh
shorewall check
shorewall restart
```
The RPMs are set up so that if you upgrade an existing Shorewall installation as part of a distribution upgrade and you have not already installed shorewall-perl, then you will end up with Shorewall-common and Shorewall-shell installed.
The ROUTE_FILTER and LOG_MARTIANS options in shorewall.conf work slightly differently in Shorewall 4.0.0. In prior releases, leaving these options empty was equivalent to setting them to 'No' which caused the corresponding flag in /proc to be reset for all interfaces. Beginning in Shorewall 4.0.0, leaving these options empty causes Shorewall to leave the flags in /proc as they are. You must set the option to 'No' in order to obtain the old behavior.
The :noah option is now the default for ipsec tunnels. Tunnels that use AH (protocol 51) must specify ipsec:ah in the TYPE column.

Versions >= 3.4.0-Beta1

Shorewall supports the notion of "default actions". A default action defines a set of rules that are applied before a policy is enforced. Default actions accomplish two goals:
1. Relieve log congestion. Default actions typically include rules to silently drop or reject traffic that would otherwise be logged when the policy is enforced.
2. Insure correct operation. Default actions can also avoid common pitfalls like dropping connection requests on TCP port 113. If these connections are dropped (rather than rejected) then you may encounter problems connecting to internet services that utilize the AUTH protocol of client authentication.
In prior Shorewall versions, default actions (action.Drop and action.Reject) were defined for DROP and REJECT policies in /usr/share/shorewall/actions.std. These could be overridden in /etc/shorewall/actions.
This approach has two drawbacks:
1. All DROP policies must use the same default action and all REJECT policies must use the same default action.
2. Now that we have modularized action processing, we need a way to define default rules for a policy that does not involve actions.
If you have not overridden the defaults using entries in /etc/shorewall/actions then you need make no changes to migrate to Shorewall version 3.4. If you have overridden either of these entries, then please read on.
The change in version 3.4 is two-fold:
- Four new options have been added to the /etc/shorewall/shorewall.conf file that allow specifying the default action for DROP, REJECT, ACCEPT and QUEUE.
  The options are DROP_DEFAULT, REJECT_DEFAULT, ACCEPT_DEFAULT and QUEUE_DEFAULT.
  DROP_DEFAULT describes the rules to be applied before a connection request is dropped by a DROP policy; REJECT_DEFAULT describes the rules to be applied if a connection request is rejected by a REJECT policy. The other two are similar for ACCEPT and QUEUE policies. The value assigned to these may be:
  1. The name of an action.
  2. The name of a macro.
  3. 'None' or 'none'
  The default values are:
  DROP_DEFAULT="Drop"
  REJECT_DEFAULT="Reject"
  ACCEPT_DEFAULT=none
  QUEUE_DEFAULT=none
  If USE_ACTIONS=Yes, then these values refer to action.Drop and action.Reject respectively. If USE_ACTIONS=No, then these values refer to macro.Drop and macro.Reject.
  If you set the value of either option to "None" then no default action will be used and the default action or macro (if any) must be specified in /etc/shorewall/policy.
- The POLICY column in /etc/shorewall/policy has been extended.
  In /etc/shorewall/policy, when the POLICY is DROP, REJECT, ACCEPT or QUEUE then the policy may be followed by ":" and one of the following:
  1. The word "None" or "none". This causes any default action defined in /etc/shorewall/shorewall.conf to be omitted for this policy.
  2. The name of an action (requires that USE_ACTIONS=Yes in shorewall.conf). That action will be invoked before the policy is enforced.
  3. The name of a macro. The rules in that macro will be applied before the policy is enforced. This does not require USE_ACTIONS=Yes.
Example:
```
#SOURCE         DEST            POLICY          LOG
#                                               LEVEL
loc             net             ACCEPT
net             all             DROP:MyDrop     info
#
# THE FOLLOWING POLICY MUST BE LAST
#
all             all             REJECT:MyReject info
```
The 'Limit' action is now a builtin. If you have 'Limit' listed in /etc/shorewall/actions, remove the entry. Also remove the files /etc/shorewall/action.Limit and/or /etc/shorewall/Limit if you have them.
This issue only applies if you have entries in /etc/shorewall/providers.
Previously, Shorewall has not attempted to undo the changes it has made to the firewall's routing as a result of entries in /etc/shorewall/providers and /etc/shorewall/routes. Beginning with this release, Shorewall will attempt to undo these changes. This change can present a migration issue in that the initial routing configuration when this version of Shorewall is installed has probably been changed by Shorewall already. Hence, when Shorewall restores the original configuration, it will be installing a configuration that the previously-installed version has already modified.
The steps to correcting this after you have installed version 3.4 or later of Shorewall are as follows:
1. shorewall[-lite] stop
2. Remove the files /var/lib/shorewall[-lite]/default_route and /var/lib/shorewall[-lite]/undo_routing if they exist.
3. Either restart networking or reboot.
4. shorewall[-lite] start
This issue only applies if you run Shorewall Lite.
The /etc/shorewall-lite/shorewall.conf file has been renamed /etc/shorewall-lite/shorewall-lite.conf. When you upgrade, your shorewall.conf file will be renamed shorewall-lite.conf.

Version >= 3.2.0

If you are upgrading from version 2.4 or earlier, please read the 3.0.0 upgrade considerations below.
A number of macros have been split into two. The macros affected are:
IMAP
LDAP
NNTP
POP3
SMTP
Each of these macros now handles only traffic on the native (plaintext) port. There is a corresponding macro with S added to the end of the name for the SSL version of the same protocol. Thus each macro results in the insertion of only one port per invocation. The Web macro has not been split, but two new macros, HTTP and HTTPS have been created. The Web macro is deprecated in favour of these new macros, and may be removed from future Shorewall releases.
These changes have been made to ensure no unexpected ports are opened due to the use of macros.
In previous Shorewall releases, DNAT and REDIRECT rules supported a special syntax for exclusion of a subnet from the effect of the rule.
Example:
Z2 is a subzone of Z1:
```
DNAT     Z1!Z2        loc:192.168.1.4        ...
```
That feature has never worked correctly when Z2 is a dynamic zone. Furthermore, now that Shorewall supports exclusion lists, the capability is redundant since the above rule can now be written in the form:
```
DNAT     Z1:!<list of exclusions>   loc:192.168.1.4   ...
```
Beginning with Shorewall 3.2.0, the special exclusion syntax will no longer be supported.
Important if you use the QUEUE target.
In the /etc/shorewall/rules file and in actions, you may now specify 'tcp:syn' in the PROTO column. 'tcp:syn' is equivalent to 'tcp' but also requires that the SYN flag is set and the RST, FIN and ACK flags be off ("--syn" is added to the iptables rule).
As part of this change, Shorewall no longer adds the "--syn" option to TCP rules that specify QUEUE as their target.

Extension Scripts may require change

In previous releases, extension scripts were executed during [re]start by using the Bourne Shell "." operator. In addition to executing commands during [re]start, these scripts had to "save" the commands to be executed during shorewall restore.

This clumsiness has been eliminated in Shorewall 3.2. In Shorewall 3.2, extension scripts are copied in-line into the compiled program and are executed in-line during start, restart and restore. This applies to all extension scripts except those associated with a chain or action -- those extension scripts continue to be processed at compile time.

This new approach has two implications for existing scripts.

It is no longer necessary to save the commands; so functions like 'save_command', 'run_and_save_command' and 'ensure_and_save_command' need no longer be called. The generated program will contain functions with these names:

save_command() - does nothing

run_and_save_command() - runs the passed command

ensure_and_save_command() - runs the passed command and stops the firewall if the command fails.

These functions should provide for transparent migration of scripts that use them until you can get around to eliminating their use completely.

When the extension script is copied into the compiled program, it is indented to line up with the surrounding code. If you have 'awk' installed on your system, the Shorewall compiler will correctly handle line continuation (last character on the line = "\"). If you do not have awk, it will not be possible to use line-continuation in your extension scripts. In no case is it possible to continue a quoted string over multiple lines without having additional whitespace inserted into the string.

Beginning with this release, the way in which packet marking in the PREROUTING chain interracts with the 'track' option in /etc/shorewall/providers has changed in two ways:
1. Packets arriving on a tracked interface are now passed to the PREROUTING marking chain so that they may be marked with a mark other than the 'track' mark (the connection still retains the 'track' mark).
2. When HIGH_ROUTE_MARKS=Yes, you can still clear the mark on packets in the PREROUTING chain (i.e., you can specify a mark value of zero).
Kernel version 2.6.16 introduces 'xtables', a new common packet filtering and connection tracking facility that supports both IPv4 and IPv6. Because a different set of kernel modules must be loaded for xtables, Shorewall now includes two 'modules' files:
1. /usr/share/shorewall/modules -- the former /etc/shorewall/modules
2. /usr/share/shorewall/xmodules -- a new file that support xtables.
If you wish to use the new file, then simply execute this command:
cp -f /usr/share/shorewall/xmodules /etc/shorewall/modules
(Versions >= 3.2.3) Previously, CLASSIFY tcrules were always processed out of the POSTROUTING chain. Beginning with this release, they are processed out of the POSTROUTING chain *except* when the SOURCE is $FW[:<address>] in which case the rule is processed out of the OUTPUT chain.
With correctly-coded rulesets, this change should have no effect. Users having incorrectly-coded tcrules may need to change them.
Example:
```
#MARK/          SOURCE  DEST    PROTO   DEST            SOURCE
#CLASSIFY                               PORTS(S)        PORT(S)
1:110           $FW     eth3    tcp     -               22
```
While the user may have expected this rule to only affect traffic from the firewall itself, the rule was really equivalent to this one:
```
#MARK/    SOURCE        DEST    PROTO   DEST            SOURCE
#CLASSIFY                               PORTS(S)        PORT(S)
1:110     0.0.0.0/0     eth3    tcp     -               22
```
So after this change, the second rule will be required rather than the first if that is what was really wanted.

Version >= 3.0.0

The "monitor" command has been eliminated.
The "DISPLAY" and "COMMENTS" columns in the /etc/shorewall/zones file have been removed and have been replaced by the former columns of the /etc/shorewall/ipsec file. The latter file has been removed.
Additionally the FW option in shorewall.conf has been deprecated and is no longer set to 'fw' by default. New users are expected to define the firewall zone in /etc/shorewall/zones.
Adhering to the principle of least astonishment, the old /etc/shorewall/ipsec file will continue to be supported. A new IPSECFILE variable in /etc/shorewall/shorewall.conf determines the name of the file that Shorewall looks in for IPSEC information. If that variable is not set or is set to the empty value then IPSECFILE=ipsec is assumed. So if you simply upgrade and don't do something idiotic like replace your current shorewall.conf file with the new one, your old configuration will continue to work. A dummy 'ipsec' file is included in the release so that your package manager (e.g., rpm) won't remove your existing file.
The shorewall.conf file included in this release sets IPSECFILE=zones so that new users are expected to use the new zone file format.
The DROPINVALID option has been removed from shorewall.conf. The behavior will be as if DROPINVALID=No had been specified. If you wish to drop invalid state packets, use the dropInvalid built-in action.
The 'nobogons' interface and hosts option as well as the BOGON_LOG_LEVEL option have been eliminated.
Most of the standard actions have been replaced by parameterized macros (see below). So for example, the action.AllowSMTP and action.DropSMTP have been removed an a parameterized macro macro.SMTP has been added to replace them.
In order that current users don't have to immediately update their rules and user-defined actions, Shorewall can substitute an invocation of the a new macro for an existing invocation of one of the old actions. So if your rules file calls AllowSMTP, Shorewall will replace that call with SMTP/ACCEPT. Because this substitution is expensive, it is conditional based on the setting of MAPOLDACTIONS in shorewall.conf. If this option is set to YES or if it is not set (such as if you are using your old shorewall.conf file) then Shorewall will perform the substitution. Once you have converted to use the new macros, you can set MAPOLDACTIONS=No and invocations of those actions will go much quicker during 'shorewall [re]start'.
The STATEDIR variable in /etc/shorewall/shorewall.conf has been removed. STATEDIR is now fixed at /var/lib/shorewall. If you have previously set STATEDIR to another directory, please copy the files from that directory to /var/lib/shorewall/ before [re]starting Shorewall after the upgrade to this version.
The "shorewall status" command now just gives the status of Shorewall (started or not-started). The previous status command has been renamed "dump". The command also shows the state relative to the state diagram at http://shorewall.net/starting_and_stopping_shorewall.htm. In addition to the state, the time and date at which that state was entered is shown.
Note that at least one "shorewall [re]start" must be issued after upgrading to this release before "shorewall status" will show anything but "Unknown" for the state.
The "shorewall forget" command now removes the dynamic blacklist save file (/var/lib/shorewall/save).
In previous versions of Shorewall, the rules generated by entries in /etc/shorewall/tunnels preceded those rules generated by entries in /etc/shorewall/rules. Beginning with this release, the rules generated by entries in the tunnels file will appear *AFTER* the rules generated by the rules file. This may cause you problems if you have REJECT, DENY or CONTINUE rules in your rules file that would cause the tunnel transport packets to not reach the rules that ACCEPT them. See http://www.shorewall.net/VPNBasics.html for information on the rules generated by entries in the tunnels file.
The NEWNOTSYN and LOGNEWNOTSYN options in shorewall.conf have been removed as have the 'newnotsyn' options in /etc/shorewall/interfaces and /etc/shorewall/hosts.
TCP new-not-syn packets may be blocked using the 'dropNotSyn' or 'rejNotSyn' built-in actions.
Example: Reject all new-not-syn packets from the net and log them at the 'info' level.
```
#ACTION          SOURCE           DEST           PROTO
SECTION NEW
rejNotSyn:info   net              all            tcp
```
Note that the rule is added at the front of the NEW section of the rules file.
A new TC_SCRIPT option replaces TC_ENABLED in shorewall.conf. If the option is not set then the internal shaper (tc4shorewall by Arne Bernin) is used. Otherwise, the script named in the variable is used.
Users who currently use an /etc/shorewall/tcstart file and wish to continue to do so should set TC_SCRIPT=/etc/shorewall/tcstart in shorewall.conf.

Version >= 2.4.0

Shorewall now enforces the restriction that mark values used in /etc/shorewall/tcrules are less than 256. If you are using mark values >= 256, you must change your configuration before you upgrade.
The value "ipp2p" is no longer accepted in the PROTO column of the /etc/shorewall/rules file. This support has never worked as intended and cannot be made to work in a consistent way. A "Howto" article on filtering P2P with Shorewall and ipp2p will be forthcoming.
LEAF/Bering packages for 2.4.0 and later releases are not available from shorewall.net. See the LEAF site for those packages.