Setting the Greenplum Recommended OS Parameters

Greenplum requires the certain operating system (OS) parameters be set on all hosts in your Greenplum Database system (masters and segments).

In general, the following categories of system parameters need to be altered:

Shared Memory - A Greenplum Database instance will not work unless the shared memory segment for your kernel is properly sized. Most default OS installations have the shared memory values set too low for Greenplum Database. On Linux systems, you must also disable the OOM (out of memory) killer. For information about Greenplum Database shared memory requirements, see the Greenplum Database server configuration parameter shared_buffers in the Greenplum Database Reference Guide.
Network - On high-volume Greenplum Database systems, certain network-related tuning parameters must be set to optimize network connections made by the Greenplum interconnect.
User Limits - User limits control the resources available to processes started by a user's shell. Greenplum Database requires a higher limit on the allowed number of file descriptors that a single process can have open. The default settings may cause some Greenplum Database queries to fail because they will run out of file descriptors needed to process the query.

Linux System Settings

Set the following parameters in the /etc/sysctl.conf file and reboot:

kernel.shmmax = 500000000
kernel.shmmni = 4096
kernel.shmall = 4000000000
kernel.sem = 250 512000 100 2048
kernel.sysrq = 1
kernel.core_uses_pid = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.msgmni = 2048
net.ipv4.tcp_syncookies = 1
net.ipv4.ip_forward = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.conf.all.arp_filter = 1
net.ipv4.ip_local_port_range = 1025 65535
net.core.netdev_max_backlog = 10000
net.core.rmem_max = 2097152
net.core.wmem_max = 2097152
vm.overcommit_memory = 2

Set the following parameters in the /etc/security/limits.conf file:
```
* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072
```
For RedHat Enterprise Linux 6.x and CentOS 6.x, parameter values in the /etc/security/limits.d/90-nproc.conf file override the values in the limits.conf file. If a parameter value is set in both conf files, ensure that the parameter is set properly in the 90-nproc.conf file. The Linux module pam_limits sets user limits by reading the values from the limits.conf file and then from the 90-nproc.conf file. For information about PAM and user limits, see the documentation on PAM and pam_limits.
XFS is the preferred file system on Linux platforms for data storage. Greenplum recommends the following XFS mount options:
```
rw,noatime,inode64,allocsize=16m
```
See the manual page (man) for the mount command for more information about using that command (man mount opens the man page).

The XFS options can also be set in the /etc/fstab file. This example entry from an fstab file specifies the XFS options.
```
/dev/data /data xfs nodev,noatime,inode64,allocsize=16m 0 0
```
The Linux disk I/O scheduler for disk access supports different policies, such as CFQ, AS, and deadline.
Pivotal recommends the following scheduler option: deadline. To specify a scheduler until the next system reboot, run the following:
```
# echo schedulername > /sys/block/devname/queue/scheduler
```
For example:
```
# echo deadline > /sys/block/sbd/queue/scheduler
```
You can specify the I/O scheduler at boot time with the elevator kernel parameter. Add the parameter elevator=deadline to the kernel command in the file /boot/grub/grub.conf, the GRUB boot loader configuration file. This is an example kernel command from a grub.conf file. The command is on multiple lines for readability.
```
kernel /vmlinuz-2.6.18-274.3.1.el5 ro root=LABEL=/
    elevator=deadline crashkernel=128M@16M  quiet console=tty1
    console=ttyS1,115200 panic=30 transparent_hugepage=never 
    initrd /initrd-2.6.18-274.3.1.el5.img
```
After adding the parameter, reboot the system.
Each disk device file should have a read-ahead (blockdev) value of 16384.
To verify the read-ahead value of a disk device:
```
# /sbin/blockdev --getra devname
```
For example:
```
# /sbin/blockdev --getra /dev/sdb
```
To set blockdev (read-ahead) on a device:
```
# /sbin/blockdev --setra bytes devname
```
For example:
```
# /sbin/blockdev --setra 16384 /dev/sdb
```
See the manual page (man) for the blockdev command for more information about using that command (man blockdev opens the man page).
Edit the /etc/hosts file and make sure that it includes the host names and all interface address names for every machine participating in your Greenplum Database system.
Disable Transparent Huge Pages (THP). RedHat Enterprise Linux 6.0 or higher enables THP by default. THP degrades Greenplum Database performance. One way to disable THP is by adding the parameter transparent_hugepage=never to the kernel command in the file /boot/grub/grub.conf, the GRUB boot loader configuration file. This is an example kernel command from a grub.conf file. The command is on multiple lines for readability:
```
kernel /vmlinuz-2.6.18-274.3.1.el5 ro root=LABEL=/
    elevator=deadline crashkernel=128M@16M  quiet console=tty1
    console=ttyS1,115200 panic=30 transparent_hugepage=never 
    initrd /initrd-2.6.18-274.3.1.el5.img
```
After adding the parameter, reboot the system.
This cat command checks the state of THP. The output indicates that THP is disabled.
```
$ cat /sys/kernel/mm/redhat_transparent_hugepage/enabled
always [never]
```
For more information about Transparent Huge Pages, see your operating system documentation.

Mac OS X System Settings

Add the following to /etc/sysctl.conf:

kern.sysv.shmmax=2147483648 
kern.sysv.shmmin=1 
kern.sysv.shmmni=64 
kern.sysv.shmseg=16 
kern.sysv.shmall=524288 
kern.maxfiles=65535 
kern.maxfilesperproc=65535 
net.inet.tcp.msl=60

Add the following line to /etc/hostconfig:
```
HOSTNAME="your_hostname"
```