2345 lines
89 KiB
Plaintext
2345 lines
89 KiB
Plaintext
<!-- doc/src/sgml/runtime.sgml -->
|
|
|
|
<chapter id="runtime">
|
|
<title>Server Setup and Operation</title>
|
|
|
|
<para>
|
|
This chapter discusses how to set up and run the database server
|
|
and its interactions with the operating system.
|
|
</para>
|
|
|
|
<sect1 id="postgres-user">
|
|
<title>The <productname>PostgreSQL</productname> User Account</title>
|
|
|
|
<indexterm>
|
|
<primary>postgres user</primary>
|
|
</indexterm>
|
|
|
|
<para>
|
|
As with any server daemon that is accessible to the outside world,
|
|
it is advisable to run <productname>PostgreSQL</productname> under a
|
|
separate user account. This user account should only own the data
|
|
that is managed by the server, and should not be shared with other
|
|
daemons. (For example, using the user <literal>nobody</literal> is a bad
|
|
idea.) It is not advisable to install executables owned by this
|
|
user because compromised systems could then modify their own
|
|
binaries.
|
|
</para>
|
|
|
|
<para>
|
|
To add a Unix user account to your system, look for a command
|
|
<command>useradd</command> or <command>adduser</command>. The user
|
|
name <systemitem>postgres</systemitem> is often used, and is assumed
|
|
throughout this book, but you can use another name if you like.
|
|
</para>
|
|
</sect1>
|
|
|
|
<sect1 id="creating-cluster">
|
|
<title>Creating a Database Cluster</title>
|
|
|
|
<indexterm>
|
|
<primary>database cluster</primary>
|
|
</indexterm>
|
|
|
|
<indexterm>
|
|
<primary>data area</primary>
|
|
<see>database cluster</see>
|
|
</indexterm>
|
|
|
|
<para>
|
|
Before you can do anything, you must initialize a database storage
|
|
area on disk. We call this a <firstterm>database cluster</firstterm>.
|
|
(<acronym>SQL</acronym> uses the term catalog cluster.) A
|
|
database cluster is a collection of databases that is managed by a
|
|
single instance of a running database server. After initialization, a
|
|
database cluster will contain a database named <literal>postgres</literal>,
|
|
which is meant as a default database for use by utilities, users and third
|
|
party applications. The database server itself does not require the
|
|
<literal>postgres</literal> database to exist, but many external utility
|
|
programs assume it exists. Another database created within each cluster
|
|
during initialization is called
|
|
<literal>template1</literal>. As the name suggests, this will be used
|
|
as a template for subsequently created databases; it should not be
|
|
used for actual work. (See <xref linkend="managing-databases"> for
|
|
information about creating new databases within a cluster.)
|
|
</para>
|
|
|
|
<para>
|
|
In file system terms, a database cluster will be a single directory
|
|
under which all data will be stored. We call this the <firstterm>data
|
|
directory</firstterm> or <firstterm>data area</firstterm>. It is
|
|
completely up to you where you choose to store your data. There is no
|
|
default, although locations such as
|
|
<filename>/usr/local/pgsql/data</filename> or
|
|
<filename>/var/lib/pgsql/data</filename> are popular. To initialize a
|
|
database cluster, use the command <xref
|
|
linkend="app-initdb">,<indexterm><primary>initdb</></> which is
|
|
installed with <productname>PostgreSQL</productname>. The desired
|
|
file system location of your database cluster is indicated by the
|
|
<option>-D</option> option, for example:
|
|
<screen>
|
|
<prompt>$</> <userinput>initdb -D /usr/local/pgsql/data</userinput>
|
|
</screen>
|
|
Note that you must execute this command while logged into the
|
|
<productname>PostgreSQL</productname> user account, which is
|
|
described in the previous section.
|
|
</para>
|
|
|
|
<tip>
|
|
<para>
|
|
As an alternative to the <option>-D</option> option, you can set
|
|
the environment variable <envar>PGDATA</envar>.
|
|
<indexterm><primary><envar>PGDATA</envar></primary></indexterm>
|
|
</para>
|
|
</tip>
|
|
|
|
<para>
|
|
Alternatively, you can run <command>initdb</command> via
|
|
the <xref linkend="app-pg-ctl">
|
|
program<indexterm><primary>pg_ctl</></> like so:
|
|
<screen>
|
|
<prompt>$</> <userinput>pg_ctl -D /usr/local/pgsql/data initdb</userinput>
|
|
</screen>
|
|
This may be more intuitive if you are
|
|
using <command>pg_ctl</command> for starting and stopping the
|
|
server (see <xref linkend="server-start">), so
|
|
that <command>pg_ctl</command> would be the sole command you use
|
|
for managing the database server instance.
|
|
</para>
|
|
|
|
<para>
|
|
<command>initdb</command> will attempt to create the directory you
|
|
specify if it does not already exist. It is likely that it will not
|
|
have the permission to do so (if you followed our advice and created
|
|
an unprivileged account). In that case you should create the
|
|
directory yourself (as root) and change the owner to be the
|
|
<productname>PostgreSQL</productname> user. Here is how this might
|
|
be done:
|
|
<screen>
|
|
root# <userinput>mkdir /usr/local/pgsql/data</userinput>
|
|
root# <userinput>chown postgres /usr/local/pgsql/data</userinput>
|
|
root# <userinput>su postgres</userinput>
|
|
postgres$ <userinput>initdb -D /usr/local/pgsql/data</userinput>
|
|
</screen>
|
|
</para>
|
|
|
|
<para>
|
|
<command>initdb</command> will refuse to run if the data directory
|
|
looks like it has already been initialized.</para>
|
|
|
|
<para>
|
|
Because the data directory contains all the data stored in the
|
|
database, it is essential that it be secured from unauthorized
|
|
access. <command>initdb</command> therefore revokes access
|
|
permissions from everyone but the
|
|
<productname>PostgreSQL</productname> user.
|
|
</para>
|
|
|
|
<para>
|
|
However, while the directory contents are secure, the default
|
|
client authentication setup allows any local user to connect to the
|
|
database and even become the database superuser. If you do not
|
|
trust other local users, we recommend you use one of
|
|
<command>initdb</command>'s <option>-W</option>, <option>--pwprompt</option>
|
|
or <option>--pwfile</option> options to assign a password to the
|
|
database superuser.<indexterm>
|
|
<primary>password</>
|
|
<secondary>of the superuser</>
|
|
</indexterm>
|
|
Also, specify <option>-A md5</> or
|
|
<option>-A password</> so that the default <literal>trust</> authentication
|
|
mode is not used; or modify the generated <filename>pg_hba.conf</filename>
|
|
file after running <command>initdb</command>, but
|
|
<emphasis>before</> you start the server for the first time. (Other
|
|
reasonable approaches include using <literal>peer</literal> authentication
|
|
or file system permissions to restrict connections. See <xref
|
|
linkend="client-authentication"> for more information.)
|
|
</para>
|
|
|
|
<para>
|
|
<command>initdb</command> also initializes the default
|
|
locale<indexterm><primary>locale</></> for the database cluster.
|
|
Normally, it will just take the locale settings in the environment
|
|
and apply them to the initialized database. It is possible to
|
|
specify a different locale for the database; more information about
|
|
that can be found in <xref linkend="locale">. The default sort order used
|
|
within the particular database cluster is set by
|
|
<command>initdb</command>, and while you can create new databases using
|
|
different sort order, the order used in the template databases that initdb
|
|
creates cannot be changed without dropping and recreating them.
|
|
There is also a performance impact for using locales
|
|
other than <literal>C</> or <literal>POSIX</>. Therefore, it is
|
|
important to make this choice correctly the first time.
|
|
</para>
|
|
|
|
<para>
|
|
<command>initdb</command> also sets the default character set encoding
|
|
for the database cluster. Normally this should be chosen to match the
|
|
locale setting. For details see <xref linkend="multibyte">.
|
|
</para>
|
|
|
|
<sect2 id="creating-cluster-nfs">
|
|
<title>Network File Systems</title>
|
|
|
|
<indexterm zone="creating-cluster-nfs">
|
|
<primary>Network File Systems</primary>
|
|
</indexterm>
|
|
<indexterm><primary><acronym>NFS</></><see>Network File Systems</></>
|
|
<indexterm><primary>Network Attached Storage (<acronym>NAS</>)</><see>Network File Systems</></>
|
|
|
|
<para>
|
|
Many installations create database clusters on network file systems.
|
|
Sometimes this is done directly via <acronym>NFS</>, or by using a
|
|
Network Attached Storage (<acronym>NAS</>) device that uses
|
|
<acronym>NFS</> internally. <productname>PostgreSQL</> does nothing
|
|
special for <acronym>NFS</> file systems, meaning it assumes
|
|
<acronym>NFS</> behaves exactly like locally-connected drives
|
|
(<acronym>DAS</>, Direct Attached Storage). If client and server
|
|
<acronym>NFS</> implementations have non-standard semantics, this can
|
|
cause reliability problems (see <ulink
|
|
url="http://www.time-travellers.org/shane/papers/NFS_considered_harmful.html"></ulink>).
|
|
Specifically, delayed (asynchronous) writes to the <acronym>NFS</>
|
|
server can cause reliability problems; if possible, mount
|
|
<acronym>NFS</> file systems synchronously (without caching) to avoid
|
|
this. Also, soft-mounting <acronym>NFS</> is not recommended.
|
|
(Storage Area Networks (<acronym>SAN</>) use a low-level
|
|
communication protocol rather than <acronym>NFS</>.)
|
|
</para>
|
|
|
|
</sect2>
|
|
|
|
</sect1>
|
|
|
|
<sect1 id="server-start">
|
|
<title>Starting the Database Server</title>
|
|
|
|
<para>
|
|
Before anyone can access the database, you must start the database
|
|
server. The database server program is called
|
|
<command>postgres</command>.<indexterm><primary>postgres</></>
|
|
The <command>postgres</command> program must know where to
|
|
find the data it is supposed to use. This is done with the
|
|
<option>-D</option> option. Thus, the simplest way to start the
|
|
server is:
|
|
<screen>
|
|
$ <userinput>postgres -D /usr/local/pgsql/data</userinput>
|
|
</screen>
|
|
which will leave the server running in the foreground. This must be
|
|
done while logged into the <productname>PostgreSQL</productname> user
|
|
account. Without <option>-D</option>, the server will try to use
|
|
the data directory named by the environment variable <envar>PGDATA</envar>.
|
|
If that variable is not provided either, it will fail.
|
|
</para>
|
|
|
|
<para>
|
|
Normally it is better to start <command>postgres</command> in the
|
|
background. For this, use the usual Unix shell syntax:
|
|
<screen>
|
|
$ <userinput>postgres -D /usr/local/pgsql/data >logfile 2>&1 &</userinput>
|
|
</screen>
|
|
It is important to store the server's <systemitem>stdout</> and
|
|
<systemitem>stderr</> output somewhere, as shown above. It will help
|
|
for auditing purposes and to diagnose problems. (See <xref
|
|
linkend="logfile-maintenance"> for a more thorough discussion of log
|
|
file handling.)
|
|
</para>
|
|
|
|
<para>
|
|
The <command>postgres</command> program also takes a number of other
|
|
command-line options. For more information, see the
|
|
<xref linkend="app-postgres"> reference page
|
|
and <xref linkend="runtime-config"> below.
|
|
</para>
|
|
|
|
<para>
|
|
This shell syntax can get tedious quickly. Therefore the wrapper
|
|
program
|
|
<xref linkend="app-pg-ctl"><indexterm><primary>pg_ctl</primary></indexterm>
|
|
is provided to simplify some tasks. For example:
|
|
<programlisting>
|
|
pg_ctl start -l logfile
|
|
</programlisting>
|
|
will start the server in the background and put the output into the
|
|
named log file. The <option>-D</option> option has the same meaning
|
|
here as for <command>postgres</command>. <command>pg_ctl</command>
|
|
is also capable of stopping the server.
|
|
</para>
|
|
|
|
<para>
|
|
Normally, you will want to start the database server when the
|
|
computer boots.<indexterm>
|
|
<primary>booting</>
|
|
<secondary>starting the server during</>
|
|
</indexterm>
|
|
Autostart scripts are operating-system-specific.
|
|
There are a few distributed with
|
|
<productname>PostgreSQL</productname> in the
|
|
<filename>contrib/start-scripts</> directory. Installing one will require
|
|
root privileges.
|
|
</para>
|
|
|
|
<para>
|
|
Different systems have different conventions for starting up daemons
|
|
at boot time. Many systems have a file
|
|
<filename>/etc/rc.local</filename> or
|
|
<filename>/etc/rc.d/rc.local</filename>. Others use <filename>init.d</filename> or
|
|
<filename>rc.d</> directories. Whatever you do, the server must be
|
|
run by the <productname>PostgreSQL</productname> user account
|
|
<emphasis>and not by root</emphasis> or any other user. Therefore you
|
|
probably should form your commands using <literal>su -c '...'
|
|
postgres</literal>. For example:
|
|
<programlisting>
|
|
su -c 'pg_ctl start -D /usr/local/pgsql/data -l serverlog' postgres
|
|
</programlisting>
|
|
</para>
|
|
|
|
<para>
|
|
Here are a few more operating-system-specific suggestions. (In each
|
|
case be sure to use the proper installation directory and user
|
|
name where we show generic values.)
|
|
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>
|
|
For <productname>FreeBSD</productname>, look at the file
|
|
<filename>contrib/start-scripts/freebsd</filename> in the
|
|
<productname>PostgreSQL</productname> source distribution.
|
|
<indexterm><primary>FreeBSD</><secondary>start script</secondary></>
|
|
</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>
|
|
On <productname>OpenBSD</productname>, add the following lines
|
|
to the file <filename>/etc/rc.local</filename>:
|
|
<indexterm><primary>OpenBSD</><secondary>start script</secondary></>
|
|
<programlisting>
|
|
if [ -x /usr/local/pgsql/bin/pg_ctl -a -x /usr/local/pgsql/bin/postgres ]; then
|
|
su - -c '/usr/local/pgsql/bin/pg_ctl start -l /var/postgresql/log -s' postgres
|
|
echo -n ' postgresql'
|
|
fi
|
|
</programlisting>
|
|
</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>
|
|
On <productname>Linux</productname> systems either add
|
|
<indexterm><primary>Linux</><secondary>start script</secondary></>
|
|
<programlisting>
|
|
/usr/local/pgsql/bin/pg_ctl start -l logfile -D /usr/local/pgsql/data
|
|
</programlisting>
|
|
to <filename>/etc/rc.d/rc.local</filename>
|
|
or <filename>/etc/rc.local</filename> or look at the file
|
|
<filename>contrib/start-scripts/linux</filename> in the
|
|
<productname>PostgreSQL</productname> source distribution.
|
|
</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>
|
|
On <productname>NetBSD</productname>, either use the
|
|
<productname>FreeBSD</productname> or
|
|
<productname>Linux</productname> start scripts, depending on
|
|
preference. <indexterm><primary>NetBSD</><secondary>start script</secondary></>
|
|
</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>
|
|
On <productname>Solaris</productname>, create a file called
|
|
<filename>/etc/init.d/postgresql</filename> that contains
|
|
the following line:
|
|
<indexterm><primary>Solaris</><secondary>start script</secondary></>
|
|
<programlisting>
|
|
su - postgres -c "/usr/local/pgsql/bin/pg_ctl start -l logfile -D /usr/local/pgsql/data"
|
|
</programlisting>
|
|
Then, create a symbolic link to it in <filename>/etc/rc3.d</> as
|
|
<filename>S99postgresql</>.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
|
|
</para>
|
|
|
|
<para>
|
|
While the server is running, its
|
|
<acronym>PID</acronym> is stored in the file
|
|
<filename>postmaster.pid</filename> in the data directory. This is
|
|
used to prevent multiple server instances from
|
|
running in the same data directory and can also be used for
|
|
shutting down the server.
|
|
</para>
|
|
|
|
<sect2 id="server-start-failures">
|
|
<title>Server Start-up Failures</title>
|
|
|
|
<para>
|
|
There are several common reasons the server might fail to
|
|
start. Check the server's log file, or start it by hand (without
|
|
redirecting standard output or standard error) and see what error
|
|
messages appear. Below we explain some of the most common error
|
|
messages in more detail.
|
|
</para>
|
|
|
|
<para>
|
|
<screen>
|
|
LOG: could not bind IPv4 socket: Address already in use
|
|
HINT: Is another postmaster already running on port 5432? If not, wait a few seconds and retry.
|
|
FATAL: could not create TCP/IP listen socket
|
|
</screen>
|
|
This usually means just what it suggests: you tried to start
|
|
another server on the same port where one is already running.
|
|
However, if the kernel error message is not <computeroutput>Address
|
|
already in use</computeroutput> or some variant of that, there might
|
|
be a different problem. For example, trying to start a server
|
|
on a reserved port number might draw something like:
|
|
<screen>
|
|
$ <userinput>postgres -p 666</userinput>
|
|
LOG: could not bind IPv4 socket: Permission denied
|
|
HINT: Is another postmaster already running on port 666? If not, wait a few seconds and retry.
|
|
FATAL: could not create TCP/IP listen socket
|
|
</screen>
|
|
</para>
|
|
|
|
<para>
|
|
A message like:
|
|
<screen>
|
|
FATAL: could not create shared memory segment: Invalid argument
|
|
DETAIL: Failed system call was shmget(key=5440001, size=4011376640, 03600).
|
|
</screen>
|
|
probably means your kernel's limit on the size of shared memory is
|
|
smaller than the work area <productname>PostgreSQL</productname>
|
|
is trying to create (4011376640 bytes in this example). Or it could
|
|
mean that you do not have System-V-style shared memory support
|
|
configured into your kernel at all. As a temporary workaround, you
|
|
can try starting the server with a smaller-than-normal number of
|
|
buffers (<xref linkend="guc-shared-buffers">). You will eventually want
|
|
to reconfigure your kernel to increase the allowed shared memory
|
|
size. You might also see this message when trying to start multiple
|
|
servers on the same machine, if their total space requested
|
|
exceeds the kernel limit.
|
|
</para>
|
|
|
|
<para>
|
|
An error like:
|
|
<screen>
|
|
FATAL: could not create semaphores: No space left on device
|
|
DETAIL: Failed system call was semget(5440126, 17, 03600).
|
|
</screen>
|
|
does <emphasis>not</emphasis> mean you've run out of disk
|
|
space. It means your kernel's limit on the number of <systemitem
|
|
class="osname">System V</> semaphores is smaller than the number
|
|
<productname>PostgreSQL</productname> wants to create. As above,
|
|
you might be able to work around the problem by starting the
|
|
server with a reduced number of allowed connections
|
|
(<xref linkend="guc-max-connections">), but you'll eventually want to
|
|
increase the kernel limit.
|
|
</para>
|
|
|
|
<para>
|
|
If you get an <quote>illegal system call</> error, it is likely that
|
|
shared memory or semaphores are not supported in your kernel at
|
|
all. In that case your only option is to reconfigure the kernel to
|
|
enable these features.
|
|
</para>
|
|
|
|
<para>
|
|
Details about configuring <systemitem class="osname">System V</>
|
|
<acronym>IPC</> facilities are given in <xref linkend="sysvipc">.
|
|
</para>
|
|
</sect2>
|
|
|
|
<sect2 id="client-connection-problems">
|
|
<title>Client Connection Problems</title>
|
|
|
|
<para>
|
|
Although the error conditions possible on the client side are quite
|
|
varied and application-dependent, a few of them might be directly
|
|
related to how the server was started. Conditions other than
|
|
those shown below should be documented with the respective client
|
|
application.
|
|
</para>
|
|
|
|
<para>
|
|
<screen>
|
|
psql: could not connect to server: Connection refused
|
|
Is the server running on host "server.joe.com" and accepting
|
|
TCP/IP connections on port 5432?
|
|
</screen>
|
|
This is the generic <quote>I couldn't find a server to talk
|
|
to</quote> failure. It looks like the above when TCP/IP
|
|
communication is attempted. A common mistake is to forget to
|
|
configure the server to allow TCP/IP connections.
|
|
</para>
|
|
|
|
<para>
|
|
Alternatively, you'll get this when attempting Unix-domain socket
|
|
communication to a local server:
|
|
<screen>
|
|
psql: could not connect to server: No such file or directory
|
|
Is the server running locally and accepting
|
|
connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
|
|
</screen>
|
|
</para>
|
|
|
|
<para>
|
|
The last line is useful in verifying that the client is trying to
|
|
connect to the right place. If there is in fact no server
|
|
running there, the kernel error message will typically be either
|
|
<computeroutput>Connection refused</computeroutput> or
|
|
<computeroutput>No such file or directory</computeroutput>, as
|
|
illustrated. (It is important to realize that
|
|
<computeroutput>Connection refused</computeroutput> in this context
|
|
does <emphasis>not</emphasis> mean that the server got your
|
|
connection request and rejected it. That case will produce a
|
|
different message, as shown in <xref
|
|
linkend="client-authentication-problems">.) Other error messages
|
|
such as <computeroutput>Connection timed out</computeroutput> might
|
|
indicate more fundamental problems, like lack of network
|
|
connectivity.
|
|
</para>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
<sect1 id="kernel-resources">
|
|
<title>Managing Kernel Resources</title>
|
|
|
|
<para>
|
|
A large <productname>PostgreSQL</> installation can quickly exhaust
|
|
various operating system resource limits. (On some systems, the
|
|
factory defaults are so low that you don't even need a really
|
|
<quote>large</> installation.) If you have encountered this kind of
|
|
problem, keep reading.
|
|
</para>
|
|
|
|
<sect2 id="sysvipc">
|
|
<title>Shared Memory and Semaphores</title>
|
|
|
|
<indexterm zone="sysvipc">
|
|
<primary>shared memory</primary>
|
|
</indexterm>
|
|
|
|
<indexterm zone="sysvipc">
|
|
<primary>semaphores</primary>
|
|
</indexterm>
|
|
|
|
<para>
|
|
Shared memory and semaphores are collectively referred to as
|
|
<quote><systemitem class="osname">System V</>
|
|
<acronym>IPC</></quote> (together with message queues, which are not
|
|
relevant for <productname>PostgreSQL</>). Almost all modern
|
|
operating systems provide these features, but many of them don't have
|
|
them turned on or sufficiently sized by default, especially as
|
|
available RAM and the demands of database applications grow.
|
|
(On <systemitem class="osname">Windows</>,
|
|
<productname>PostgreSQL</> provides its own replacement
|
|
implementation of these facilities, so most of this section
|
|
can be disregarded.)
|
|
</para>
|
|
|
|
<para>
|
|
The complete lack of these facilities is usually manifested by an
|
|
<errorname>Illegal system call</> error upon server start. In
|
|
that case there is no alternative but to reconfigure your
|
|
kernel. <productname>PostgreSQL</> won't work without them.
|
|
This situation is rare, however, among modern operating systems.
|
|
</para>
|
|
|
|
<para>
|
|
When <productname>PostgreSQL</> exceeds one of the various hard
|
|
<acronym>IPC</> limits, the server will refuse to start and
|
|
should leave an instructive error message describing the problem
|
|
and what to do about it. (See also <xref
|
|
linkend="server-start-failures">.) The relevant kernel
|
|
parameters are named consistently across different systems; <xref
|
|
linkend="sysvipc-parameters"> gives an overview. The methods to set
|
|
them, however, vary. Suggestions for some platforms are given below.
|
|
</para>
|
|
|
|
|
|
<table id="sysvipc-parameters">
|
|
<title><systemitem class="osname">System V</> <acronym>IPC</> Parameters</title>
|
|
|
|
<tgroup cols="3">
|
|
<thead>
|
|
<row>
|
|
<entry>Name</>
|
|
<entry>Description</>
|
|
<entry>Reasonable values</>
|
|
</row>
|
|
</thead>
|
|
|
|
<tbody>
|
|
<row>
|
|
<entry><varname>SHMMAX</></>
|
|
<entry>Maximum size of shared memory segment (bytes)</>
|
|
<entry>at least several megabytes (see text)</entry>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SHMMIN</></>
|
|
<entry>Minimum size of shared memory segment (bytes)</>
|
|
<entry>1</>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SHMALL</></>
|
|
<entry>Total amount of shared memory available (bytes or pages)</>
|
|
<entry>if bytes, same as <varname>SHMMAX</varname>; if pages, <literal>ceil(SHMMAX/PAGE_SIZE)</literal></>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SHMSEG</></>
|
|
<entry>Maximum number of shared memory segments per process</>
|
|
<entry>only 1 segment is needed, but the default is much higher</>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SHMMNI</></>
|
|
<entry>Maximum number of shared memory segments system-wide</>
|
|
<entry>like <varname>SHMSEG</> plus room for other applications</>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SEMMNI</></>
|
|
<entry>Maximum number of semaphore identifiers (i.e., sets)</>
|
|
<entry>at least <literal>ceil((max_connections + autovacuum_max_workers + 4) / 16)</literal></>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SEMMNS</></>
|
|
<entry>Maximum number of semaphores system-wide</>
|
|
<entry><literal>ceil((max_connections + autovacuum_max_workers + 4) / 16) * 17</literal> plus room for other applications</>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SEMMSL</></>
|
|
<entry>Maximum number of semaphores per set</>
|
|
<entry>at least 17</>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SEMMAP</></>
|
|
<entry>Number of entries in semaphore map</>
|
|
<entry>see text</>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><varname>SEMVMX</></>
|
|
<entry>Maximum value of semaphore</>
|
|
<entry>at least 1000 (The default is often 32767; do not change unless necessary)</>
|
|
</row>
|
|
|
|
</tbody>
|
|
</tgroup>
|
|
</table>
|
|
|
|
|
|
<para>
|
|
<indexterm><primary>SHMMAX</primary></indexterm> The most important
|
|
shared memory parameter is <varname>SHMMAX</>, the maximum size, in
|
|
bytes, of a shared memory segment. If you get an error message from
|
|
<function>shmget</> like <quote>Invalid argument</>, it is
|
|
likely that this limit has been exceeded. The size of the required
|
|
shared memory segment varies depending on several
|
|
<productname>PostgreSQL</> configuration parameters, as shown in
|
|
<xref linkend="shared-memory-parameters">. (Any error message you might
|
|
get will include the exact size of the failed allocation request.)
|
|
You can, as a temporary solution, lower some of those settings to
|
|
avoid the failure. While it is possible to get
|
|
<productname>PostgreSQL</> to run with <varname>SHMMAX</> as small as
|
|
2 MB, you need considerably more for acceptable performance. Desirable
|
|
settings are in the hundreds of megabytes to a few gigabytes.
|
|
</para>
|
|
|
|
<para>
|
|
Some systems also have a limit on the total amount of shared memory in
|
|
the system (<varname>SHMALL</>). Make sure this is large enough
|
|
for <productname>PostgreSQL</> plus any other applications that
|
|
are using shared memory segments. Note that <varname>SHMALL</>
|
|
is measured in pages rather than bytes on many systems.
|
|
</para>
|
|
|
|
<para>
|
|
Less likely to cause problems is the minimum size for shared
|
|
memory segments (<varname>SHMMIN</>), which should be at most
|
|
approximately 500 kB for <productname>PostgreSQL</> (it is
|
|
usually just 1). The maximum number of segments system-wide
|
|
(<varname>SHMMNI</>) or per-process (<varname>SHMSEG</>) are unlikely
|
|
to cause a problem unless your system has them set to zero.
|
|
</para>
|
|
|
|
<para>
|
|
<productname>PostgreSQL</> uses one semaphore per allowed connection
|
|
(<xref linkend="guc-max-connections">) and allowed autovacuum worker
|
|
process (<xref linkend="guc-autovacuum-max-workers">), in sets of 16.
|
|
Each such set will
|
|
also contain a 17th semaphore which contains a <quote>magic
|
|
number</quote>, to detect collision with semaphore sets used by
|
|
other applications. The maximum number of semaphores in the system
|
|
is set by <varname>SEMMNS</>, which consequently must be at least
|
|
as high as <varname>max_connections</> plus
|
|
<varname>autovacuum_max_workers</>, plus one extra for each 16
|
|
allowed connections plus workers (see the formula in <xref
|
|
linkend="sysvipc-parameters">). The parameter <varname>SEMMNI</>
|
|
determines the limit on the number of semaphore sets that can
|
|
exist on the system at one time. Hence this parameter must be at
|
|
least <literal>ceil((max_connections + autovacuum_max_workers + 4) / 16)</>.
|
|
Lowering the number
|
|
of allowed connections is a temporary workaround for failures,
|
|
which are usually confusingly worded <quote>No space
|
|
left on device</>, from the function <function>semget</>.
|
|
</para>
|
|
|
|
<para>
|
|
In some cases it might also be necessary to increase
|
|
<varname>SEMMAP</> to be at least on the order of
|
|
<varname>SEMMNS</>. This parameter defines the size of the semaphore
|
|
resource map, in which each contiguous block of available semaphores
|
|
needs an entry. When a semaphore set is freed it is either added to
|
|
an existing entry that is adjacent to the freed block or it is
|
|
registered under a new map entry. If the map is full, the freed
|
|
semaphores get lost (until reboot). Fragmentation of the semaphore
|
|
space could over time lead to fewer available semaphores than there
|
|
should be.
|
|
</para>
|
|
|
|
<para>
|
|
The <varname>SEMMSL</> parameter, which determines how many
|
|
semaphores can be in a set, must be at least 17 for
|
|
<productname>PostgreSQL</>.
|
|
</para>
|
|
|
|
<para>
|
|
Various other settings related to <quote>semaphore undo</>, such as
|
|
<varname>SEMMNU</> and <varname>SEMUME</>, do not affect
|
|
<productname>PostgreSQL</>.
|
|
</para>
|
|
|
|
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term><systemitem class="osname">AIX</></term>
|
|
<indexterm><primary>AIX</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
At least as of version 5.1, it should not be necessary to do
|
|
any special configuration for such parameters as
|
|
<varname>SHMMAX</varname>, as it appears this is configured to
|
|
allow all memory to be used as shared memory. That is the
|
|
sort of configuration commonly used for other databases such
|
|
as <application>DB/2</application>.</para>
|
|
|
|
<para> It might, however, be necessary to modify the global
|
|
<command>ulimit</command> information in
|
|
<filename>/etc/security/limits</filename>, as the default hard
|
|
limits for file sizes (<varname>fsize</varname>) and numbers of
|
|
files (<varname>nofiles</varname>) might be too low.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">BSD/OS</></term>
|
|
<indexterm><primary>BSD/OS</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<formalpara>
|
|
<title>Shared Memory</title>
|
|
<para>
|
|
By default, only 4 MB of shared memory is supported. Keep in
|
|
mind that shared memory is not pageable; it is locked in RAM.
|
|
To increase the amount of shared memory supported by your
|
|
system, add something like the following to your kernel configuration
|
|
file:
|
|
<programlisting>
|
|
options "SHMALL=8192"
|
|
options "SHMMAX=\(SHMALL*PAGE_SIZE\)"
|
|
</programlisting>
|
|
<varname>SHMALL</> is measured in 4 kB pages, so a value of
|
|
1024 represents 4 MB of shared memory. Therefore the above increases
|
|
the maximum shared memory area to 32 MB.
|
|
For those running 4.3 or later, you will probably also need to increase
|
|
<varname>KERNEL_VIRTUAL_MB</> above the default <literal>248</>.
|
|
Once all changes have been made, recompile the kernel, and reboot.
|
|
</para>
|
|
</formalpara>
|
|
|
|
<formalpara>
|
|
<title>Semaphores</title>
|
|
<para>
|
|
You will probably want to increase the number of semaphores
|
|
as well; the default system total of 60 will only allow about
|
|
50 <productname>PostgreSQL</productname> connections. Set the
|
|
values you want in your kernel configuration file, e.g.:
|
|
<programlisting>
|
|
options "SEMMNI=40"
|
|
options "SEMMNS=240"
|
|
</programlisting>
|
|
</para>
|
|
</formalpara>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">FreeBSD</></term>
|
|
<indexterm><primary>FreeBSD</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
The default settings are only suitable for small installations
|
|
(for example, default <varname>SHMMAX</varname> is 32
|
|
MB). Changes can be made via the <command>sysctl</command> or
|
|
<command>loader</command> interfaces. The following
|
|
parameters can be set using <command>sysctl</command>:
|
|
<screen>
|
|
<prompt>$</prompt> <userinput>sysctl -w kern.ipc.shmall=32768</userinput>
|
|
<prompt>$</prompt> <userinput>sysctl -w kern.ipc.shmmax=134217728</userinput>
|
|
<prompt>$</prompt> <userinput>sysctl -w kern.ipc.semmap=256</userinput>
|
|
</screen>
|
|
To have these settings persist over reboots, modify
|
|
<filename>/etc/sysctl.conf</filename>.
|
|
</para>
|
|
|
|
<para>
|
|
The remaining semaphore settings are read-only as far as
|
|
<command>sysctl</command> is concerned, but can be changed
|
|
before boot using the <command>loader</command> prompt:
|
|
<screen>
|
|
<prompt>(loader)</prompt> <userinput>set kern.ipc.semmni=256</userinput>
|
|
<prompt>(loader)</prompt> <userinput>set kern.ipc.semmns=512</userinput>
|
|
<prompt>(loader)</prompt> <userinput>set kern.ipc.semmnu=256</userinput>
|
|
</screen>
|
|
Similarly these can be saved between reboots in
|
|
<filename>/boot/loader.conf</filename>.
|
|
</para>
|
|
|
|
<para>
|
|
You might also want to configure your kernel to lock shared
|
|
memory into RAM and prevent it from being paged out to swap.
|
|
This can be accomplished using the <command>sysctl</command>
|
|
setting <literal>kern.ipc.shm_use_phys</literal>.
|
|
</para>
|
|
|
|
<para>
|
|
If running in FreeBSD jails by enabling <application>sysctl</>'s
|
|
<literal>security.jail.sysvipc_allowed</>, <application>postmaster</>s
|
|
running in different jails should be run by different operating system
|
|
users. This improves security because it prevents non-root users
|
|
from interfering with shared memory or semaphores in different jails,
|
|
and it allows the PostgreSQL IPC cleanup code to function properly.
|
|
(In FreeBSD 6.0 and later the IPC cleanup code does not properly detect
|
|
processes in other jails, preventing the running of postmasters on the
|
|
same port in different jails.)
|
|
</para>
|
|
|
|
<para>
|
|
<systemitem class="osname">FreeBSD</> versions before 4.0 work like
|
|
<systemitem class="osname">NetBSD</> and <systemitem class="osname">
|
|
OpenBSD</> (see below).
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">NetBSD</></term>
|
|
<term><systemitem class="osname">OpenBSD</></term>
|
|
<indexterm><primary>NetBSD</><secondary>IPC configuration</></>
|
|
<indexterm><primary>OpenBSD</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
The options <varname>SYSVSHM</> and <varname>SYSVSEM</> need
|
|
to be enabled when the kernel is compiled. (They are by
|
|
default.) The maximum size of shared memory is determined by
|
|
the option <varname>SHMMAXPGS</> (in pages). The following
|
|
shows an example of how to set the various parameters on
|
|
<systemitem class="osname">NetBSD</>
|
|
(<systemitem class="osname">OpenBSD</> uses <literal>option</> instead):
|
|
<programlisting>
|
|
options SYSVSHM
|
|
options SHMMAXPGS=4096
|
|
options SHMSEG=256
|
|
|
|
options SYSVSEM
|
|
options SEMMNI=256
|
|
options SEMMNS=512
|
|
options SEMMNU=256
|
|
options SEMMAP=256
|
|
</programlisting>
|
|
</para>
|
|
|
|
<para>
|
|
You might also want to configure your kernel to lock shared
|
|
memory into RAM and prevent it from being paged out to swap.
|
|
This can be accomplished using the <command>sysctl</command>
|
|
setting <literal>kern.ipc.shm_use_phys</literal>.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">HP-UX</></term>
|
|
<indexterm><primary>HP-UX</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
The default settings tend to suffice for normal installations.
|
|
On <productname>HP-UX</> 10, the factory default for
|
|
<varname>SEMMNS</> is 128, which might be too low for larger
|
|
database sites.
|
|
</para>
|
|
<para>
|
|
<acronym>IPC</> parameters can be set in the <application>System
|
|
Administration Manager</> (<acronym>SAM</>) under
|
|
<menuchoice><guimenu>Kernel
|
|
Configuration</><guimenuitem>Configurable Parameters</></>. Choose
|
|
<guibutton>Create A New Kernel</> when you're done.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">Linux</></term>
|
|
<indexterm><primary>Linux</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
The default maximum segment size is 32 MB, which is only adequate
|
|
for very small <productname>PostgreSQL</productname>
|
|
installations. The default maximum total size is 2097152
|
|
pages. A page is almost always 4096 bytes except in unusual
|
|
kernel configurations with <quote>huge pages</quote>
|
|
(use <literal>getconf PAGE_SIZE</literal> to verify). That
|
|
makes a default limit of 8 GB, which is often enough, but not
|
|
always.
|
|
</para>
|
|
|
|
<para>
|
|
The shared memory size settings can be changed via the
|
|
<command>sysctl</command> interface. For example, to allow 16 GB:
|
|
<screen>
|
|
<prompt>$</prompt> <userinput>sysctl -w kernel.shmmax=17179869184</userinput>
|
|
<prompt>$</prompt> <userinput>sysctl -w kernel.shmall=4194304</userinput>
|
|
</screen>
|
|
In addition these settings can be preserved between reboots in
|
|
the file <filename>/etc/sysctl.conf</filename>. Doing that is
|
|
highly recommended.
|
|
</para>
|
|
|
|
<para>
|
|
Ancient distributions might not have the <command>sysctl</command> program,
|
|
but equivalent changes can be made by manipulating the
|
|
<filename>/proc</filename> file system:
|
|
<screen>
|
|
<prompt>$</prompt> <userinput>echo 17179869184 >/proc/sys/kernel/shmmax</userinput>
|
|
<prompt>$</prompt> <userinput>echo 4194304 >/proc/sys/kernel/shmall</userinput>
|
|
</screen>
|
|
</para>
|
|
|
|
<para>
|
|
The remaining defaults are quite generously sized, and usually
|
|
do not require changes.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">Mac OS X</></term>
|
|
<indexterm><primary>Mac OS X</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
The recommended method for configuring shared memory in OS X
|
|
is to create a file named <filename>/etc/sysctl.conf</>,
|
|
containing variable assignments such as:
|
|
<programlisting>
|
|
kern.sysv.shmmax=4194304
|
|
kern.sysv.shmmin=1
|
|
kern.sysv.shmmni=32
|
|
kern.sysv.shmseg=8
|
|
kern.sysv.shmall=1024
|
|
</programlisting>
|
|
Note that in some OS X versions,
|
|
<emphasis>all five</> shared-memory parameters must be set in
|
|
<filename>/etc/sysctl.conf</>, else the values will be ignored.
|
|
</para>
|
|
|
|
<para>
|
|
Beware that recent releases of OS X ignore attempts to set
|
|
<varname>SHMMAX</> to a value that isn't an exact multiple of 4096.
|
|
</para>
|
|
|
|
<para>
|
|
<varname>SHMALL</> is measured in 4 kB pages on this platform.
|
|
</para>
|
|
|
|
<para>
|
|
In older OS X versions, you will need to reboot to have changes in the
|
|
shared memory parameters take effect. As of 10.5 it is possible to
|
|
change all but <varname>SHMMNI</> on the fly, using
|
|
<application>sysctl</>. But it's still best to set up your preferred
|
|
values via <filename>/etc/sysctl.conf</>, so that the values will be
|
|
kept across reboots.
|
|
</para>
|
|
|
|
<para>
|
|
The file <filename>/etc/sysctl.conf</> is only honored in OS X
|
|
10.3.9 and later. If you are running a previous 10.3.x release,
|
|
you must edit the file <filename>/etc/rc</>
|
|
and change the values in the following commands:
|
|
<programlisting>
|
|
sysctl -w kern.sysv.shmmax
|
|
sysctl -w kern.sysv.shmmin
|
|
sysctl -w kern.sysv.shmmni
|
|
sysctl -w kern.sysv.shmseg
|
|
sysctl -w kern.sysv.shmall
|
|
</programlisting>
|
|
Note that
|
|
<filename>/etc/rc</> is usually overwritten by OS X system updates,
|
|
so you should expect to have to redo these edits after each update.
|
|
</para>
|
|
|
|
<para>
|
|
In OS X 10.2 and earlier, instead edit these commands in the file
|
|
<filename>/System/Library/StartupItems/SystemTuning/SystemTuning</>.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">SCO OpenServer</></term>
|
|
<indexterm><primary>SCO OpenServer</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
In the default configuration, only 512 kB of shared memory per
|
|
segment is allowed. To increase the setting, first change to the
|
|
directory <filename>/etc/conf/cf.d</>. To display the current value of
|
|
<varname>SHMMAX</>, run:
|
|
<programlisting>
|
|
./configure -y SHMMAX
|
|
</programlisting>
|
|
To set a new value for <varname>SHMMAX</>, run:
|
|
<programlisting>
|
|
./configure SHMMAX=<replaceable>value</>
|
|
</programlisting>
|
|
where <replaceable>value</> is the new value you want to use
|
|
(in bytes). After setting <varname>SHMMAX</>, rebuild the kernel:
|
|
<programlisting>
|
|
./link_unix
|
|
</programlisting>
|
|
and reboot.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">Solaris</> 2.6 to 2.9 (Solaris
|
|
6 to Solaris 9)</term>
|
|
<indexterm><primary>Solaris</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
The default maximum size of a shared memory segment is too low for
|
|
<productname>PostgreSQL</>. The relevant settings can be changed in
|
|
<filename>/etc/system</>, for example:
|
|
<programlisting>
|
|
set shmsys:shminfo_shmmax=0x2000000
|
|
set shmsys:shminfo_shmmin=1
|
|
set shmsys:shminfo_shmmni=256
|
|
set shmsys:shminfo_shmseg=256
|
|
|
|
set semsys:seminfo_semmap=256
|
|
set semsys:seminfo_semmni=512
|
|
set semsys:seminfo_semmns=512
|
|
set semsys:seminfo_semmsl=32
|
|
</programlisting>
|
|
You need to reboot for the changes to take effect. See also
|
|
<ulink url="http://sunsite.uakom.sk/sunworldonline/swol-09-1997/swol-09-insidesolaris.html"></ulink>
|
|
for information on shared memory under older versions of Solaris.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">Solaris</> 2.10 (Solaris
|
|
10)</term>
|
|
<term><systemitem class="osname">OpenSolaris</></term>
|
|
<listitem>
|
|
<para>
|
|
In Solaris 10 and OpenSolaris, the default shared memory and
|
|
semaphore settings are good enough for most
|
|
<productname>PostgreSQL</> applications. Solaris now defaults
|
|
to a <varname>SHMMAX</> of one-quarter of system <acronym>RAM</>. If
|
|
you need to increase this in order to set shared memory settings
|
|
slightly higher, you should use a project setting associated
|
|
with the <literal>postgres</> user. For example, run the
|
|
following as <literal>root</>:
|
|
<programlisting>
|
|
projadd -c "PostgreSQL DB User" -K "project.max-shm-memory=(privileged,8GB,deny)" -U postgres -G postgres user.postgres
|
|
</programlisting>
|
|
</para>
|
|
|
|
<para>
|
|
This command adds the <literal>user.postgres</> project and
|
|
raises the shared memory maximum for the <literal>postgres</>
|
|
user to 8GB, and takes effect the next time that user logs
|
|
in, or when you restart <productname>PostgreSQL</> (not reload).
|
|
The above assumes that <productname>PostgreSQL</> is run by
|
|
the <literal>postgres</> user in the <literal>postgres</>
|
|
group. No server reboot is required.
|
|
</para>
|
|
|
|
<para>
|
|
Other recommended kernel setting changes for database servers which will
|
|
have a large number of connections are:
|
|
<programlisting>
|
|
project.max-shm-ids=(priv,32768,deny)
|
|
project.max-sem-ids=(priv,4096,deny)
|
|
project.max-msg-ids=(priv,4096,deny)
|
|
</programlisting>
|
|
</para>
|
|
|
|
<para>
|
|
Additionally, if you are running <productname>PostgreSQL</>
|
|
inside a zone, you may need to raise the zone resource usage
|
|
limits as well. See "Chapter2: Projects and Tasks" in the
|
|
<citetitle>Solaris 10 System Administrator's Guide</> for more
|
|
information on <literal>projects</> and <command>prctl</>.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
|
|
<varlistentry>
|
|
<term><systemitem class="osname">UnixWare</></term>
|
|
<indexterm><primary>UnixWare</><secondary>IPC configuration</></>
|
|
<listitem>
|
|
<para>
|
|
On <productname>UnixWare</> 7, the maximum size for shared
|
|
memory segments is only 512 kB in the default configuration.
|
|
To display the current value of <varname>SHMMAX</>, run:
|
|
<programlisting>
|
|
/etc/conf/bin/idtune -g SHMMAX
|
|
</programlisting>
|
|
which displays the current, default, minimum, and maximum
|
|
values. To set a new value for <varname>SHMMAX</>,
|
|
run:
|
|
<programlisting>
|
|
/etc/conf/bin/idtune SHMMAX <replaceable>value</>
|
|
</programlisting>
|
|
where <replaceable>value</> is the new value you want to use
|
|
(in bytes). After setting <varname>SHMMAX</>, rebuild the
|
|
kernel:
|
|
<programlisting>
|
|
/etc/conf/bin/idbuild -B
|
|
</programlisting>
|
|
and reboot.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
</variablelist>
|
|
|
|
|
|
<table id="shared-memory-parameters">
|
|
<title><productname>PostgreSQL</productname> Shared Memory Usage</>
|
|
|
|
<tgroup cols="2">
|
|
<thead>
|
|
<row>
|
|
<entry>Usage</>
|
|
<entry>Approximate shared memory bytes required (as of 8.3)</>
|
|
</row>
|
|
</thead>
|
|
|
|
<tbody>
|
|
<row>
|
|
<entry>Connections</>
|
|
<entry>(1800 + 270 * <xref
|
|
linkend="guc-max-locks-per-transaction">) * <xref
|
|
linkend="guc-max-connections"></entry>
|
|
</row>
|
|
|
|
<row>
|
|
<entry>Autovacuum workers</>
|
|
<entry>(1800 + 270 * <xref
|
|
linkend="guc-max-locks-per-transaction">) * <xref
|
|
linkend="guc-autovacuum-max-workers"></entry>
|
|
</row>
|
|
|
|
<row>
|
|
<entry>Prepared transactions</>
|
|
<entry>(770 + 270 * <xref
|
|
linkend="guc-max-locks-per-transaction">) * <xref linkend="guc-max-prepared-transactions"></entry>
|
|
</row>
|
|
|
|
<row>
|
|
<entry>Shared disk buffers</>
|
|
<entry>(<xref linkend="guc-block-size"> + 208) * <xref linkend="guc-shared-buffers"></entry>
|
|
</row>
|
|
|
|
<row>
|
|
<entry>WAL buffers</>
|
|
<entry>(<xref linkend="guc-wal-block-size"> + 8) * <xref linkend="guc-wal-buffers"></entry>
|
|
</row>
|
|
|
|
<row>
|
|
<entry>Fixed space requirements</>
|
|
<entry>770 kB</entry>
|
|
</row>
|
|
</tbody>
|
|
</tgroup>
|
|
</table>
|
|
|
|
</sect2>
|
|
|
|
<sect2>
|
|
<title>Resource Limits</title>
|
|
|
|
<para>
|
|
Unix-like operating systems enforce various kinds of resource limits
|
|
that might interfere with the operation of your
|
|
<productname>PostgreSQL</productname> server. Of particular
|
|
importance are limits on the number of processes per user, the
|
|
number of open files per process, and the amount of memory available
|
|
to each process. Each of these have a <quote>hard</quote> and a
|
|
<quote>soft</quote> limit. The soft limit is what actually counts
|
|
but it can be changed by the user up to the hard limit. The hard
|
|
limit can only be changed by the root user. The system call
|
|
<function>setrlimit</function> is responsible for setting these
|
|
parameters. The shell's built-in command <command>ulimit</command>
|
|
(Bourne shells) or <command>limit</command> (<application>csh</>) is
|
|
used to control the resource limits from the command line. On
|
|
BSD-derived systems the file <filename>/etc/login.conf</filename>
|
|
controls the various resource limits set during login. See the
|
|
operating system documentation for details. The relevant
|
|
parameters are <varname>maxproc</varname>,
|
|
<varname>openfiles</varname>, and <varname>datasize</varname>. For
|
|
example:
|
|
<programlisting>
|
|
default:\
|
|
...
|
|
:datasize-cur=256M:\
|
|
:maxproc-cur=256:\
|
|
:openfiles-cur=256:\
|
|
...
|
|
</programlisting>
|
|
(<literal>-cur</literal> is the soft limit. Append
|
|
<literal>-max</literal> to set the hard limit.)
|
|
</para>
|
|
|
|
<para>
|
|
Kernels can also have system-wide limits on some resources.
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>
|
|
On <productname>Linux</productname>
|
|
<filename>/proc/sys/fs/file-max</filename> determines the
|
|
maximum number of open files that the kernel will support. It can
|
|
be changed by writing a different number into the file or by
|
|
adding an assignment in <filename>/etc/sysctl.conf</filename>.
|
|
The maximum limit of files per process is fixed at the time the
|
|
kernel is compiled; see
|
|
<filename>/usr/src/linux/Documentation/proc.txt</filename> for
|
|
more information.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</para>
|
|
|
|
<para>
|
|
The <productname>PostgreSQL</productname> server uses one process
|
|
per connection so you should provide for at least as many processes
|
|
as allowed connections, in addition to what you need for the rest
|
|
of your system. This is usually not a problem but if you run
|
|
several servers on one machine things might get tight.
|
|
</para>
|
|
|
|
<para>
|
|
The factory default limit on open files is often set to
|
|
<quote>socially friendly</quote> values that allow many users to
|
|
coexist on a machine without using an inappropriate fraction of
|
|
the system resources. If you run many servers on a machine this
|
|
is perhaps what you want, but on dedicated servers you might want to
|
|
raise this limit.
|
|
</para>
|
|
|
|
<para>
|
|
On the other side of the coin, some systems allow individual
|
|
processes to open large numbers of files; if more than a few
|
|
processes do so then the system-wide limit can easily be exceeded.
|
|
If you find this happening, and you do not want to alter the
|
|
system-wide limit, you can set <productname>PostgreSQL</>'s <xref
|
|
linkend="guc-max-files-per-process"> configuration parameter to
|
|
limit the consumption of open files.
|
|
</para>
|
|
</sect2>
|
|
|
|
<sect2 id="linux-memory-overcommit">
|
|
<title>Linux Memory Overcommit</title>
|
|
|
|
<para>
|
|
In Linux 2.4 and later, the default virtual memory behavior is not
|
|
optimal for <productname>PostgreSQL</productname>. Because of the
|
|
way that the kernel implements memory overcommit, the kernel might
|
|
terminate the <productname>PostgreSQL</productname> server (the
|
|
master server process) if the memory demands of
|
|
another process cause the system to run out of virtual memory.
|
|
</para>
|
|
|
|
<para>
|
|
If this happens, you will see a kernel message that looks like
|
|
this (consult your system documentation and configuration on where
|
|
to look for such a message):
|
|
<programlisting>
|
|
Out of Memory: Killed process 12345 (postgres).
|
|
</programlisting>
|
|
This indicates that the <filename>postgres</filename> process
|
|
has been terminated due to memory pressure.
|
|
Although existing database connections will continue to function
|
|
normally, no new connections will be accepted. To recover,
|
|
<productname>PostgreSQL</productname> will need to be restarted.
|
|
</para>
|
|
|
|
<para>
|
|
One way to avoid this problem is to run
|
|
<productname>PostgreSQL</productname> on a machine where you can
|
|
be sure that other processes will not run the machine out of
|
|
memory. If memory is tight, increasing the swap space of the
|
|
operating system can help avoid the problem, because the
|
|
out-of-memory (OOM) killer is invoked only when physical memory and
|
|
swap space are exhausted.
|
|
</para>
|
|
|
|
<para>
|
|
On Linux 2.6 and later, it is possible to modify the
|
|
kernel's behavior so that it will not <quote>overcommit</> memory.
|
|
Although this setting will not prevent the <ulink
|
|
url="http://lwn.net/Articles/104179/">OOM killer</> from being invoked
|
|
altogether, it will lower the chances significantly and will therefore
|
|
lead to more robust system behavior. This is done by selecting strict
|
|
overcommit mode via <command>sysctl</command>:
|
|
<programlisting>
|
|
sysctl -w vm.overcommit_memory=2
|
|
</programlisting>
|
|
or placing an equivalent entry in <filename>/etc/sysctl.conf</>.
|
|
You might also wish to modify the related setting
|
|
<varname>vm.overcommit_ratio</>. For details see the kernel documentation
|
|
file <filename>Documentation/vm/overcommit-accounting</>.
|
|
</para>
|
|
|
|
<para>
|
|
Another approach, which can be used with or without altering
|
|
<varname>vm.overcommit_memory</>, is to set the process-specific
|
|
<varname>oom_adj</> value for the postmaster process to <literal>-17</>,
|
|
thereby guaranteeing it will not be targeted by the OOM killer. The
|
|
simplest way to do this is to execute
|
|
<programlisting>
|
|
echo -17 > /proc/self/oom_adj
|
|
</programlisting>
|
|
in the postmaster's startup script just before invoking the postmaster.
|
|
Note that this action must be done as root, or it will have no effect;
|
|
so a root-owned startup script is the easiest place to do it. If you
|
|
do this, you may also wish to build <productname>PostgreSQL</>
|
|
with <literal>-DLINUX_OOM_ADJ=0</> added to <varname>CPPFLAGS</>.
|
|
That will cause postmaster child processes to run with the normal
|
|
<varname>oom_adj</> value of zero, so that the OOM killer can still
|
|
target them at need.
|
|
</para>
|
|
|
|
<note>
|
|
<para>
|
|
Some vendors' Linux 2.4 kernels are reported to have early versions
|
|
of the 2.6 overcommit <command>sysctl</command> parameter. However, setting
|
|
<literal>vm.overcommit_memory</> to 2
|
|
on a 2.4 kernel that does not have the relevant code will make
|
|
things worse, not better. It is recommended that you inspect
|
|
the actual kernel source code (see the function
|
|
<function>vm_enough_memory</> in the file <filename>mm/mmap.c</>)
|
|
to verify what is supported in your kernel before you try this in a 2.4
|
|
installation. The presence of the <filename>overcommit-accounting</>
|
|
documentation file should <emphasis>not</> be taken as evidence that the
|
|
feature is there. If in any doubt, consult a kernel expert or your
|
|
kernel vendor.
|
|
</para>
|
|
</note>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
|
|
<sect1 id="server-shutdown">
|
|
<title>Shutting Down the Server</title>
|
|
|
|
<indexterm zone="server-shutdown">
|
|
<primary>shutdown</>
|
|
</indexterm>
|
|
|
|
<para>
|
|
There are several ways to shut down the database server. You control
|
|
the type of shutdown by sending different signals to the master
|
|
<command>postgres</command> process.
|
|
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term><systemitem>SIGTERM</systemitem><indexterm><primary>SIGTERM</></></term>
|
|
<listitem>
|
|
<para>
|
|
This is the <firstterm>Smart Shutdown</firstterm> mode.
|
|
After receiving <systemitem>SIGTERM</systemitem>, the server
|
|
disallows new connections, but lets existing sessions end their
|
|
work normally. It shuts down only after all of the sessions terminate.
|
|
If the server is in online backup mode, it additionally waits
|
|
until online backup mode is no longer active. While backup mode is
|
|
active, new connections will still be allowed, but only to superusers
|
|
(this exception allows a superuser to connect to terminate
|
|
online backup mode). If the server is in recovery when a smart
|
|
shutdown is requested, recovery and streaming replication will be
|
|
stopped only after all regular sessions have terminated.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><systemitem>SIGINT</systemitem><indexterm><primary>SIGINT</></></term>
|
|
<listitem>
|
|
<para>
|
|
This is the <firstterm>Fast Shutdown</firstterm> mode.
|
|
The server disallows new connections and sends all existing
|
|
server processes <systemitem>SIGTERM</systemitem>, which will cause them
|
|
to abort their current transactions and exit promptly. It then
|
|
waits for all server processes to exit and finally shuts down.
|
|
If the server is in online backup mode, backup mode will be
|
|
terminated, rendering the backup useless.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term><systemitem>SIGQUIT</systemitem><indexterm><primary>SIGQUIT</></></term>
|
|
<listitem>
|
|
<para>
|
|
This is the <firstterm>Immediate Shutdown</firstterm> mode.
|
|
The master <command>postgres</command> process will send a
|
|
<systemitem>SIGQUIT</systemitem> to all child processes and exit
|
|
immediately, without properly shutting itself down. The child processes
|
|
likewise exit immediately upon receiving
|
|
<systemitem>SIGQUIT</systemitem>. This will lead to recovery (by
|
|
replaying the WAL log) upon next start-up. This is recommended
|
|
only in emergencies.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist>
|
|
</para>
|
|
|
|
<para>
|
|
The <xref linkend="app-pg-ctl"> program provides a convenient
|
|
interface for sending these signals to shut down the server.
|
|
Alternatively, you can send the signal directly using <command>kill</>
|
|
on non-Windows systems.
|
|
The <acronym>PID</> of the <command>postgres</command> process can be
|
|
found using the <command>ps</command> program, or from the file
|
|
<filename>postmaster.pid</filename> in the data directory. For
|
|
example, to do a fast shutdown:
|
|
<screen>
|
|
$ <userinput>kill -INT `head -1 /usr/local/pgsql/data/postmaster.pid`</userinput>
|
|
</screen>
|
|
</para>
|
|
|
|
<important>
|
|
<para>
|
|
It is best not to use <systemitem>SIGKILL</systemitem> to shut down
|
|
the server. Doing so will prevent the server from releasing
|
|
shared memory and semaphores, which might then have to be done
|
|
manually before a new server can be started. Furthermore,
|
|
<systemitem>SIGKILL</systemitem> kills the <command>postgres</command>
|
|
process without letting it relay the signal to its subprocesses,
|
|
so it will be necessary to kill the individual subprocesses by hand as
|
|
well.
|
|
</para>
|
|
</important>
|
|
|
|
<para>
|
|
To terminate an individual session while allowing other sessions to
|
|
continue, use <function>pg_terminate_backend()</> (see <xref
|
|
linkend="functions-admin-signal-table">) or send a
|
|
<systemitem>SIGTERM</> signal to the child process associated with
|
|
the session.
|
|
</para>
|
|
</sect1>
|
|
|
|
<sect1 id="upgrading">
|
|
<title>Upgrading a <productname>PostgreSQL</> Cluster</title>
|
|
|
|
<indexterm zone="upgrading">
|
|
<primary>upgrading</primary>
|
|
</indexterm>
|
|
|
|
<indexterm zone="upgrading">
|
|
<primary>version</primary>
|
|
<secondary>compatibility</secondary>
|
|
</indexterm>
|
|
|
|
<para>
|
|
This section discusses how to upgrade your database data from one
|
|
<productname>PostgreSQL</> release to a newer one.
|
|
</para>
|
|
|
|
<para>
|
|
<productname>PostgreSQL</> major versions are represented by the
|
|
first two digit groups of the version number, e.g., 8.4.
|
|
<productname>PostgreSQL</> minor versions are represented by the
|
|
third group of version digits, e.g., 8.4.2 is the second minor
|
|
release of 8.4. Minor releases never change the internal storage
|
|
format and are always compatible with earlier and later minor
|
|
releases of the same major version number, e.g., 8.4.2 is compatible
|
|
with 8.4, 8.4.1 and 8.4.6. To update between compatible versions,
|
|
you simply replace the executables while the server is down and
|
|
restart the server. The data directory remains unchanged —
|
|
minor upgrades are that simple.
|
|
</para>
|
|
|
|
<para>
|
|
For <emphasis>major</> releases of <productname>PostgreSQL</>, the
|
|
internal data storage format is subject to change, thus complicating
|
|
upgrades. The traditional method for moving data to a new major version
|
|
is to dump and reload the database. Other methods are available,
|
|
as discussed below.
|
|
</para>
|
|
|
|
<para>
|
|
New major versions also typically introduce some user-visible
|
|
incompatibilities, so application programming changes might be required.
|
|
All user-visible changes are listed in the release notes (<xref
|
|
linkend="release">); pay particular attention to the section
|
|
labeled "Migration". If you are upgrading across several major
|
|
versions, be sure to read the release notes for each intervening
|
|
version.
|
|
</para>
|
|
|
|
<para>
|
|
Cautious users will want to test their client applications on the new
|
|
version before switching over fully; therefore, it's often a good idea to
|
|
set up concurrent installations of old and new versions. When
|
|
testing a <productname>PostgreSQL</> major upgrade, consider the
|
|
following categories of possible changes:
|
|
</para>
|
|
|
|
<variablelist>
|
|
|
|
<varlistentry>
|
|
<term>Administration</term>
|
|
<listitem>
|
|
<para>
|
|
The capabilities available for administrators to monitor and control
|
|
the server often change and improve in each major release.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>SQL</term>
|
|
<listitem>
|
|
<para>
|
|
Typically this includes new SQL command capabilities and not changes
|
|
in behavior, unless specifically mentioned in the release notes.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>Library API</term>
|
|
<listitem>
|
|
<para>
|
|
Typically libraries like <application>libpq</> only add new
|
|
functionality, again unless mentioned in the release notes.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>System Catalogs</term>
|
|
<listitem>
|
|
<para>
|
|
System catalog changes usually only affect database management tools.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>Server C-language API</term>
|
|
<listitem>
|
|
<para>
|
|
This involves changes in the backend function API, which is written
|
|
in the C programming language. Such changes affect code that
|
|
references backend functions deep inside the server.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
</variablelist>
|
|
|
|
<sect2 id="upgrade-methods-pgdump">
|
|
<title>Upgrading Data via <application>pg_dump</></title>
|
|
|
|
<para>
|
|
To dump data from one major version of <productname>PostgreSQL</> and
|
|
reload it in another, you must use <application>pg_dump</>; file system
|
|
level backup methods will not work. (There are checks in place that prevent
|
|
you from using a data directory with an incompatible version of
|
|
<productname>PostgreSQL</productname>, so no great harm can be done by
|
|
trying to start the wrong server version on a data directory.)
|
|
</para>
|
|
|
|
<para>
|
|
It is recommended that you use the <application>pg_dump</> and
|
|
<application>pg_dumpall</> programs from the newer version of
|
|
<productname>PostgreSQL</>, to take advantage of enhancements
|
|
that might have been made in these programs. Current releases of the
|
|
dump programs can read data from any server version back to 7.0.
|
|
</para>
|
|
|
|
<para>
|
|
These instructions assume that your existing installation is under the
|
|
<filename>/usr/local/pgsql</> directory, and that the data area is in
|
|
<filename>/usr/local/pgsql/data</>. Substitute your paths
|
|
appropriately.
|
|
</para>
|
|
|
|
<procedure>
|
|
<step>
|
|
<para>
|
|
If making a backup, make sure that your database is not being updated.
|
|
This does not affect the integrity of the backup, but the changed
|
|
data would of course not be included. If necessary, edit the
|
|
permissions in the file <filename>/usr/local/pgsql/data/pg_hba.conf</>
|
|
(or equivalent) to disallow access from everyone except you.
|
|
See <xref linkend="client-authentication"> for additional information on
|
|
access control.
|
|
</para>
|
|
|
|
<para>
|
|
<indexterm>
|
|
<primary>pg_dumpall</primary>
|
|
<secondary>use during upgrade</secondary>
|
|
</indexterm>
|
|
|
|
To back up your database installation, type:
|
|
<screen>
|
|
<userinput>pg_dumpall > <replaceable>outputfile</></userinput>
|
|
</screen>
|
|
If you need to preserve OIDs (such as when using them as
|
|
foreign keys), then use the <option>-o</option> option when running
|
|
<application>pg_dumpall</>.
|
|
</para>
|
|
|
|
<para>
|
|
To make the backup, you can use the <application>pg_dumpall</application>
|
|
command from the version you are currently running. For best
|
|
results, however, try to use the <application>pg_dumpall</application>
|
|
command from <productname>PostgreSQL</productname> &version;,
|
|
since this version contains bug fixes and improvements over older
|
|
versions. While this advice might seem idiosyncratic since you
|
|
haven't installed the new version yet, it is advisable to follow
|
|
it if you plan to install the new version in parallel with the
|
|
old version. In that case you can complete the installation
|
|
normally and transfer the data later. This will also decrease
|
|
the downtime.
|
|
</para>
|
|
</step>
|
|
|
|
<step>
|
|
<para>
|
|
Shut down the old server:
|
|
<screen>
|
|
<userinput>pg_ctl stop</>
|
|
</screen>
|
|
On systems that have <productname>PostgreSQL</> started at boot time,
|
|
there is probably a start-up file that will accomplish the same thing. For
|
|
example, on a <systemitem class="osname">Red Hat Linux</> system one
|
|
might find that this works:
|
|
<screen>
|
|
<userinput>/etc/rc.d/init.d/postgresql stop</userinput>
|
|
</screen>
|
|
See <xref linkend="runtime"> for details about starting and
|
|
stopping the server.
|
|
</para>
|
|
</step>
|
|
|
|
<step>
|
|
<para>
|
|
If restoring from backup, rename or delete the old installation
|
|
directory. It is a good idea to rename the directory, rather than
|
|
delete it, in case you have trouble and need to revert to it. Keep
|
|
in mind the directory might consume significant disk space. To rename
|
|
the directory, use a command like this:
|
|
<screen>
|
|
<userinput>mv /usr/local/pgsql /usr/local/pgsql.old</>
|
|
</screen>
|
|
(Be sure to move the directory as a single unit so relative paths
|
|
remain unchanged.)
|
|
</para>
|
|
</step>
|
|
|
|
<step>
|
|
<para>
|
|
Install the new version of <productname>PostgreSQL</productname> as
|
|
outlined in <![%standalone-include[the next section.]]>
|
|
<![%standalone-ignore[<xref linkend="install-procedure">.]]>
|
|
</para>
|
|
</step>
|
|
|
|
<step>
|
|
<para>
|
|
Create a new database cluster if needed. Remember that you must
|
|
execute these commands while logged in to the special database user
|
|
account (which you already have if you are upgrading).
|
|
<programlisting>
|
|
<userinput>/usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data</>
|
|
</programlisting>
|
|
</para>
|
|
</step>
|
|
|
|
<step>
|
|
<para>
|
|
Restore your previous <filename>pg_hba.conf</> and any
|
|
<filename>postgresql.conf</> modifications.
|
|
</para>
|
|
</step>
|
|
|
|
<step>
|
|
<para>
|
|
Start the database server, again using the special database user
|
|
account:
|
|
<programlisting>
|
|
<userinput>/usr/local/pgsql/bin/postgres -D /usr/local/pgsql/data</>
|
|
</programlisting>
|
|
</para>
|
|
</step>
|
|
|
|
<step>
|
|
<para>
|
|
Finally, restore your data from backup with:
|
|
<screen>
|
|
<userinput>/usr/local/pgsql/bin/psql -d postgres -f <replaceable>outputfile</></userinput>
|
|
</screen>
|
|
using the <emphasis>new</> <application>psql</>.
|
|
</para>
|
|
</step>
|
|
</procedure>
|
|
|
|
<para>
|
|
The least downtime can be achieved by installing the new server in
|
|
a different directory and running both the old and the new servers
|
|
in parallel, on different ports. Then you can use something like:
|
|
|
|
<programlisting>
|
|
pg_dumpall -p 5432 | psql -d postgres -p 5433
|
|
</programlisting>
|
|
to transfer your data.
|
|
</para>
|
|
|
|
</sect2>
|
|
|
|
<sect2 id="upgrading-methods-other">
|
|
<title>Non-Dump Upgrade Methods</title>
|
|
|
|
<para>
|
|
The <link linkend="pgupgrade">pg_upgrade</link> module allows an
|
|
installation to be migrated in-place from one major
|
|
<productname>PostgreSQL</> version to the next. Upgrades can be
|
|
performed in minutes.
|
|
</para>
|
|
|
|
<para>
|
|
It is also possible to use certain replication methods, such as
|
|
<productname>Slony</>, to create a standby server with the updated version of
|
|
<productname>PostgreSQL</>. This is possible because Slony supports
|
|
replication between different major versions of
|
|
<productname>PostgreSQL</>. The standby can be on the same computer or
|
|
a different computer. Once it has synced up with the master server
|
|
(running the older version of <productname>PostgreSQL</>), you can
|
|
switch masters and make the standby the master and shut down the older
|
|
database instance. Such a switch-over results in only several seconds
|
|
of downtime for an upgrade.
|
|
</para>
|
|
|
|
</sect2>
|
|
</sect1>
|
|
|
|
<sect1 id="preventing-server-spoofing">
|
|
<title>Preventing Server Spoofing</title>
|
|
|
|
<indexterm zone="preventing-server-spoofing">
|
|
<primary>server spoofing</primary>
|
|
</indexterm>
|
|
|
|
<para>
|
|
While the server is running, it is not possible for a malicious user
|
|
to take the place of the normal database server. However, when the
|
|
server is down, it is possible for a local user to spoof the normal
|
|
server by starting their own server. The spoof server could read
|
|
passwords and queries sent by clients, but could not return any data
|
|
because the <varname>PGDATA</> directory would still be secure because
|
|
of directory permissions. Spoofing is possible because any user can
|
|
start a database server; a client cannot identify an invalid server
|
|
unless it is specially configured.
|
|
</para>
|
|
|
|
<para>
|
|
The simplest way to prevent spoofing for <literal>local</>
|
|
connections is to use a Unix domain socket directory (<xref
|
|
linkend="guc-unix-socket-directory">) that has write permission only
|
|
for a trusted local user. This prevents a malicious user from creating
|
|
their own socket file in that directory. If you are concerned that
|
|
some applications might still reference <filename>/tmp</> for the
|
|
socket file and hence be vulnerable to spoofing, during operating system
|
|
startup create a symbolic link <filename>/tmp/.s.PGSQL.5432</> that points
|
|
to the relocated socket file. You also might need to modify your
|
|
<filename>/tmp</> cleanup script to prevent removal of the symbolic link.
|
|
</para>
|
|
|
|
<para>
|
|
To prevent spoofing on TCP connections, the best solution is to use
|
|
SSL certificates and make sure that clients check the server's certificate.
|
|
To do that, the server
|
|
must be configured to accept only <literal>hostssl</> connections (<xref
|
|
linkend="auth-pg-hba-conf">) and have SSL
|
|
<filename>server.key</filename> (key) and
|
|
<filename>server.crt</filename> (certificate) files (<xref
|
|
linkend="ssl-tcp">). The TCP client must connect using
|
|
<literal>sslmode=verify-ca</> or
|
|
<literal>verify-full</> and have the appropriate root certificate
|
|
file installed (<xref linkend="libpq-connect">).
|
|
</para>
|
|
</sect1>
|
|
|
|
<sect1 id="encryption-options">
|
|
<title>Encryption Options</title>
|
|
|
|
<indexterm zone="encryption-options">
|
|
<primary>encryption</primary>
|
|
</indexterm>
|
|
|
|
<para>
|
|
<productname>PostgreSQL</productname> offers encryption at several
|
|
levels, and provides flexibility in protecting data from disclosure
|
|
due to database server theft, unscrupulous administrators, and
|
|
insecure networks. Encryption might also be required to secure
|
|
sensitive data such as medical records or financial transactions.
|
|
</para>
|
|
|
|
<variablelist>
|
|
|
|
<varlistentry>
|
|
<term>Password Storage Encryption</term>
|
|
<listitem>
|
|
|
|
<para>
|
|
By default, database user passwords are stored as MD5 hashes, so
|
|
the administrator cannot determine the actual password assigned
|
|
to the user. If MD5 encryption is used for client authentication,
|
|
the unencrypted password is never even temporarily present on the
|
|
server because the client MD5-encrypts it before being sent
|
|
across the network.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>Encryption For Specific Columns</term>
|
|
|
|
<listitem>
|
|
<para>
|
|
The <xref linkend="pgcrypto"> module allows certain fields to be
|
|
stored encrypted.
|
|
This is useful if only some of the data is sensitive.
|
|
The client supplies the decryption key and the data is decrypted
|
|
on the server and then sent to the client.
|
|
</para>
|
|
|
|
<para>
|
|
The decrypted data and the decryption key are present on the
|
|
server for a brief time while it is being decrypted and
|
|
communicated between the client and server. This presents a brief
|
|
moment where the data and keys can be intercepted by someone with
|
|
complete access to the database server, such as the system
|
|
administrator.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>Data Partition Encryption</term>
|
|
|
|
<listitem>
|
|
<para>
|
|
On Linux, encryption can be layered on top of a file system
|
|
using a <quote>loopback device</quote>. This allows an entire
|
|
file system partition to be encrypted on disk, and decrypted by the
|
|
operating system. On FreeBSD, the equivalent facility is called
|
|
GEOM Based Disk Encryption (<acronym>gbde</acronym>), and many
|
|
other operating systems support this functionality, including Windows.
|
|
</para>
|
|
|
|
<para>
|
|
This mechanism prevents unencrypted data from being read from the
|
|
drives if the drives or the entire computer is stolen. This does
|
|
not protect against attacks while the file system is mounted,
|
|
because when mounted, the operating system provides an unencrypted
|
|
view of the data. However, to mount the file system, you need some
|
|
way for the encryption key to be passed to the operating system,
|
|
and sometimes the key is stored somewhere on the host that mounts
|
|
the disk.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>Encrypting Passwords Across A Network</term>
|
|
|
|
<listitem>
|
|
<para>
|
|
The <literal>MD5</> authentication method double-encrypts the
|
|
password on the client before sending it to the server. It first
|
|
MD5-encrypts it based on the user name, and then encrypts it
|
|
based on a random salt sent by the server when the database
|
|
connection was made. It is this double-encrypted value that is
|
|
sent over the network to the server. Double-encryption not only
|
|
prevents the password from being discovered, it also prevents
|
|
another connection from using the same encrypted password to
|
|
connect to the database server at a later time.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>Encrypting Data Across A Network</term>
|
|
|
|
<listitem>
|
|
<para>
|
|
SSL connections encrypt all data sent across the network: the
|
|
password, the queries, and the data returned. The
|
|
<filename>pg_hba.conf</> file allows administrators to specify
|
|
which hosts can use non-encrypted connections (<literal>host</>)
|
|
and which require SSL-encrypted connections
|
|
(<literal>hostssl</>). Also, clients can specify that they
|
|
connect to servers only via SSL. <application>Stunnel</> or
|
|
<application>SSH</> can also be used to encrypt transmissions.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>SSL Host Authentication</term>
|
|
|
|
<listitem>
|
|
<para>
|
|
It is possible for both the client and server to provide SSL
|
|
certificates to each other. It takes some extra configuration
|
|
on each side, but this provides stronger verification of identity
|
|
than the mere use of passwords. It prevents a computer from
|
|
pretending to be the server just long enough to read the password
|
|
sent by the client. It also helps prevent <quote>man in the middle</>
|
|
attacks where a computer between the client and server pretends to
|
|
be the server and reads and passes all data between the client and
|
|
server.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>Client-Side Encryption</term>
|
|
|
|
<listitem>
|
|
<para>
|
|
If the system administrator for the server's machine cannot be trusted,
|
|
it is necessary
|
|
for the client to encrypt the data; this way, unencrypted data
|
|
never appears on the database server. Data is encrypted on the
|
|
client before being sent to the server, and database results have
|
|
to be decrypted on the client before being used.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
</variablelist>
|
|
|
|
</sect1>
|
|
|
|
<sect1 id="ssl-tcp">
|
|
<title>Secure TCP/IP Connections with SSL</title>
|
|
|
|
<indexterm zone="ssl-tcp">
|
|
<primary>SSL</primary>
|
|
</indexterm>
|
|
|
|
<para>
|
|
<productname>PostgreSQL</> has native support for using
|
|
<acronym>SSL</> connections to encrypt client/server communications
|
|
for increased security. This requires that
|
|
<productname>OpenSSL</productname> is installed on both client and
|
|
server systems and that support in <productname>PostgreSQL</> is
|
|
enabled at build time (see <xref linkend="installation">).
|
|
</para>
|
|
|
|
<para>
|
|
With <acronym>SSL</> support compiled in, the
|
|
<productname>PostgreSQL</> server can be started with
|
|
<acronym>SSL</> enabled by setting the parameter
|
|
<xref linkend="guc-ssl"> to <literal>on</> in
|
|
<filename>postgresql.conf</>. The server will listen for both normal
|
|
and <acronym>SSL</> connections on the same TCP port, and will negotiate
|
|
with any connecting client on whether to use <acronym>SSL</>. By
|
|
default, this is at the client's option; see <xref
|
|
linkend="auth-pg-hba-conf"> about how to set up the server to require
|
|
use of <acronym>SSL</> for some or all connections.
|
|
</para>
|
|
|
|
<para>
|
|
<productname>PostgreSQL</productname> reads the system-wide
|
|
<productname>OpenSSL</productname> configuration file. By default, this
|
|
file is named <filename>openssl.cnf</filename> and is located in the
|
|
directory reported by <literal>openssl version -d</>.
|
|
This default can be overridden by setting environment variable
|
|
<envar>OPENSSL_CONF</envar> to the name of the desired configuration file.
|
|
</para>
|
|
|
|
<para>
|
|
<productname>OpenSSL</productname> supports a wide range of ciphers
|
|
and authentication algorithms, of varying strength. While a list of
|
|
ciphers can be specified in the <productname>OpenSSL</productname>
|
|
configuration file, you can specify ciphers specifically for use by
|
|
the database server by modifying <xref linkend="guc-ssl-ciphers"> in
|
|
<filename>postgresql.conf</>.
|
|
</para>
|
|
|
|
<note>
|
|
<para>
|
|
It is possible to have authentication without encryption overhead by
|
|
using <literal>NULL-SHA</> or <literal>NULL-MD5</> ciphers. However,
|
|
a man-in-the-middle could read and pass communications between client
|
|
and server. Also, encryption overhead is minimal compared to the
|
|
overhead of authentication. For these reasons NULL ciphers are not
|
|
recommended.
|
|
</para>
|
|
</note>
|
|
|
|
<para>
|
|
To start in <acronym>SSL</> mode, the files <filename>server.crt</>
|
|
and <filename>server.key</> must exist in the server's data directory.
|
|
These files should contain the server certificate and private key,
|
|
respectively.
|
|
On Unix systems, the permissions on <filename>server.key</filename> must
|
|
disallow any access to world or group; achieve this by the command
|
|
<command>chmod 0600 server.key</command>.
|
|
If the private key is protected with a passphrase, the
|
|
server will prompt for the passphrase and will not start until it has
|
|
been entered.
|
|
</para>
|
|
|
|
<para>
|
|
In some cases, the server certificate might be signed by an
|
|
<quote>intermediate</> certificate authority, rather than one that is
|
|
directly trusted by clients. To use such a certificate, append the
|
|
certificate of the signing authority to the <filename>server.crt</> file,
|
|
then its parent authority's certificate, and so on up to a <quote>root</>
|
|
authority that is trusted by the clients. The root certificate should
|
|
be included in every case where <filename>server.crt</> contains more than
|
|
one certificate.
|
|
</para>
|
|
|
|
<sect2 id="ssl-client-certificates">
|
|
<title>Using Client Certificates</title>
|
|
|
|
<para>
|
|
To require the client to supply a trusted certificate, place
|
|
certificates of the certificate authorities (<acronym>CA</acronym>s)
|
|
you trust in the file <filename>root.crt</filename> in the data
|
|
directory, and set the <literal>clientcert</literal> parameter
|
|
to 1 on the appropriate <literal>hostssl</> line(s) in
|
|
<filename>pg_hba.conf</>.
|
|
A certificate will then be requested from the client during
|
|
SSL connection startup. (See <xref linkend="libpq-ssl"> for a
|
|
description of how to set up certificates on the client.) The server will
|
|
verify that the client's certificate is signed by one of the trusted
|
|
certificate authorities. Certificate Revocation List (CRL) entries
|
|
are also checked if the file <filename>root.crl</filename> exists.
|
|
<!-- If this URL changes replace it with a URL to www.archive.org. -->
|
|
(See <ulink
|
|
url="http://h71000.www7.hp.com/DOC/83final/BA554_90007/ch04s02.html"></>
|
|
for diagrams showing SSL certificate usage.)
|
|
</para>
|
|
|
|
<para>
|
|
The <literal>clientcert</literal> option in <filename>pg_hba.conf</> is
|
|
available for all authentication methods, but only for rows specified as
|
|
<literal>hostssl</>. When <literal>clientcert</literal> is not specified
|
|
or is set to 0, the server will still verify presented client
|
|
certificates against <filename>root.crt</filename> if that file exists
|
|
— but it will not insist that a client certificate be presented.
|
|
</para>
|
|
|
|
<para>
|
|
Note that <filename>root.crt</filename> lists the top-level CAs that are
|
|
considered trusted for signing client certificates. In principle it need
|
|
not list the CA that signed the server's certificate, though in most cases
|
|
that CA would also be trusted for client certificates.
|
|
</para>
|
|
|
|
<para>
|
|
If you are setting up client certificates, you may wish to use
|
|
the <literal>cert</> authentication method, so that the certificates
|
|
control user authentication as well as providing connection security.
|
|
See <xref linkend="auth-cert"> for details.
|
|
</para>
|
|
</sect2>
|
|
|
|
<sect2 id="ssl-server-files">
|
|
<title>SSL Server File Usage</title>
|
|
|
|
<para>
|
|
<xref linkend="ssl-file-usage"> summarizes the files that are
|
|
relevant to the SSL setup on the server.
|
|
</para>
|
|
|
|
<table id="ssl-file-usage">
|
|
<title>SSL Server File Usage</title>
|
|
<tgroup cols="3">
|
|
<thead>
|
|
<row>
|
|
<entry>File</entry>
|
|
<entry>Contents</entry>
|
|
<entry>Effect</entry>
|
|
</row>
|
|
</thead>
|
|
|
|
<tbody>
|
|
|
|
<row>
|
|
<entry><filename>$PGDATA/server.crt</></entry>
|
|
<entry>server certificate</entry>
|
|
<entry>sent to client to indicate server's identity</entry>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><filename>$PGDATA/server.key</></entry>
|
|
<entry>server private key</entry>
|
|
<entry>proves server certificate was sent by the owner; does not indicate
|
|
certificate owner is trustworthy</entry>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><filename>$PGDATA/root.crt</></entry>
|
|
<entry>trusted certificate authorities</entry>
|
|
<entry>checks that client certificate is
|
|
signed by a trusted certificate authority</entry>
|
|
</row>
|
|
|
|
<row>
|
|
<entry><filename>$PGDATA/root.crl</></entry>
|
|
<entry>certificates revoked by certificate authorities</entry>
|
|
<entry>client certificate must not be on this list</entry>
|
|
</row>
|
|
|
|
</tbody>
|
|
</tgroup>
|
|
</table>
|
|
|
|
<para>
|
|
The files <filename>server.key</>, <filename>server.crt</>,
|
|
<filename>root.crt</filename>, and <filename>root.crl</filename>
|
|
are only examined during server start; so you must restart
|
|
the server for changes in them to take effect.
|
|
</para>
|
|
</sect2>
|
|
|
|
<sect2 id="ssl-certificate-creation">
|
|
<title>Creating a Self-signed Certificate</title>
|
|
|
|
<para>
|
|
To create a quick self-signed certificate for the server, use the
|
|
following <productname>OpenSSL</productname> command:
|
|
<programlisting>
|
|
openssl req -new -text -out server.req
|
|
</programlisting>
|
|
Fill out the information that <application>openssl</> asks for. Make sure
|
|
you enter the local host name as <quote>Common Name</>; the challenge
|
|
password can be left blank. The program will generate a key that is
|
|
passphrase protected; it will not accept a passphrase that is less
|
|
than four characters long. To remove the passphrase (as you must if
|
|
you want automatic start-up of the server), run the commands:
|
|
<programlisting>
|
|
openssl rsa -in privkey.pem -out server.key
|
|
rm privkey.pem
|
|
</programlisting>
|
|
Enter the old passphrase to unlock the existing key. Now do:
|
|
<programlisting>
|
|
openssl req -x509 -in server.req -text -key server.key -out server.crt
|
|
</programlisting>
|
|
to turn the certificate into a self-signed certificate and to copy
|
|
the key and certificate to where the server will look for them.
|
|
Finally do:
|
|
<programlisting>
|
|
chmod og-rwx server.key
|
|
</programlisting>
|
|
because the server will reject the file if its permissions are more
|
|
liberal than this.
|
|
For more details on how to create your server private key and
|
|
certificate, refer to the <productname>OpenSSL</> documentation.
|
|
</para>
|
|
|
|
<para>
|
|
A self-signed certificate can be used for testing, but a certificate
|
|
signed by a certificate authority (<acronym>CA</>) (either one of the
|
|
global <acronym>CAs</> or a local one) should be used in production
|
|
so that clients can verify the server's identity. If all the clients
|
|
are local to the organization, using a local <acronym>CA</> is
|
|
recommended.
|
|
</para>
|
|
|
|
</sect2>
|
|
|
|
</sect1>
|
|
|
|
<sect1 id="ssh-tunnels">
|
|
<title>Secure TCP/IP Connections with <application>SSH</application> Tunnels</title>
|
|
|
|
<indexterm zone="ssh-tunnels">
|
|
<primary>ssh</primary>
|
|
</indexterm>
|
|
|
|
<para>
|
|
It is possible to use <application>SSH</application> to encrypt the network
|
|
connection between clients and a
|
|
<productname>PostgreSQL</productname> server. Done properly, this
|
|
provides an adequately secure network connection, even for non-SSL-capable
|
|
clients.
|
|
</para>
|
|
|
|
<para>
|
|
First make sure that an <application>SSH</application> server is
|
|
running properly on the same machine as the
|
|
<productname>PostgreSQL</productname> server and that you can log in using
|
|
<command>ssh</command> as some user. Then you can establish a secure
|
|
tunnel with a command like this from the client machine:
|
|
<programlisting>
|
|
ssh -L 63333:localhost:5432 joe@foo.com
|
|
</programlisting>
|
|
The first number in the <option>-L</option> argument, 63333, is the
|
|
port number of your end of the tunnel; it can be any unused port.
|
|
(IANA reserves ports 49152 through 65535 for private use.) The
|
|
second number, 5432, is the remote end of the tunnel: the port
|
|
number your server is using. The name or IP address between the
|
|
port numbers is the host with the database server you are going to
|
|
connect to, as seen from the host you are logging in to, which
|
|
is <literal>foo.com</literal> in this example. In order to connect
|
|
to the database server using this tunnel, you connect to port 63333
|
|
on the local machine:
|
|
<programlisting>
|
|
psql -h localhost -p 63333 postgres
|
|
</programlisting>
|
|
To the database server it will then look as though you are really
|
|
user <literal>joe</literal> on host <literal>foo.com</literal>
|
|
connecting to <literal>localhost</literal> in that context, and it
|
|
will use whatever authentication procedure was configured for
|
|
connections from this user and host. Note that the server will not
|
|
think the connection is SSL-encrypted, since in fact it is not
|
|
encrypted between the
|
|
<application>SSH</application> server and the
|
|
<productname>PostgreSQL</productname> server. This should not pose any
|
|
extra security risk as long as they are on the same machine.
|
|
</para>
|
|
|
|
<para>
|
|
In order for the
|
|
tunnel setup to succeed you must be allowed to connect via
|
|
<command>ssh</command> as <literal>joe@foo.com</literal>, just
|
|
as if you had attempted to use <command>ssh</command> to create a
|
|
terminal session.
|
|
</para>
|
|
|
|
<para>
|
|
You could also have set up the port forwarding as
|
|
<programlisting>
|
|
ssh -L 63333:foo.com:5432 joe@foo.com
|
|
</programlisting>
|
|
but then the database server will see the connection as coming in
|
|
on its <literal>foo.com</literal> interface, which is not opened by
|
|
the default setting <literal>listen_addresses =
|
|
'localhost'</literal>. This is usually not what you want.
|
|
</para>
|
|
|
|
<para>
|
|
If you have to <quote>hop</quote> to the database server via some
|
|
login host, one possible setup could look like this:
|
|
<programlisting>
|
|
ssh -L 63333:db.foo.com:5432 joe@shell.foo.com
|
|
</programlisting>
|
|
Note that this way the connection
|
|
from <literal>shell.foo.com</literal>
|
|
to <literal>db.foo.com</literal> will not be encrypted by the SSH
|
|
tunnel.
|
|
SSH offers quite a few configuration possibilities when the network
|
|
is restricted in various ways. Please refer to the SSH
|
|
documentation for details.
|
|
</para>
|
|
|
|
<tip>
|
|
<para>
|
|
Several other applications exist that can provide secure tunnels using
|
|
a procedure similar in concept to the one just described.
|
|
</para>
|
|
</tip>
|
|
|
|
</sect1>
|
|
|
|
<sect1 id="event-log-registration">
|
|
<title>Registering <application>Event Log</> on <systemitem
|
|
class="osname">Windows</></title>
|
|
|
|
<indexterm zone="event-log-registration">
|
|
<primary>event log</primary>
|
|
<secondary>event log</secondary>
|
|
</indexterm>
|
|
|
|
<para>
|
|
To register a <systemitem class="osname">Windows</>
|
|
<application>event log</> library with the operating system,
|
|
issue this command:
|
|
<screen>
|
|
<userinput>regsvr32 <replaceable>pgsql_library_directory</>/pgevent.dll</>
|
|
</screen>
|
|
This creates registry entries used by the event viewer, under the default
|
|
event source named <literal>PostgreSQL</literal>.
|
|
</para>
|
|
|
|
<para>
|
|
To specify a different event source name (see
|
|
<xref linkend="guc-event-source">), use the <literal>/n</literal>
|
|
and <literal>/i</literal> options:
|
|
<screen>
|
|
<userinput>regsvr32 /n /i:<replaceable>event_source_name</> <replaceable>pgsql_library_directory</>/pgevent.dll</>
|
|
</screen>
|
|
</para>
|
|
|
|
<para>
|
|
To unregister the <application>event log</> library from
|
|
the operating system, issue this command:
|
|
<screen>
|
|
<userinput>regsvr32 /u [/i:<replaceable>event_source_name</>] <replaceable>pgsql_library_directory</>/pgevent.dll</>
|
|
</screen>
|
|
</para>
|
|
|
|
<note>
|
|
<para>
|
|
To enable event logging in the database server, modify
|
|
<xref linkend="guc-log-destination"> to include
|
|
<literal>eventlog</literal> in <filename>postgresql.conf</filename>.
|
|
</para>
|
|
</note>
|
|
</sect1>
|
|
|
|
</chapter>
|