PostgreSQL uses a message-based protocol for communication between frontends and backends. The protocol is implemented over TCP/IP and also on UNIX domain sockets.
Higher level features built on this protocol (for example, how libpq passes certain environment variables after the connection is established) are covered elsewhere.
The three major components are the frontend (running on the client), the postmaster and backend (running on the server). The postmaster and backend have different roles but may be implemented by the same executable.
A frontend opens a connection to the server and sends a start-up packet to the postmaster. This includes the names of the user and the database the user wants to connect to. The postmaster then uses this and the information in the pg_hba.conf file to determine what further authentication information it requires the frontend to send (if any) and responds to the frontend accordingly.
The frontend then sends any required authentication information. Once the postmaster validates this, it responds to the frontend that the user is authenticated and hands over the connection to a backend. The backend then sends a message indicating successful start-up (normal case) or failure (for example, an invalid database name).
In order to serve multiple clients efficiently, the server launches a new "backend" process for each client. This is transparent to the protocol, however. In the current implementation, a new child process is created immediately after an incoming connection is detected.
When the frontend wishes to disconnect, it sends an appropriate packet and closes the connection without waiting for a response from the backend.
Packets are sent as a data stream. The first byte determines what should be expected in the rest of the packet. The exception is packets sent from a frontend to the postmaster, which consist of a packet length followed by the packet itself. The difference is historical.