[Summary view] [Print] [Text view]

   1  @title Diffusion User Guide: Repository Hosting
   2  @group userguide
   3  
   4  Guide to configuring Phabricator repository hosting.
   5  
   6  = Overview =
   7  
   8  Phabricator can host repositories and provide authenticated read and write
   9  access to them over HTTP and SSH. This document describes how to configure
  10  repository hosting.
  11  
  12  = Understanding Supported Protocols =
  13  
  14  Phabricator supports hosting over these protocols:
  15  
  16  | VCS | SSH | HTTP |
  17  |-----|-----|------|
  18  | Git | Supported | Supported |
  19  | Mercurial | Supported | Supported |
  20  | Subversion | Supported | Not Supported |
  21  
  22  All supported protocols handle reads (pull/checkout/clone) and writes
  23  (push/commit). Of the two protocols, SSH is generally more robust, secure and
  24  performant, but HTTP is easier to set up and supports anonymous access.
  25  
  26  | | SSH | HTTP |
  27  | |-----|------|
  28  | Reads | Yes | Yes |
  29  | Writes | Yes | Yes |
  30  | Authenticated Access | Yes | Yes |
  31  | Push Logs | Yes | Yes |
  32  | Commit Hooks | Yes | Yes |
  33  | Anonymous Access | No | Yes |
  34  | Security | Better (Asymmetric Key) | Okay (Password) |
  35  | Performance | Better | Okay |
  36  | Setup | Hard | Easy |
  37  
  38  Each repository can be configured individually, and you can use either protocol,
  39  or both, or a mixture across different repositories.
  40  
  41  SSH is recommended unless you need anonymous access, or are not able to
  42  configure it for technical reasons.
  43  
  44  = Configuring System User Accounts =
  45  
  46  Phabricator uses as many as three user accounts. This section will guide you
  47  through creating and configuring them. These are system user accounts on the
  48  machine Phabricator runs on, not Phabricator user accounts.
  49  
  50  The system accounts are:
  51  
  52    - The user the daemons run as. We'll call this `daemon-user`. For more
  53      information on the daemons, see @{article:Managing Daemons with phd}. This
  54      user is the only user which will interact with the repositories directly.
  55      Other accounts will `sudo` to this account in order to perform VCS
  56      operations.
  57    - The user the webserver runs as. We'll call this `www-user`. If you do not
  58      plan to make repositories available over HTTP, you do not need to perform
  59      any special configuration for this user.
  60    - The user that users will connect over SSH as. We'll call this `vcs-user`.
  61      If you do not plan to make repositories available over SSH, you do not need
  62      to perform any special configuration for this user.
  63  
  64  To configure these users:
  65  
  66    - Create a `daemon-user` if one does not already exist (you can call this user
  67      whatever you want, or use an existing account). When you start the daemons,
  68      start them using this user.
  69    - Create a `www-user` if one does not already exist. Run your webserver as
  70      this user. In most cases, this user will already exist.
  71    - Create a `vcs-user` if one does not already exist. Common names for this
  72      user are `git` or `hg`. When users clone repositories, they will use a URI
  73      like `[email protected]`.
  74  
  75  Now, allow the `vcs-user` and `www-user` to `sudo` as the `daemon-user`. Add
  76  this to `/etc/sudoers`, using `visudo` or `sudoedit`.
  77  
  78  If you plan to use SSH:
  79  
  80    vcs-user ALL=(daemon-user) SETENV: NOPASSWD: /path/to/bin/git-upload-pack, /path/to/bin/git-receive-pack, /path/to/bin/hg, /path/to/bin/svnserve
  81  
  82  If you plan to use HTTP:
  83  
  84    www-user ALL=(daemon-user) SETENV: NOPASSWD: /usr/bin/git-http-backend, /usr/bin/hg
  85  
  86  Replace `vcs-user`, `www-user` and `daemon-user` with the right usernames for
  87  your configuration. Make sure all the paths point to the real locations of the
  88  binaries on your system. You can omit any binaries associated with VCSes you do
  89  not use.
  90  
  91  Adding these commands to `sudoers` will allow the daemon and webserver users to
  92  write to repositories as the daemon user.
  93  
  94  Before saving and closing `/etc/sudoers`, look for this line:
  95  
  96    Defaults    requiretty
  97  
  98  If it's present, comment it out by putting a `#` at the beginning of the line.
  99  With this option enabled, VCS SSH sessions won't be able to use `sudo`.
 100  
 101  If you're planning to use SSH, you should also edit `/etc/passwd` and
 102  `/etc/shadow` to make sure the `vcs-user` account is set up correctly.
 103  
 104    - Open `/etc/shadow` and find the line for the `vcs-user` account.
 105      - The second field (which is the password field) must not be set to
 106        `!!`. This value will prevent login. If it is set to `!!`, edit it
 107        and set it to `NP` ("no password") instead.
 108    - Open `/etc/passwd` and find the line for the `vcs-user` account.
 109      - The last field (which is the login shell) must be set to a real shell.
 110        If it is set to something like `/bin/false`, then `sshd` will not be able
 111        to execute commands. Instead, you should set it to a real shell, like
 112        `/bin/sh`.
 113  
 114  Finally, once you've configured `/etc/sudoers`, `/etc/shadow` and `/etc/passwd`,
 115  set `phd.user` to the `daemon-user`:
 116  
 117    phabricator/ $ ./bin/config set phd.user daemon-user
 118  
 119  If you're using a `vcs-user`, you should also configure that here:
 120  
 121    phabricator/ $ ./bin/config set diffusion.ssh-user vcs-user
 122  
 123  = Configuring HTTP =
 124  
 125  If you plan to use authenticated HTTP, you need to set
 126  `diffusion.allow-http-auth` in Config. If you don't plan to use HTTP, or plan to
 127  use only anonymous HTTP, you can leave this setting disabled.
 128  
 129  If you plan to use authenticated HTTP, you'll also need to configure a VCS
 130  password in {nav Settings > VCS Password}. This is a different password than
 131  your main Phabricator password primarily for security reasons.
 132  
 133  Otherwise, if you've configured system accounts above, you're all set. No
 134  additional server configuration is required to make HTTP work.
 135  
 136  = Configuring SSH =
 137  
 138  SSH access requires some additional setup. Here's an overview of how setup
 139  works:
 140  
 141    - You'll move the normal `sshd` daemon to another port, like `222`. When
 142      connecting to the machine to administrate it, you'll use this alternate
 143      port to get a normal login shell.
 144    - You'll run a highly restricted `sshd` on port 22, with a special locked-down
 145      configuration that uses Phabricator to authorize users and execute commands.
 146    - The `sshd` on port 22 **MUST** be 6.2 or newer, because Phabricator relies
 147      on the `AuthorizedKeysCommand` option.
 148  
 149  Here's a walkthrough of how to perform this configuration in detail:
 150  
 151  **Move Normal SSHD**: Be careful when editing the configuration for `sshd`. If
 152  you get it wrong, you may lock yourself out of the machine. Restarting `sshd`
 153  generally will not interrupt existing connections, but you should exercise
 154  caution. Two strategies you can use to mitigate this risk are: smoke-test
 155  configuration by starting a second `sshd`; and use a `screen` session which
 156  automatically repairs configuration unless stopped.
 157  
 158  To smoke-test a configuration, just start another `sshd` using the `-f` flag:
 159  
 160    sudo /path/to/sshd -f /path/to/config_file.edited
 161  
 162  You can then connect and make sure the edited config file is valid before
 163  replacing your primary configuration file.
 164  
 165  To automatically repair configuration, start a `screen` session with a command
 166  like this in it:
 167  
 168    sleep 60 ; mv sshd_config.good sshd_config ; /etc/init.d/sshd restart
 169  
 170  The specific command may vary for your system, but the general idea is to have
 171  the machine automatically restore configuration after some period of time if
 172  you don't stop it. If you lock yourself out, this will fix things automatically.
 173  
 174  Now that you're ready to edit your configuration, open up your `sshd` config
 175  (often `/etc/ssh/sshd_config`) and change the `Port` setting to some other port,
 176  like `222` (you can choose any port other than 22).
 177  
 178    Port 222
 179  
 180  Very carefully, restart `sshd`. Verify that you can connect on the new port:
 181  
 182    ssh -p 222 ...
 183  
 184  **Configure and Start Phabricator SSHD**: Now, configure and start a second
 185  `sshd` instance which will run on port `22`. This instance will use a special
 186  locked-down configuration that uses Phabricator to handle authentication and
 187  command execution.
 188  
 189  There are three major steps:
 190  
 191    - Create a `phabricator-ssh-hook.sh` file.
 192    - Create a `sshd_phabricator` config file.
 193    - Start a copy of `sshd` using the new configuration.
 194  
 195  **Create `phabricator-ssh-hook.sh`**: Copy the template in
 196  `phabricator/resources/sshd/phabricator-ssh-hook.sh` to somewhere like
 197  `/usr/libexec/phabricator-ssh-hook.sh` and edit it to have the correct
 198  settings. Then make it owned by `root` and restrict editing:
 199  
 200    sudo chown root /path/to/phabricator-ssh-hook.sh
 201    sudo chmod 755 /path/to/phabricator-ssh-hook.sh
 202  
 203  If you don't do this, `sshd` will refuse to execute the hook.
 204  
 205  **Create `sshd_config` for Phabricator**: Copy the template in
 206  `phabricator/resources/sshd/sshd_config.phabricator.example` to somewhere like
 207  `/etc/ssh/sshd_config.phabricator`.
 208  
 209  Open the file and edit the `AuthorizedKeysCommand`,
 210  `AuthorizedKeysCommandUser`, and `AllowUsers` settings to be correct for your
 211  system.
 212  
 213  **Start SSHD**: Now, start the Phabricator `sshd`:
 214  
 215    sudo /path/to/sshd -f /path/to/sshd_config.phabricator
 216  
 217  If you did everything correctly, you should be able to run this:
 218  
 219    echo {} | ssh [email protected] conduit conduit.ping
 220  
 221  ...and get a response like this:
 222  
 223    {"result":"orbital","error_code":null,"error_info":null}
 224  
 225  (If you get an authentication error, make sure you added your public key in
 226  **Settings > SSH Public Keys**.) If you're having trouble, check the
 227  troubleshooting section below.
 228  
 229  = Authentication Over HTTP =
 230  
 231  To authenticate over HTTP, users should configure a **VCS Password** in the
 232  **Settings** screen. This panel is available only if `diffusion.allow-http-auth`
 233  is enabled.
 234  
 235  = Authentication Over SSH =
 236  
 237  To authenticate over SSH, users should add **SSH Public Keys** in the
 238  **Settings** screen.
 239  
 240  = Cloning a Repository =
 241  
 242  If you've already set up a hosted repository, you can try cloning it now. To
 243  do this, browse to the repository's main screen in Diffusion. You should see
 244  clone commands at the top of the page.
 245  
 246  To clone the repository, just run the appropriate command.
 247  
 248  If you don't see the commands or running them doesn't work, see below for tips
 249  on troubleshooting.
 250  
 251  = Troubleshooting HTTP =
 252  
 253  Some general tips for troubleshooting problems with HTTP:
 254  
 255    - Make sure `diffusion.allow-http-auth` is enabled in your Phabricator config.
 256    - Make sure HTTP serving is enabled for the repository you're trying to clone.
 257      You can find this in {nav Edit Repository > Hosting}.
 258    - Make sure you've configured a VCS password. This is separate from your main
 259      account password. You can configure this in {nav Settings > VCS Password}.
 260    - Make sure the main repository screen in Diffusion shows a clone/checkout
 261      command for HTTP. If it doesn't, something above isn't set up correctly:
 262      double-check your configuration. You should see a `svn checkout http://...`,
 263      `git clone http://...` or `hg clone http://...` command. Run that command
 264      verbatim to clone the repository.
 265  
 266  If you're using Git, using `GIT_CURL_VERBOSE` may help assess login failures.
 267  To do so, specify it on the command line before the `git clone` command, like
 268  this:
 269  
 270    $ GIT_CURL_VERBOSE=1 git clone ...
 271  
 272  This will make `git` print out a lot more information. Particularly, the line
 273  with the HTTP response is likely to be useful:
 274  
 275    < HTTP/1.1 403 Invalid credentials.
 276  
 277  In many cases, this can give you more information about what's wrong.
 278  
 279  = Troubleshooting SSH =
 280  
 281  Some general tips for troubleshooting problems with SSH:
 282  
 283    - Check that you've configured `diffusion.ssh-user`.
 284    - Check that you've configured `phd.user`.
 285    - Make sure SSH serving is enabled for the repository you're trying to clone.
 286      You can change this setting from a main repository screen in Diffusion by
 287      {nav Edit Repository >
 288      Edit Hosting >
 289      Host Repository on Phabricator >
 290      Save and Continue >
 291      SSH Read Only or Read/Write >
 292      Save Changes}.
 293    - Make sure you've added an SSH public key to your account. You can do this
 294      in {nav Settings > SSH Public Keys}.
 295    - Make sure the main repository screen in Diffusion shows a clone/checkout
 296      command for SSH. If it doesn't, something above isn't set up correctly.
 297      You should see an `svn checkout svn+ssh://...`, `git clone ssh://...` or
 298      `hg clone ssh://...` command. Run that command verbatim to clone the
 299      repository.
 300    - Check your `phabricator-ssh-hook.sh` file for proper settings.
 301    - Check your `sshd_config.phabricator` file for proper settings.
 302  
 303  To troubleshoot SSH setup: connect to the server with `ssh`, without running
 304  a command. You may need to use the `-T` flag. You should see a message like
 305  this one:
 306  
 307    $ ssh -T [email protected]
 308    phabricator-ssh-exec: Welcome to Phabricator.
 309  
 310    You are logged in as alincoln.
 311  
 312    You haven't specified a command to run. This means you're requesting an
 313    interactive shell, but Phabricator does not provide an interactive shell over
 314    SSH.
 315  
 316    Usually, you should run a command like `git clone` or `hg push` rather than
 317    connecting directly with SSH.
 318  
 319    Supported commands are: conduit, git-receive-pack, git-upload-pack, hg,
 320    svnserve.
 321  
 322  If you see this message, all your SSH stuff is configured correctly. **If you
 323  get a login shell instead, you've missed some major setup step: review the
 324  documentation above.** If you get some other sort of error, double check these
 325  settings:
 326  
 327    - You're connecting as the `vcs-user`.
 328    - The `vcs-user` has `NP` in `/etc/shadow`.
 329    - The `vcs-user` has `/bin/sh` or some other valid shell in `/etc/passwd`.
 330    - Your SSH key is correct, and you've added it to Phabricator in the Settings
 331      panel.
 332  
 333  If you can get this far, but can't execute VCS commands like `git clone`, there
 334  is probably an issue with your `sudoers` configuration. Check:
 335  
 336    - Your `sudoers` file is set up as instructed above.
 337    - You've commented out `Defaults requiretty` in `sudoers`.
 338    - You don't have multiple copies of the VCS binaries (like `git-upload-pack`)
 339      on your system. You may have granted sudo access to one, while the VCS user
 340      is trying to run a different one.
 341    - You've configured `phd.user`.
 342    - The `phd.user` has read and write access to the repositories.
 343  
 344  It may also be helpful to run `sshd` in debug mode:
 345  
 346    $ /path/to/sshd -d -d -d -f /path/to/sshd_config.phabricator
 347  
 348  This will run it in the foreground and emit a large amount of debugging
 349  information.
 350  
 351  Finally, you can usually test that `sudoers` is configured correctly by
 352  doing something like this:
 353  
 354    $ su vcs-user
 355    $ sudo -E -n -u daemon-user -- /path/to/some/vcs-binary --help
 356  
 357  That will try to run the binary via `sudo` in a manner similar to the way that
 358  Phabricator will run it. This can give you better error messages about issues
 359  with `sudoers` configuration.
 360  
 361  = Miscellaneous Troubleshooting =
 362  
 363    - If you're getting an error about `svnlook` not being found, add the path
 364      where `svnlook` is located to the Phabricator configuration
 365      `environment.append-paths` (even if it already appears in PATH). This issue
 366      is caused by SVN wiping the environment (including PATH) when invoking
 367      commit hooks.
 368  
 369  = Next Steps =
 370  
 371  Once hosted repositories are set up:
 372  
 373    - learn about commit hooks with @{article:Diffusion User Guide: Commit Hooks}.