[Summary view] [Print] [Text view]

   1  @title Configuring Inbound Email
   2  @group config
   3  
   4  This document contains instructions for configuring inbound email, so users
   5  may update Differential and Maniphest by replying to messages and create
   6  Maniphest tasks via email.
   7  
   8  = Preamble =
   9  
  10  This can be extremely difficult to configure correctly. This is doubly true if
  11  you use a local MTA.
  12  
  13  There are a few approaches available:
  14  
  15  | Receive Mail With | Setup | Cost | Notes |
  16  |--------|-------|------|-------|
  17  | Mailgun | Easy | Cheap | Recommended |
  18  | SendGrid | Easy | Cheap | |
  19  | Local MTA | Extremely Difficult | Free | Strongly discouraged! |
  20  
  21  The remainder of this document walks through configuring Phabricator to
  22  receive mail, and then configuring your chosen transport to deliver mail
  23  to Phabricator.
  24  
  25  = Configuring Phabricator =
  26  
  27  By default, Phabricator uses a `[email protected]` email address
  28  as the 'From' (configurable with `metamta.default-address`) and sets
  29  'Reply-To' to the user generating the email (e.g., by making a comment), if the
  30  mail was generated by a user action. This means that users can reply (or
  31  reply-all) to email to discuss changes, but the conversation won't be recorded
  32  in Phabricator and users will not be able to take actions like claiming tasks or
  33  requesting changes to revisions.
  34  
  35  To change this behavior so that users can interact with objects in Phabricator
  36  over email, set these configuration keys:
  37  
  38    - ##metamta.differential.reply-handler-domain##: enables email replies for
  39      Differential.
  40    - ##metamta.maniphest.reply-handler-domain##: enables email replies for
  41      Maniphest.
  42  
  43  Set these keys to some domain which you configure according to the instructions
  44  below, e.g. `phabricator.example.com`. You can set these both to the same
  45  domain, and will generally want to. Once you set these keys, emails will use a
  46  'Reply-To' like `[email protected]`, which -- when
  47  configured correctly, according to the instructions below -- will parse incoming
  48  email and allow users to interact with Maniphest tasks and Differential
  49  revisions over email.
  50  
  51  If you don't want Phabricator to take up an entire domain (or subdomain) you
  52  can configure a general prefix so you can use a single mailbox to receive mail
  53  on. To make use of this set `metamta.single-reply-handler-prefix` to the
  54  prefix of your choice, and Phabricator will prepend this to the 'Reply-To'
  55  mail address. This works because everything up to the first (optional) '+'
  56  character in an email-address is considered the receiver, and everything
  57  after is essentially ignored.
  58  
  59  You can also set up a task creation email address, like `[email protected]`,
  60  which will create a Maniphest task out of any email which is set to it. To do
  61  this, set `metamta.maniphest.public-create-email` in your configuration. This
  62  has some mild security implications, see below.
  63  
  64  = Security =
  65  
  66  The email reply channel is "somewhat" authenticated. Each reply-to address is
  67  unique to the recipient and includes a hash of user information and a unique
  68  object ID, so it can only be used to update that object and only be used to act
  69  on behalf of the recipient.
  70  
  71  However, if an address is leaked (which is fairly easy -- for instance,
  72  forwarding an email will leak a live reply address, or a user might take a
  73  screenshot), //anyone// who can send mail to your reply-to domain may interact
  74  with the object the email relates to as the user who leaked the mail. Because
  75  the authentication around email has this weakness, some actions (like accepting
  76  revisions) are not permitted over email.
  77  
  78  This implementation is an attempt to balance utility and security, but makes
  79  some sacrifices on both sides to achieve it because of the difficulty of
  80  authenticating senders in the general case (e.g., where you are an open source
  81  project and need to interact with users whose email accounts you have no control
  82  over).
  83  
  84  If you leak a bunch of reply-to addresses by accident, you can change
  85  `phabricator.mail-key` in your configuration to invalidate all the old hashes.
  86  
  87  You can also set `metamta.public-replies`, which will change how Phabricator
  88  delivers email. Instead of sending each recipient a unique mail with a personal
  89  reply-to address, it will send a single email to everyone with a public reply-to
  90  address. This decreases security because anyone who can spoof a "From" address
  91  can act as another user, but increases convenience if you use mailing lists and,
  92  practically, is a reasonable setting for many installs. The reply-to address
  93  will still contain a hash unique to the object it represents, so users who have
  94  not received an email about an object can not blindly interact with it.
  95  
  96  If you enable `metamta.maniphest.public-create-email`, that address also uses
  97  the weaker "From" authentication mechanism.
  98  
  99  NOTE: Phabricator does not currently attempt to verify "From" addresses because
 100  this is technically complex, seems unreasonably difficult in the general case,
 101  and no installs have had a need for it yet. If you have a specific case where a
 102  reasonable mechanism exists to provide sender verification (e.g., DKIM
 103  signatures are sufficient to authenticate the sender under your configuration,
 104  or you are willing to require all users to sign their email), file a feature
 105  request.
 106  
 107  = Testing and Debugging Inbound Email =
 108  
 109  You can use the `bin/mail` utility to test and review inbound mail. This can
 110  help you determine if mail is being delivered to Phabricator or not:
 111  
 112    phabricator/ $ ./bin/mail list-inbound   # List inbound messages.
 113    phabricator/ $ ./bin-mail show-inbound   # Show details about a message.
 114  
 115  You can also test receiving mail, but note that this just simulates receiving
 116  the mail and doesn't send any information over the network. It is
 117  primarily aimed at developing email handlers: it will still work properly
 118  if your inbound email configuration is incorrect or even disabled.
 119  
 120    phabricator/ $ ./bin/mail receive-test   # Receive test message.
 121  
 122  Run `bin/mail help <command>` for detailed help on using these commands.
 123  
 124  = Mailgun Setup =
 125  
 126  To use Mailgun, you need a Mailgun account. You can sign up at
 127  <http://www.mailgun.com>. Provided you have such an account, configure it
 128  like this:
 129  
 130    - Configure a mail domain according to Mailgun's instructions.
 131    - Add a Mailgun route with a `catch_all()` rule which takes the action
 132      `forward("https://phabricator.example.com/mail/mailgun/")`. Replace the
 133      example domain with your actual domain.
 134  
 135  = SendGrid Setup =
 136  
 137  To use SendGrid, you need a SendGrid account with access to the "Parse API" for
 138  inbound email. Provided you have such an account, configure it like this:
 139  
 140    - Configure an MX record according to SendGrid's instructions, i.e. add
 141      ##phabricator.example.com MX 10 mx.sendgrid.net.## or similar.
 142    - Go to the "Parse Incoming Emails" page on SendGrid
 143      (<http://sendgrid.com/developer/reply>) and add the domain as the
 144      "Hostname".
 145    - Add the URL ##https://phabricator.example.com/mail/sendgrid/## as the "Url",
 146      using your domain (and HTTP instead of HTTPS if you are not configured with
 147      SSL).
 148    - If you get an error that the hostname "can't be located or verified", it
 149      means your MX record is either incorrectly configured or hasn't propagated
 150      yet.
 151    - Set ##metamta.maniphest.reply-handler-domain## and/or
 152      ##metamta.differential.reply-handler-domain## to
 153      "##phabricator.example.com##" (whatever you configured the MX record for),
 154      depending on whether you want to support email replies for Maniphest,
 155      Differential, or both.
 156  
 157  That's it! If everything is working properly you should be able to send email
 158  to ##[email protected]## and it should appear in
 159  `bin/mail list-inbound` within a few seconds.
 160  
 161  = Local MTA: Installing Mailparse =
 162  
 163  If you're going to run your own MTA, you need to install the PECL mailparse
 164  extension. In theory, you can do that with:
 165  
 166    $ sudo pecl install mailparse
 167  
 168  You may run into an error like "needs mbstring". If so, try:
 169  
 170    $ sudo yum install php-mbstring # or equivalent
 171    $ sudo pecl install -n mailparse
 172  
 173  If you get a linker error like this:
 174  
 175    COUNTEREXAMPLE
 176    PHP Warning:  PHP Startup: Unable to load dynamic library
 177    '/usr/lib64/php/modules/mailparse.so' - /usr/lib64/php/modules/mailparse.so:
 178    undefined symbol: mbfl_name2no_encoding in Unknown on line 0
 179  
 180  ...you need to edit your php.ini file so that mbstring.so is loaded **before**
 181  mailparse.so. This is not the default if you have individual files in
 182  `php.d/`.
 183  
 184  = Local MTA: Configuring Sendmail =
 185  
 186  Before you can configure Sendmail, you need to install Mailparse. See the
 187  section "Installing Mailparse" above.
 188  
 189  Sendmail is very difficult to configure. First, you need to configure it for
 190  your domain so that mail can be delivered correctly. In broad strokes, this
 191  probably means something like this:
 192  
 193    - add an MX record;
 194    - make sendmail listen on external interfaces;
 195    - open up port 25 if necessary (e.g., in your EC2 security policy);
 196    - add your host to /etc/mail/local-host-names; and
 197    - restart sendmail.
 198  
 199  Now, you can actually configure sendmail to deliver to Phabricator. In
 200  `/etc/aliases`, add an entry like this:
 201  
 202    phabricator: "| /path/to/phabricator/scripts/mail/mail_handler.php"
 203  
 204  If you use the `PHABRICATOR_ENV` environmental variable to select a
 205  configuration, you can pass the value to the script as an argument:
 206  
 207    .../path/to/mail_handler.php <ENV>
 208  
 209  This is an advanced feature which is rarely used. Most installs should run
 210  without an argument.
 211  
 212  After making this change, run `sudo newaliases`. Now you likely need to symlink
 213  this script into `/etc/smrsh/`:
 214  
 215    sudo ln -s /path/to/phabricator/scripts/mail/mail_handler.php /etc/smrsh/
 216  
 217  Finally, edit ##/etc/mail/virtusertable## and add an entry like this:
 218  
 219    @yourdomain.com phabricator@localhost
 220  
 221  That will forward all mail to @yourdomain.com to the Phabricator processing
 222  script. Run ##sudo /etc/mail/make## or similar and then restart sendmail with
 223  `sudo /etc/init.d/sendmail restart`.
 224  
 225  = Local MTA: Configuring Lamson =
 226  
 227  Before you can configure Lamson, you need to install Mailparse. See the section
 228  "Installing Mailparse" above.
 229  
 230  In contrast to Sendmail, Lamson is relatively easy to configure. It is fairly
 231  minimal, and is suitable for a development or testing environment. Lamson
 232  listens for incoming SMTP mails and passes the content directly to Phabricator.
 233  
 234  To get started, follow the provided instructions
 235  (<http://lamsonproject.org/docs/getting_started.html>) to set up an instance.
 236  One likely deployment issue is that binding to port 25 requires root
 237  privileges. Lamson is capable of starting as root then dropping privileges, but
 238  you must supply ##-uid## and ##-gid## arguments to do so, as demonstrated by
 239  Step 8 in Lamson's deployment tutorial (located here:
 240  <http://lamsonproject.org/docs/deploying_oneshotblog.html>).
 241  
 242  The Lamson handler code itself is very concise; it merely needs to pass the
 243  content of the email to Phabricator:
 244  
 245    import logging, subprocess
 246    from lamson.routing import route, stateless
 247    from lamson import view
 248  
 249    PHABRICATOR_ROOT = "/path/to/phabricator"
 250    PHABRICATOR_ENV = "custom/myconf"
 251    LOGGING_ENABLED = True
 252  
 253    @route("(address)@(host)", address=".+")
 254    @stateless
 255    def START(message, address=None, host=None):
 256        if LOGGING_ENABLED:
 257            logging.debug("%s", message.original)
 258        process = subprocess.Popen([PHABRICATOR_ROOT + "scripts/mail/mail_handler.php",PHABRICATOR_ENV],stdin=subprocess.PIPE)
 259        process.communicate(message.original)