[Summary view] [Print] [Text view]

   1  @title Arcanist User Guide: Lint
   2  @group userguide
   3  
   4  Guide to lint, linters, and linter configuration.
   5  
   6  This is a configuration guide that helps you set up advanced features. If you're
   7  just getting started, you don't need to look at this yet. Instead, start with
   8  the @{article:Arcanist User Guide}.
   9  
  10  This guide explains how lint works when configured in an `arc` project. If
  11  you haven't set up a project yet, do that first. For instructions, see
  12  @{article:Arcanist User Guide: Configuring a New Project}.
  13  
  14  
  15  Overview
  16  ========
  17  
  18  "Lint" refers to a general class of programming tools which analyze source code
  19  and raise warnings and errors about it. For example, a linter might raise
  20  warnings about syntax errors, uses of undeclared variables, calls to deprecated
  21  functions, spacing and formatting conventions, misuse of scope, implicit
  22  fallthrough in switch statements, missing license headers, use of dangerous
  23  language features, or a variety of other issues.
  24  
  25  Integrating lint into your development pipeline has two major benefits:
  26  
  27    - you can detect and prevent a large class of programming errors; and
  28    - you can simplify code review by addressing many mechanical and formatting
  29      problems automatically.
  30  
  31  When arc is integrated with a lint toolkit, it enables the `arc lint` command
  32  and runs lint on changes during `arc diff`. The user is prompted to fix errors
  33  and warnings before sending their code for review, and lint issues which are
  34  not fixed are visible during review.
  35  
  36  There are many lint and static analysis tools available for a wide variety of
  37  languages. Arcanist ships with bindings for many popular tools, and you can
  38  write new bindings fairly easily if you have custom tools.
  39  
  40  
  41  Available Linters
  42  =================
  43  
  44  To see a list of available linters, run:
  45  
  46    $ arc linters
  47  
  48  Arcanist ships with bindings for a number of linters that can check for errors
  49  or problems in JS, CSS, PHP, Python, C, C++, C#, Less, Puppet, Ruby, JSON, XML,
  50  and several other languages.
  51  
  52  Some general purpose linters are also available. These linters can check for
  53  cross-language issues like sensible filenames, trailing or mixed whitespace,
  54  character sets, spelling mistakes, and unresolved merge conflicts.
  55  
  56  If you have a tool you'd like to use as a linter that isn't supported by
  57  default, you can write bindings for it. For information on writing new linter
  58  bindings, see @{article:Arcanist User Guide: Customizing Lint, Unit Tests and
  59  Workflows}.
  60  
  61  
  62  Configuring Lint
  63  ================
  64  
  65  To configure lint integration for your project, create a file called `.arclint`
  66  at the project root. This file should be in JSON format, and look like this:
  67  
  68  ```lang=js
  69  {
  70    "linters": {
  71      "sample": {
  72        "type": "pep8"
  73      }
  74    }
  75  }
  76  ```
  77  
  78  Here, the key ("sample") is a human-readable label identifying the linter. It
  79  does not affect linter behavior, so just choose something that makes sense to
  80  you.
  81  
  82  The `type` specifies which linter to run. Use `arc linters` to find the names of
  83  the available linters.
  84  
  85  **Including and Excluding Files**: By default, a linter will run on every file.
  86  This is appropriate for some linters (like the Filename linter), but normally
  87  you only want to run a linter like **pep8** on Python files. To include or
  88  exclude files, use `include` and `exclude`:
  89  
  90  ```lang=js
  91  {
  92    "linters": {
  93      "sample": {
  94        "type": "pep8",
  95        "include": "(\\.py$)",
  96        "exclude": "(^third-party/)"
  97      }
  98    }
  99  }
 100  ```
 101  
 102  The `include` key is a regular expression (or list of regular expressions)
 103  identifying paths the linter should be run on, while `exclude` is a regular
 104  expression (or list of regular expressions) identifying paths which it should
 105  not run on.
 106  
 107  Thus, this configures a **pep8** linter named "sample" which will run on files
 108  ending in ".py", unless they are inside the "third-party/" directory.
 109  
 110  In these examples, regular expressions are written in this style:
 111  
 112    "(example/path)"
 113  
 114  They can be specified with any delimiters, but using `(` and `)` means you don't
 115  have to escape slashes in the expression, so it may be more convenient to
 116  specify them like this. If you prefer, these are all equivalent:
 117  
 118    "(example/path)i"
 119    "/example\\/path/i"
 120    "@example/path@i"
 121  
 122  You can also exclude files globally, so no linters run on them at all. Do this
 123  by specifying `exclude` at top level:
 124  
 125  ```lang=js
 126  {
 127    "exclude": "(^tests/data/)",
 128    "linters": {
 129      "sample": {
 130        "type": "pep8",
 131        "include": "(\\.py$)",
 132        "exclude": "(^third-party/)"
 133      }
 134    }
 135  }
 136  ```
 137  
 138  Here, the addition of a global `exclude` rule means no linter will be run on
 139  files in "tests/data/".
 140  
 141  **Running Multiple Linters**: Often, you will want to run several different
 142  linters. Perhaps your project has a mixture of Python and Javascript code, or
 143  you have some PHP and some JSON files. To run multiple linters, just list
 144  them in the `linters` map:
 145  
 146  ```lang=js
 147  {
 148    "linters": {
 149      "jshint": {
 150        "type": "jshint",
 151        "include": "(\\.js$)"
 152      },
 153      "xml": {
 154        "type": "xml",
 155        "include": "(\\.xml$)"
 156      }
 157    }
 158  }
 159  ```
 160  
 161  This will run JSHint on `.js` files, and SimpleXML on `.xml` files.
 162  
 163  **Adjusting Message Severities**: Arcanist raises lint messages at various
 164  severities. Each message has a different severity: for example, lint might
 165  find a syntax error and raise an `error` about it, and find trailing whitespace
 166  and raise a `warning` about it.
 167  
 168  Normally, you will be presented with lint messages as you are sending code for
 169  review. In that context, the severities behave like this:
 170  
 171    - `error` When a file contains lint errors, they are always reported. These
 172      are intended to be severe problems, like a syntax error. Unresoved lint
 173      errors require you to confirm that you want to continue.
 174    - `warning` When a file contains warnings, they are reported by default only
 175      if they appear on lines that you have changed. They are intended to be
 176      minor problems, like unconventional whitespace. Unresolved lint warnings
 177      require you to confirm that you want to continue.
 178    - `autofix` This level is like `warning`, but if the message includes patches
 179      they will be applied automatically without prompting.
 180    - `advice` Like warnings, these messages are only reported on changed lines.
 181      They are intended to be very minor issues which may merit a note, like a
 182      "TODO" comment. They do not require confirmation.
 183    - `disabled` This level suppresses messages. They are not displayed. You can
 184      use this to turn off a message if you don't care about the issue it
 185      detects.
 186  
 187  By default, Arcanist tries to select reasonable severities for each message.
 188  However, you may want to make a message more or less severe, or disable it
 189  entirely.
 190  
 191  For many linters, you can do this by providing a `severity` map:
 192  
 193  ```lang=js
 194  {
 195    "linters": {
 196      "sample": {
 197        "type": "pep8",
 198        "severity": {
 199          "E221": "disabled",
 200          "E401": "warning"
 201        }
 202      }
 203    }
 204  }
 205  ```
 206  
 207  Here, the lint message "E221" (which is "multiple spaces before operator") is
 208  disabled, so it won't be shown. The message "E401" (which is "multiple imports
 209  on one line") is set to "warning" severity.
 210  
 211  If you want to remap a large number of messages, you can use `severity.rules`
 212  and specify regular expressions:
 213  
 214  ```lang=js
 215  {
 216    "linters": {
 217      "sample": {
 218        "type": "pep8",
 219        "severity.rules": {
 220          "(^E)": "warning",
 221          "(^W)": "advice"
 222        }
 223      }
 224    }
 225  }
 226  ```
 227  
 228  This adjusts the severity of all "E" codes to "warning", and all "W" codes to
 229  "advice".
 230  
 231  **Locating Binaries and Interpreters**: Normally, Arcanist expects to find
 232  external linters (like `pep8`) in `$PATH`, and be able to run them without any
 233  special qualifiers. That is, it will run a command similar to:
 234  
 235    $ pep8 example.py
 236  
 237  If you want to use a different copy of a linter binary, or invoke it in an
 238  explicit way, you can use `interpreter` and `bin`. These accept strings (or
 239  lists of strings) identifying places to look for linters. For example:
 240  
 241  
 242  ```lang=js
 243  {
 244    "linters": {
 245      "sample": {
 246        "type": "pep8",
 247        "interpreter": ["python2.6", "python"],
 248        "bin": ["/usr/local/bin/pep8-1.5.6", "/usr/local/bin/pep8"]
 249      }
 250    }
 251  }
 252  ```
 253  
 254  When configured like this, `arc` will walk the `interpreter` list to find an
 255  available interpreter, then walk the `bin` list to find an available binary.
 256  If it can locate an appropriate interpreter and binary, it will execute those
 257  instead of the defaults. For example, this might cause it to execute a command
 258  similar to:
 259  
 260    $ python2.6 /usr/local/bin/pep8-1.5.6 example.py
 261  
 262  **Additional Options**: Some linters support additional options to configure
 263  their behavior. You can run this command get a list of these options and
 264  descriptions of what they do and how to configure them:
 265  
 266    $ arc linters --verbose
 267  
 268  This will show the available options for each linter in detail.
 269  
 270  **Running Different Rules on Different Files**: Sometimes, you may want to
 271  run the same linter with different rulesets on different files. To do this,
 272  create two copies of the linter and just give them different keys in the
 273  `linters` map:
 274  
 275  ```lang=js
 276  {
 277    "linters": {
 278      "pep8-relaxed": {
 279        "type": "pep8",
 280        "include": "(^legacy/.*\\.py$)",
 281        "severity.rules": {
 282          "(.*)": "advice"
 283        }
 284      },
 285      "pep8-normal": {
 286        "type": "pep8",
 287        "include": "(\\.py$)",
 288        "exclude": "(^legacy/)"
 289      }
 290    }
 291  }
 292  ```
 293  
 294  This example will run a relaxed version of the linter (which raises every
 295  message as advice) on Python files in "legacy/", and a normal version everywhere
 296  else.
 297  
 298  **Example .arclint Files**: You can find a collection of example files in
 299  `arcanist/resources/arclint/` to use as a starting point or refer to while
 300  configuring your own `.arclint` file.
 301  
 302  Advanced Configuration: Lint Engines
 303  ====================================
 304  
 305  If you need to specify how linters execute in greater detail than is possible
 306  with `.arclint`, you can write a lint engine in PHP to extend Arcanist. This is
 307  an uncommon, advanced use case. The remainder of this section overviews how the
 308  lint internals work, and discusses how to extend Arcanist with a custom lint
 309  engine. If your needs are met by `.arclint`, you can skip to the next section
 310  of this document.
 311  
 312  The lint pipeline has two major components: linters and lint engines.
 313  
 314  **Linters** are programs which detect problems in a source file. Usually a
 315  linter is an external script, which Arcanist runs and passes a path to, like
 316  `jshint` or `pep8`.
 317  
 318  The script emits some messages, and Arcanist parses the output into structured
 319  errors. A piece of glue code (like @{class@arcanist:ArcanistJSHintLinter} or
 320  @{class@arcanist:ArcanistPEP8Linter}) handles calling the external script and
 321  interpreting its output.
 322  
 323  **Lint engines** coordinate linters, and decide which linters should run on
 324  which files. For instance, you might want to run `jshint` on all your `.js`
 325  files, and `pep8.py` on all your `.py` files. And you might not want to lint
 326  anything in `externals/` or `third-party/`, and maybe there are other files
 327  which you want to exclude or apply special rules for.
 328  
 329  By default, Arcanist uses the
 330  @{class@arcanist:ArcanistConfigurationDrivenLintEngine} engine if there is
 331  an `.arclint` file present in the working copy. This engine reads the `.arclint`
 332  file and uses it to decide which linters should be run on which paths. If no
 333  `.arclint` is present, Arcanist does not select an engine by default.
 334  
 335  You can write a custom lint engine instead, which can make a more powerful set
 336  of decisions about which linters to run on which paths. For instructions on
 337  writing a custom lint engine, see @{article:Arcanist User Guide: Customizing
 338  Lint, Unit Tests and Workflows}.
 339  
 340  To name an alternate lint engine, set `lint.engine` in your `.arcconfig` to the
 341  name of a class which extends @{class@arcanist:ArcanistLintEngine}. For more
 342  information on `.arcconfig`, see @{article:Arcanist User Guide: Configuring a
 343  New Project}.
 344  
 345  You can also set a default lint engine by setting `lint.engine` in your global
 346  user config with `arc set-config lint.engine`, or specify one explicitly with
 347  `arc lint --engine <engine>`. This can be useful for testing.
 348  
 349  There are several other engines bundled with Arcanist, but they are primarily
 350  predate `.arclint` and are effectively obsolete.
 351  
 352  
 353  Using Lint to Improve Code Review
 354  =================================
 355  
 356  Code review is most valuable when it's about the big ideas in a change. It is
 357  substantially less valuable when it devolves into nitpicking over style,
 358  formatting, and naming conventions.
 359  
 360  The best response to receiving a review request full of style problems is
 361  probably to reject it immediately, point the author at your coding convention
 362  documentation, and ask them to fix it before sending it for review. But even
 363  this is a pretty negative experience for both parties, and less experienced
 364  reviewers sometimes go through the whole review and point out every problem
 365  individually.
 366  
 367  Lint can greatly reduce the negativity of this whole experience (and the amount
 368  of time wasted arguing about these things) by enforcing style and formatting
 369  rules automatically. Arcanist supports linters that not only raise warnings
 370  about these problems, but provide patches and fix the problems for the author --
 371  before the code goes to review.
 372  
 373  Good linter integration means that code is pretty much mechanically correct by
 374  the time any reviewer sees it, provides clear rules about style which are
 375  especially helpful to new authors, and has the overall effect of pushing
 376  discussion away from stylistic nitpicks and toward useful examination of large
 377  ideas.
 378  
 379  It can also provide a straightforward solution to arguments about style, if you
 380  adopt a policy like this:
 381  
 382    - If a rule is important enough that it should be enforced, the proponent must
 383      add it to lint so it is automatically detected or fixed in the future and
 384      no one has to argue about it ever again.
 385    - If it's not important enough for them to do the legwork to add it to lint,
 386      they have to stop complaining about it.
 387  
 388  This may or may not be an appropriate methodology to adopt at your organization,
 389  but it generally puts the incentives in the right places.
 390  
 391  
 392  Philosophy of Lint
 393  ==================
 394  
 395  Some general thoughts on how to develop lint effectively, based on building
 396  lint tools at Facebook:
 397  
 398    - Don't write regex-based linters to enforce language rules. Use a real parser
 399      or AST-based tool. This is not a domain you can get right at any nontrivial
 400      complexity with raw regexes. That is not a challenge. Just don't do this.
 401    - False positives are pretty bad and should be avoided. You should aim to
 402      implement only rules that have very few false positives, and provide ways to
 403      mark false positives as OK. If running lint always raises 30 warnings about
 404      irrelevant nonsense, it greatly devalues the tool.
 405    - Move toward autocorrect rules. Most linters do not automatically correct
 406      the problems they detect, but Arcanist supports this and it's quite
 407      valuable to have the linter not only say "the convention is to put a space
 408      after comma in a function call" but to fix it for you.
 409  
 410  = Next Steps =
 411  
 412  Continue by:
 413  
 414    - integrating and customizing built-in linters and lint bindings with
 415      @{article:Arcanist User Guide: Customizing Existing Linters}; or
 416    - use a linter that hasn't been integrated into Arcanist with
 417      @{article:Arcanist User Guide: Script and Regex Linter}; or
 418    - learning how to add new linters and lint engines with
 419      @{article:Arcanist User Guide: Customizing Lint, Unit Tests and Workflows}.