clang API Documentation
#include <TokenConcatenation.h>
Public Member Functions | |
| TokenConcatenation (Preprocessor &PP) | |
| bool | AvoidConcat (const Token &PrevPrevTok, const Token &PrevTok, const Token &Tok) const |
TokenConcatenation class, which answers the question of "Is it safe to emit two tokens without a whitespace between them, or would that cause implicit concatenation of the tokens?"
For example, it emitting two identifiers "foo" and "bar" next to each other would cause the lexer to produce one "foobar" token. Emitting "1" and ")" next to each other is safe.
Definition at line 31 of file TokenConcatenation.h.
Definition at line 70 of file TokenConcatenation.cpp.
References clang::Preprocessor::getLangOpts().
| bool TokenConcatenation::AvoidConcat | ( | const Token & | PrevPrevTok, |
| const Token & | PrevTok, | ||
| const Token & | Tok | ||
| ) | const |
AvoidConcat - If printing PrevTok immediately followed by Tok would cause the two individual tokens to be lexed as a single token, return true (which causes a space to be printed between them). This allows the output of -E mode to be lexed to the same token stream as lexing the input directly would.
This code must conservatively return true if it doesn't want to be 100% accurate. This will cause the output to include extra space characters, but the resulting output won't have incorrect concatenations going on. Examples include "..", which we print with a space between, because we don't want to track enough to tell "x.." from "...".
Definition at line 157 of file TokenConcatenation.cpp.
References GetFirstChar(), clang::Token::getIdentifierInfo(), clang::Token::getKind(), clang::Preprocessor::getLangOpts(), clang::Token::getLength(), clang::Token::getLocation(), clang::SourceLocation::getLocWithOffset(), clang::Preprocessor::getSourceManager(), clang::SourceManager::getSpellingLoc(), clang::Token::hasUDSuffix(), clang::Token::is(), clang::Token::isAnnotation(), clang::isDigit(), clang::Token::isNot(), and clang::isPreprocessingNumberBody().
Referenced by clang::html::HighlightMacros().