clang 8.0.0

Public Member Functions | List of all members
clang::ast_matchers::dynamic::Parser::CodeTokenizer Class Reference

Simple tokenizer for the parser. More...

## Public Member Functions
| CodeTokenizer (StringRef MatcherCode, Diagnostics *Error)
| CodeTokenizer (StringRef MatcherCode, Diagnostics *Error, unsigned CodeCompletionOffset)
const TokenInfo & | peekNextToken () const
| Returns but doesn't consume the next token. More...
TokenInfo | consumeNextToken ()
| Consumes and returns the next token. More...
TokenInfo::TokenKind | nextTokenKind () const

## Detailed Description

Simple tokenizer for the parser.

Definition at line 67 of file Parser.cpp.

## Constructor & Destructor Documentation

## ◆ CodeTokenizer() [1/2]

clang::ast_matchers::dynamic::Parser::CodeTokenizer::CodeTokenizer | ( | StringRef | MatcherCode,
|  | Diagnostics * | Error
| ) |  |
| inline explicit

Definition at line 69 of file Parser.cpp.

## ◆ CodeTokenizer() [2/2]

clang::ast_matchers::dynamic::Parser::CodeTokenizer::CodeTokenizer | ( | StringRef | MatcherCode,
|  | Diagnostics * | Error,
|  | unsigned | CodeCompletionOffset
| ) |  |
| inline

Definition at line 74 of file Parser.cpp.

## Member Function Documentation

## ◆ consumeNextToken()

TokenInfo clang::ast_matchers::dynamic::Parser::CodeTokenizer::consumeNextToken | ( |  | ) |
| inline

Consumes and returns the next token.

Definition at line 85 of file Parser.cpp.

## ◆ nextTokenKind()

TokenInfo::TokenKind clang::ast_matchers::dynamic::Parser::CodeTokenizer::nextTokenKind | ( |  | ) | const
| inline

Definition at line 91 of file Parser.cpp.

References clang::ast_matchers::dynamic::SourceLocation::Column, clang::ast_matchers::dynamic::SourceRange::End, clang::isAlphanumeric(), clang::isHexDigit(), clang::isWhitespace(), clang::ast_matchers::dynamic::Parser::TokenInfo::Kind, clang::ast_matchers::dynamic::SourceLocation::Line, Line, clang::ast_matchers::dynamic::Parser::TokenInfo::Range, clang::ast_matchers::dynamic::SourceRange::Start, clang::ast_matchers::dynamic::Parser::TokenInfo::Text, clang::ast_matchers::dynamic::Parser::TokenInfo::TK_CloseParen, clang::ast_matchers::dynamic::Parser::TokenInfo::TK_CodeCompletion, clang::ast_matchers::dynamic::Parser::TokenInfo::TK_Comma, clang::ast_matchers::dynamic::Parser::TokenInfo::TK_Eof, clang::ast_matchers::dynamic::Parser::TokenInfo::TK_Error, clang::ast_matchers::dynamic::Parser::TokenInfo::TK_Ident, clang::ast_matchers::dynamic::Parser::TokenInfo::TK_InvalidChar, clang::ast_matchers::dynamic::Parser::TokenInfo::TK_Literal, clang::ast_matchers::dynamic::Parser::TokenInfo::TK_OpenParen, clang::ast_matchers::dynamic::Parser::TokenInfo::TK_Period, clang::toLowercase(), clang::ast_matchers::dynamic::Parser::TokenInfo::Value, and clang::ast_matchers::dynamic::Parser::Sema::~Sema().

## ◆ peekNextToken()

const TokenInfo& clang::ast_matchers::dynamic::Parser::CodeTokenizer::peekNextToken | ( |  | ) | const
| inline

Returns but doesn't consume the next token.

Definition at line 82 of file Parser.cpp.

Referenced by clang::ast_matchers::dynamic::Parser::parseExpression().

The documentation for this class was generated from the following file:

- lib/ASTMatchers/Dynamic/Parser.cpp