Token Types

There are several standard token types available for use and styling. In addition every type can have subtypes, for example we can have single quoted and double quoted string - both are strings, but different. So we can define string.single and string.double both are strings but string.single is not string.double.

Standard token types:

Token type Description Example
string String/Char literals 'single quoted' "double quoted"
number Number literals integers, doubles, floats etc. 10 .15 10.5e10
symbol Code symbols - class names, function names etc. Symbol
symbol.class Class Name SomeClass
symbol.class.interface Interface Name SomeInterface
symbol.function Function definition (NOT CALL!) FizzBuzz
symbol.annotation Annotation / Attribute / Decorator @annotation #[annotation]
symbol.parameter Parameter name for call --parameter [parameter: value]
constant Constants defined in language true IS_DEBUG
variable Variables, including sigils (like $ in PHP) $var
variable.property Property of an object $var->property variable.property
call Functions/subroutine/directives calls call()
format Formatting styles
format.italics Italic text italics
format.bold Bold text bold
format.strike Strike text strike
format.underline Underlined text underline
keyword Keywords foreach
operator Operators += / %
operator.punctuation Punctuation operators ; , .
operator.punctuation.brackets Various brackets () {} []
delimiter Language/sections delimiters <?php <%
language.name Embedded name language
comment Comment // some comment
comment.docblock Documentation block comment /// Documentation comment
preprocessor Preprocessor definition #preprocessor directive

Every language can actually define new top-level kinds of tokens if needed - but it is highly unrecommended as it'd be impossible for style makers to know them all. Instead of creating new top-level token kinds it's recommended to stick to those predefined and extend them if needed.

For example, in LaTeX we have math mode for defining math expressions, and we have no default token for such thing. So, instead of creating new top-level math token type, we should extend generic expression token and create expression.math. This way styles without knowledge about LaTeX specifics could still handle math as part of a generic expression.