Token Types

There are several standard token types available for use and styling. In addition every type can have subtypes, for example we can have single quoted and double quoted string - both are strings, but different. So we can define string.single and string.double both are strings but string.single is not string.double.

Standard token types:

Token type Description Example
string String/Char literals 'single quoted' "double quoted"
number Number literals integers, doubles, floats etc. 10 .15 10.5e10
symbol Code symbols - class names, function names etc. Symbol
symbol.class Class Name SomeClass
symbol.class.interface Interface Name SomeInterface
symbol.function Function definition (NOT CALL!) FizzBuzz
symbol.annotation Annotation @annotation [annotation]
constant Constants defined in language true IS_DEBUG
variable Variables, including sigils (like $ in PHP) $var
variable.property Property of object $var->property variable.property
call Functions/subroutine/directives calls call()
format Formatting styles
format.italics Italic text italics
format.bold Bold text bold
format.strike Strike text strike
format.underline Underlined text underline
keyword Keywords foreach
operator Operators += / %
operator.punctuation Punctuation operators ; , .
delimiter Language delimiters <?php <%
language.name Embedded name language
comment Comment // some comment
comment.docblock Documentation block comment /// Documentation comment
preprocessor Preprocessor definition #preprocessor directive

However every language can define new types of tokens, but I suggest to stick with standard and just extend them. Create new tokens only when it's really necessary and none from list above makes semantic sense.