Insidion Engine
Share
Explore
Insidion Engine

Lexing

⁠
Lexing, or lexical analysis, or tokenisation, is basically taking the character stream and producing a token stream. A token has the following attributes:
type: Type
id: String
lineNo: int
columnNo: int
children: List<Token>

Gallery
Share
 
Want to print your doc?
This is not the way.
Try clicking the ⋯ next to your doc name or using a keyboard shortcut (
CtrlP
) instead.