Module Beluga_parser.Lexer

Tokenization of UTF-8 text input.

val lex_string : initial_location:Beluga_syntax.Location.t -> string -> Located_token.t Support.Seq.t

lex_string initial_location input is the persistent sequence of tokens lexed from the string input, with token locations starting at initial_location.

val lex_input_channel : initial_location:Beluga_syntax.Location.t -> Stdlib.in_channel -> Located_token.t Support.Seq.t

lex_string initial_location input is the persistent sequence of tokens lexed from input channel input, with token locations starting at initial_location. You need to make sure to keep the input channel open throughout lexing, otherwise reading of subsequent tokens will fail.