Beluga_parser.LexerTokenization of UTF-8 text input.
val lex_string :
initial_location:Beluga_syntax.Location.t ->
string ->
Located_token.t Support.Seq.tlex_string initial_location input is the persistent sequence of tokens lexed from the string input, with token locations starting at initial_location.
val lex_input_channel :
initial_location:Beluga_syntax.Location.t ->
Stdlib.in_channel ->
Located_token.t Support.Seq.tlex_string initial_location input is the persistent sequence of tokens lexed from input channel input, with token locations starting at initial_location. You need to make sure to keep the input channel open throughout lexing, otherwise reading of subsequent tokens will fail.