Code review for a basic lexer

[uneven sleet]

First of all, please run rustfmt

about is_digit and is_alpha: they shouldn’t be associated methods of Scanner since they’re not really related to the type?

So just put them as small functions at the bottom

Also &char shouldn’t ever exist as a type

always use char instead

char is always smaller and faster to work with

[uneven sleet]

Same goes for write_char_to_str and str_to_token

Also write_char_to_str can be replaced with just ```rs
string.push(c);

there’s a dedicated method for it

https://doc.rust-lang.org/stable/std/string/struct.String.html#method.push

Also EOF should be in PascalCase like the rest of the variants: Eof

You can just do .copied() to go Option<&char> → Option<char> btw