There is no absolute answer to this question. A typical processing method is to create a special pattern for reading characters until it reaches (for example) a semicolon at the end of the line, giving a reasonable signal that everything after that is intended as a new declaration, definition, statement, etc. .d., and then re-parse the parsing from this point (keeping enough context to know that, for example, you are currently parsing the function body, so you accept / reject the input on this basis).
source share