These are not spaces, these are empty lines. Your line:
"1 normal 1 [(o, 21) (o, 17) (t, 3)]"
It is broken as follows according to your regular expression:
Token = "1" Delimiter = " " Token = "normal" Delimiter = " " Token = "1" Delimiter = " " Token = "" <-- empty string Delimiter = "[(" Token = "o" ... end so on
When two adjacent delimiters appear, he believes that there is an empty token between them.
To fix this, you can change your regular expression, for example, as follows:
"[ \\(\\)\\[\\,\\]]+"
Thus, any number of " ()[,]" adjacent characters will be considered a delimiter.
source share