How tokenize regexp pattern yourself?

Question

Lots of questions on how tokenize some string using regexp.

I look, however, to tokenize the regex pattern, I'm sure there are some posts on this, but I cannot find them.

Examples:

^\w$                    -> ['^', '\w', '&']
[3-7]*                  -> ['[3-7]*']
\w+\s\w+                -> ['\w+', '\s', '\w+']
(xyz)*\s[a-zA-Z]+[0-9]? -> ['(xyz)*','\s','[a-zA-Z]+','[0-9]?']

I assume this work is done in python under the hood when a function is called regexp.

Answer 1

: PyPy Python ( ) Python. re.py sre-compile.c: _compile(), . , , .

. , Javascript XRegExp . .