RegEx with variable data in it - ply.lex

Question

im using python module ply.lexto write a lexer. I got some of my tokens given with regex, but now I'm stuck with it. I have list of Keywordsone that should be token. data- This is a list of 1000 keywords that should all be recognized as one of the keywords. It can be, for example: _Function1 _UDFType2etc. All words in the list are separated by spaces. I just want lexer to recognize the words in this list so that it returns a token like `KEYWORD.

data = 'Keyword1 Keyword2 Keyword3 Keyword4'
def t_KEYWORD(t):
    # ... r'\$' + data ??
    return t

text = '''
Some test data


even more

$var = 2231




$[]Test this 2.31 + / &
'''

autoit = lex.lex()
autoit.input(text)
while True:
    tok = autoit.token()
    if not tok: break
    print(tok)

So, I tried to add a variable to this regex, but that didn't work. I always get:   No regular expression defined for rule 't_KEYWORD'.

Thank you in advance! John

Answer 1

@DSM, TOKEN. cat dog 'cat|dog' (.. , '|', ). :

from ply.lex import TOKEN
data = data.split() #make data a list of keywords

@TOKEN('|'.join(data))
def t_KEYWORD(t):
    return t

Answer 2

ply.lex docstring . , , , , .

.

docstring , .

:

def f():
    "this is " + "my help"  #not a docstring :(
f.func_doc #is None
f.func_doc = "this is " + "my help" #now it is!

, :

def t_KEYWORD(token):
    return token
t_KEYWORD.func_doc=r'REGULAR EXPRESSION HERE' #can be an expression

Answer 3

, ply, docstring __doc__ , , , , __doc__ ply .