Haskell Alex - an error in the wrapper template

Question

I am trying to understand Alex and lexers in general, but I have problems starting my lexer.

I wrote lexers in the "basic" and "posn" chips, but could not in the "monad" shell. I think I should use the monad wrapper because I need to collect the lines and token positions in the input. I also need several states. At the moment, I'm trying to run this simple exmaple:

 { module Main (main) where } %wrapper "monad" $whitespace = [\ \b\t\n\f\v\r] $digit = 0-9 $alpha = [a-zA-Z_] $upper = [AZ] $lower = [az] @tidentifier = $upper($alpha|_|$digit)* @identifier = $lower($alpha|_|$digit)* tokens :- $whitespace+ ; $upper $alpha+ { typeId } $lower $alpha+ { id_ } $digit+ { int } { data Lexeme = L AlexPosn LexemeClass String data LexemeClass = TypeId String | Id String | Int Int | EOF deriving (Show, Eq) typeId :: AlexInput -> Int -> Alex Lexeme typeId = undefined id_ :: AlexInput -> Int -> Alex Lexeme id_ = undefined int :: AlexInput -> Int -> Alex Lexeme int = undefined alexEOF = return (L undefined EOF "") main :: IO () main = do s <- getContents let r = runAlex s $ do return alexMonadScan print r } 

My actions are undefined now. When I try to compile it, I get this error:

 ➜ haskell ghc --make Tokens.hs [1 of 1] Compiling Main ( Tokens.hs, Tokens.o ) templates/wrappers.hs:208:17: Couldn't match expected type `(AlexPosn, Char, [Byte], String)' with actual type `(t0, t1, t2)' Expected type: AlexInput Actual type: (t0, t1, t2) In the return type of a call of `ignorePendingBytes' In the first argument of `action', namely `(ignorePendingBytes inp)' 

I also get various errors when I try to compile examples in Alex github repo, can this be due to version mismatch? I installed alex from cabal with ghc 7.0.4. Any ideas?

Answer 1

This seems like a bug in Alex 3.0.1. It works great in version 2.3.3 after addressing some other unrelated issues in code ¹ . The problem is this line in the generated code:

 ignorePendingBytes (p,c,ps,s) = (p,c,s) 

Following the types in the generated code, it seems that this function should be of type AlexInput -> AlexInput , but AlexInput , obviously, cannot be either a 3-tuple or a 4-tuple.

This probably happened because the definition of AlexInput was changed between the two versions.

 type AlexInput = (AlexPosn, Char, String) -- v2.3.3 type AlexInput = (AlexPosn, Char, [Byte], String) -- v3.0.1 

From what I can say, the correct code should be

 ignorePendingBytes (p,c,ps,s) = (p,c,[],s) 

and manually making this change to the generated code makes it compile after solving other problems.

However, if you do not need something from 3.0.1, I suggest lowering it until it is fixed, since the availability of corrections from the generated code is usually more of a problem than it costs.

_{¹ There is no Show instance for Lexeme in your code, and you also call return on alexMonadScan , which is already in the monad Alex .sub>}