Filter ANSI escape sequences from ByteString with Conduit

Question

I am trying to make a Conduit that filters ANSI escape codes from ByteStrings. I came up with a function that converts a ByteString to a Word8 stream, performs filtering, and converts back to a ByteStream at the end.

It seems to work fine when I use it in GHCi:

> runConduit $ yield "hello\27[23;1m world" .| ansiFilter .| printC "hello world" 

When I use it in my application, cables containing ansiFilter do not seem to miss anything. Here is the complete source:

 {-# LANGUAGE OverloadedStrings #-} module Main where import Conduit import Control.Concurrent.Async import Control.Concurrent.STM import Data.ByteString (ByteString) import qualified Data.ByteString as BS import Data.Conduit.TQueue import Data.Word8 (Word8) import qualified Data.Word8 as Word8 main :: IO () main = do queue <- atomically $ newTBQueue 25 let qSource = sourceTBQueue queue atomically $ writeTBQueue queue ("hello" :: ByteString) race_ (putInputIntoQueue queue) (doConversionAndPrint qSource) putInputIntoQueue q = runConduit $ stdinC .| iterMC (atomically . writeTBQueue q) .| sinkNull doConversionAndPrint src = runConduit $ src .| ansiFilter .| stdoutC ansiFilter :: MonadIO m => ConduitM ByteString ByteString m () ansiFilter = toWord8 .| ansiFilter' .| toByteString where ansiFilter' = awaitForever $ \first -> do msecond <- peekC case (first, msecond) of (0x1b, Just 0x5b) -> do dropWhileC (not . Word8.isLetter) dropC 1 _ -> yield first toWord8 = concatC toByteString :: Monad m => ConduitM Word8 ByteString m () toByteString = (mapC BS.singleton .| foldC) >>= yield 

This program should track the filtered contents of stdin , but nothing returns.

However, if I comment on ansiFilter in doConversionAndPrint , an echo, it does work that makes the ansiFilter function ansiFilter .

Any help would be greatly appreciated!

Answer 1

I redefined ansiFilter in terms of higher level data functions in a combinator channel like takeWhileCE . This seems to work and should be more efficient, allowing more data to remain in an efficient memory representation:

 ansiFilter :: MonadIO m => ConduitM ByteString ByteString m () ansiFilter = loop where loop = do takeWhileCE (/= 0x1b) mfirst <- headCE case mfirst of Nothing -> return () Just first -> assert (first == 0x1b) $ do msecond <- peekCE case msecond of Just 0x5b -> do dropWhileCE (not . Word8.isLetter) dropCE 1 _ -> yield $ BS.singleton first loop 

Answer 2

I went a little different and I was more fortunate than just ByteStrings. I think this renders some of the streaming material, but acceptable for my use.

 ansiFilter :: Monad m => Conduit ByteString m ByteString ansiFilter = mapC (go "") where csi = "\27[" go acc "" = acc go acc remaining = go (acc <> filtered) (stripCode unfiltered) where (filtered, unfiltered) = BS.breakSubstring csi remaining stripCode bs = BS.drop 1 (BS.dropWhile (not . Word8.isLetter) bs)