How to perform complex I / O processing and implicit cache in Haskell?

Question

Large applications often use several levels of IO caching (Hibernate L1 and L2, Spring, etc.), which are usually abstracted, so the caller does not need to know that a particular implementation does IO. With some caveats (volume, transactions), it allows you to simplify the interface between components.

For example, if component A needs to query the database, it does not need to know if the result will already be cached. It may have been found by B or C, of ​​which A knows nothing, but they usually participate in some sessions or transactions - often implicitly.

Frames typically make this call indistinguishable from simply calling an object method using methods such as AOP.

Is it possible for Haskell applications to benefit? What does the client interface look like?

Answer 1

In Haskell, there are many ways to compute components, which are their separate responsibilities. This can be done at the data level with data types and functions ( http://www.haskellforall.com/2012/05/scrap-your-type-classes.html ) or using type classes. In Haskell, you can view each data type, type, function, signature, class, etc. As an interface; if you have something else of the same type, you can replace the component with something compatible.

When we want to talk about calculations in Haskell, we often use an abstraction of a Monad . A Monad is an interface for building computations. A basic calculation can be built using return , and they can be composed along with functions that perform other calculations using >>= . When we want to add a few responsibilities to the calculations presented by monads, we make monad transformers. In the code below, there are four different monad transformers that capture various aspects of a tiered system:

DatabaseT s adds a database with a schema of type s . It processes Operation data, storing data or retrieving it from the database. CacheT s intercepts Operation data for schema s and retrieves data from memory, if available. OpperationLoggerT writes Operation to standard output ResultLoggerT writes Operation results to standard output

These four components are combined together using a class (interface) of type MonadOperation s , which requires that the components implementing it provide a way to perform a Operation and return its result.

The same type describes what is required to use the MonadOperation s system. This requires that someone using the interface provide an implementation of the type classes on which the database and cache will be based. There are also two types of data that are part of this interface, Operation and CRUD . Please note that the interface should not know anything about domain objects or the database schema, and also should not know about the different monad transformers that will implement it. Monad transformers do not know anything about circuit or domain objects, and domain objects and sample code do not know anything about monad transformers that build the system.

The only thing the example code knows is access to MonadOperation s due to its type example :: (MonadOperation TableName m) => m () .

The main program runs the example twice in two different contexts. The first time the program talks to the database, its Operations and responses are recorded in the standard version.

 Running example program once with an empty database Operation Articles (Create (Article {title = "My first article", author = "Cirdec", contents = "Lorem ipsum dolor sit amet."})) ArticleId 0 Operation Articles (Read (ArticleId 0)) Just (Article {title = "My first article", author = "Cirdec", contents = "Lorem ipsum dolor sit amet."}) Operation Articles (Read (ArticleId 0)) Just (Article {title = "My first article", author = "Cirdec", contents = "Lorem ipsum dolor sit amet."}) 

The second run registers the responses received by the program, passes Operation through the cache, and logs the requests before they reach the database. Due to the new caching transparent to the program, requests for reading an article never occur, but the program still gets a response:

 Running example program once with an empty cache and an empty database Operation Articles (Create (Article {title = "My first article", author = "Cirdec", contents = "Lorem ipsum dolor sit amet."})) ArticleId 0 Just (Article {title = "My first article", author = "Cirdec", contents = "Lorem ipsum dolor sit amet."}) Just (Article {title = "My first article", author = "Cirdec", contents = "Lorem ipsum dolor sit amet."}) 

Here is the whole source code. You should think of it as four independent pieces of code: a program written for our domain, starting with example . An application, which is a complete assembly of the program, the discourse area and various tools that create it, starting with main . The following two sections, ending with the TableName schema, describe the blog post domain; their sole purpose is to illustrate how other components are combined, rather than serve as an example of designing data structures in Haskell. The following section describes a small interface through which components can communicate with data; this is not necessarily a good interface. Finally, the remainder of the source code implements the logs, database, and caches that are put together to form the application. To separate the tools and the interface from the domain, there are several disgusting tricks with the type and dynamics here, this does not mean that you can demonstrate a good way to handle castings and generics.

 {-# LANGUAGE StandaloneDeriving, GADTs, DeriveDataTypeable, FlexibleInstances, FlexibleContexts, GeneralizedNewtypeDeriving, MultiParamTypeClasses, ScopedTypeVariables, KindSignatures, FunctionalDependencies, UndecidableInstances #-} module Main ( main ) where import Data.Typeable import qualified Data.Map as Map import Control.Monad.State import Control.Monad.State.Class import Control.Monad.Trans import Data.Dynamic -- Example example :: (MonadOperation TableName m) => m () example = do id <- perform $ Operation Articles $ Create $ Article { title = "My first article", author = "Cirdec", contents = "Lorem ipsum dolor sit amet." } perform $ Operation Articles $ Read id perform $ Operation Articles $ Read id cid <- perform $ Operation Comments $ Create $ Comment { article = id, user = "Cirdec", comment = "Commenting on my own article!" } perform $ Operation Equality $ Create False perform $ Operation Equality $ Create True perform $ Operation Inequality $ Create True perform $ Operation Inequality $ Create False perform $ Operation Articles $ List perform $ Operation Comments $ List perform $ Operation Equality $ List perform $ Operation Inequality $ List return () -- Run the example twice, changing the cache transparently to the code main :: IO () main = do putStrLn "Running example program once with an empty database" runDatabaseT (runOpperationLoggerT (runResultLoggerT example)) Types { types = Map.empty } putStrLn "\nRunning example program once with an empty cache and an empty database" runDatabaseT (runOpperationLoggerT (runCacheT (runResultLoggerT example) Types { types = Map.empty })) Types { types = Map.empty } return () -- Domain objects data Article = Article { title :: String, author :: String, contents :: String } deriving instance Eq Article deriving instance Ord Article deriving instance Show Article deriving instance Typeable Article newtype ArticleId = ArticleId Int deriving instance Eq ArticleId deriving instance Ord ArticleId deriving instance Show ArticleId deriving instance Typeable ArticleId deriving instance Enum ArticleId data Comment = Comment { article :: ArticleId, user :: String, comment :: String } deriving instance Eq Comment deriving instance Ord Comment deriving instance Show Comment deriving instance Typeable Comment newtype CommentId = CommentId Int deriving instance Eq CommentId deriving instance Ord CommentId deriving instance Show CommentId deriving instance Typeable CommentId deriving instance Enum CommentId -- Database Schema data TableName kv where Articles :: TableName ArticleId Article Comments :: TableName CommentId Comment Equality :: TableName Bool Bool Inequality :: TableName Bool Bool deriving instance Eq (TableName kv) deriving instance Ord (TableName kv) deriving instance Show (TableName kv) deriving instance Typeable2 TableName -- Data interface (Persistance library types) data CRUD kvr where Create :: v -> CRUD kvk Read :: k -> CRUD kv (Maybe v) List :: CRUD kv [(k,v)] Update :: k -> v -> CRUD kv (Maybe ()) Delete :: k -> CRUD kv (Maybe ()) deriving instance (Eq k, Eq v) => Eq (CRUD kvr) deriving instance (Ord k, Ord v) => Ord (CRUD kvr) deriving instance (Show k, Show v) => Show (CRUD kvr) data Operation stkvr where Operation :: t ~ skv => t -> CRUD kvr -> Operation stkvr deriving instance (Eq (skv), Eq k, Eq v) => Eq (Operation stkvr) deriving instance (Ord (skv), Ord k, Ord v) => Ord (Operation stkvr) deriving instance (Show (skv), Show k, Show v) => Show (Operation stkvr) class (Monad m) => MonadOperation sm | m -> s where perform :: (Typeable2 s, Typeable k, Typeable v, t ~ skv, Show t, Ord v, Ord k, Enum k, Show k, Show v, Show r) => Operation stkvr -> mr -- Database implementation data Tables tkv = Tables { tables :: Map.Map String (Map.Map kv) } deriving instance Typeable3 Tables emptyTablesFor :: Operation stkvr -> Tables tkv emptyTablesFor _ = Tables {tables = Map.empty} data Types = Types { types :: Map.Map TypeRep Dynamic } -- Database emulator mapOperation :: (Enum k, Ord k, MonadState (Map.Map kv) m) => (CRUD kvr) -> mr mapOperation (Create value) = do current <- get let id = case Map.null current of True -> toEnum 0 _ -> succ maxId where (maxId, _) = Map.findMax current put (Map.insert id value current) return id mapOperation (Read key) = do current <- get return (Map.lookup key current) mapOperation List = do current <- get return (Map.toList current) mapOperation (Update key value) = do current <- get case (Map.member key current) of True -> do put (Map.update (\_ -> Just value) key current) return (Just ()) _ -> return Nothing mapOperation (Delete key) = do current <- get case (Map.member key current) of True -> do put (Map.delete key current) return (Just ()) _ -> return Nothing tableOperation :: (Enum k, Ord k, Ord v, t ~ skv, Show t, MonadState (Tables tkv) m) => Operation stkvr -> mr tableOperation (Operation tableName op) = do current <- get let currentTables = tables current let tableKey = show tableName let table = Map.findWithDefault (Map.empty) tableKey currentTables let (result,newState) = runState (mapOperation op) table put Tables { tables = Map.insert tableKey newState currentTables } return result typeOperation :: (Enum k, Ord k, Ord v, t ~ skv, Show t, Typeable2 s, Typeable k, Typeable v, MonadState Types m) => Operation stkvr -> mr typeOperation op = do current <- get let currentTypes = types current let empty = emptyTablesFor op let typeKey = typeOf (empty) let typeMap = fromDyn (Map.findWithDefault (toDyn empty) typeKey currentTypes) empty let (result, newState) = runState (tableOperation op) typeMap put Types { types = Map.insert typeKey (toDyn newState) currentTypes } return result -- Database monad transformer (clone of StateT) newtype DatabaseT (s :: * -> * -> *) ma = DatabaseT { databaseStateT :: StateT Types ma } runDatabaseT :: DatabaseT sma -> Types -> m (a, Types) runDatabaseT = runStateT . databaseStateT instance (Monad m) => Monad (DatabaseT sm) where return = DatabaseT . return (DatabaseT m) >>= k = DatabaseT (m >>= \x -> databaseStateT (kx)) instance MonadTrans (DatabaseT s) where lift = DatabaseT . lift instance (MonadIO m) => MonadIO (DatabaseT sm) where liftIO = DatabaseT . liftIO instance (Monad m) => MonadOperation s (DatabaseT sm) where perform = DatabaseT . typeOperation -- State monad transformer can preserve operations instance (MonadOperation sm) => MonadOperation s (StateT state m) where perform = lift . perform -- Cache implementation (very similar to emulated database) cacheMapOperation :: (Enum k, Ord k, Ord v, t ~ skv, Show t, Show k, Show v, Typeable2 s, Typeable k, Typeable v, MonadState (Map.Map kv) m, MonadOperation sm) => Operation stkvr -> mr cacheMapOperation op@ (Operation _ (Create value)) = do key <- perform op modify (Map.insert key value) return key cacheMapOperation op@ (Operation _ (Read key)) = do current <- get case (Map.lookup key current) of Just value -> return (Just value) _ -> do value <- perform op modify (Map.update (\_ -> value) key) return value cacheMapOperation op@ (Operation _ (List)) = do values <- perform op modify (Map.union (Map.fromList values)) current <- get return (Map.toList current) cacheMapOperation op@ (Operation _ (Update key value)) = do successful <- perform op modify (Map.update (\_ -> (successful >>= (\_ -> Just value))) key) return successful cacheMapOperation op@ (Operation _ (Delete key)) = do result <- perform op modify (Map.delete key) return result cacheTableOperation :: (Enum k, Ord k, Ord v, t ~ skv, Show t, Show k, Show v, Typeable2 s, Typeable k, Typeable v, MonadState (Tables tkv) m, MonadOperation sm) => Operation stkvr -> mr cacheTableOperation op@ (Operation tableName _) = do current <- get let currentTables = tables current let tableKey = show tableName let table = Map.findWithDefault (Map.empty) tableKey currentTables (result,newState) <- runStateT (cacheMapOperation op) table put Tables { tables = Map.insert tableKey newState currentTables } return result cacheTypeOperation :: (Enum k, Ord k, Ord v, t ~ skv, Show t, Show k, Show v, Typeable2 s, Typeable k, Typeable v, MonadState Types m, MonadOperation sm) => Operation stkvr -> mr cacheTypeOperation op = do current <- get let currentTypes = types current let empty = emptyTablesFor op let typeKey = typeOf (empty) let typeMap = fromDyn (Map.findWithDefault (toDyn empty) typeKey currentTypes) empty (result, newState) <- runStateT (cacheTableOperation op) typeMap put Types { types = Map.insert typeKey (toDyn newState) currentTypes } return result -- Cache monad transformer newtype CacheT (s :: * -> * -> *) ma = CacheT { cacheStateT :: StateT Types ma } runCacheT :: CacheT sma -> Types -> m (a, Types) runCacheT = runStateT . cacheStateT instance (Monad m) => Monad (CacheT sm) where return = CacheT . return (CacheT m) >>= k = CacheT (m >>= \x -> cacheStateT (kx)) instance MonadTrans (CacheT s) where lift = CacheT . lift instance (MonadIO m) => MonadIO (CacheT sm) where liftIO = CacheT . liftIO instance (Monad m, MonadOperation sm) => MonadOperation s (CacheT sm) where perform = CacheT . cacheTypeOperation -- Logger monad transform newtype OpperationLoggerT ma = OpperationLoggerT { runOpperationLoggerT :: ma } instance (Monad m) => Monad (OpperationLoggerT m) where return = OpperationLoggerT . return (OpperationLoggerT m) >>= k = OpperationLoggerT (m >>= \x -> runOpperationLoggerT (kx)) instance MonadTrans (OpperationLoggerT) where lift = OpperationLoggerT instance (MonadIO m) => MonadIO (OpperationLoggerT m) where liftIO = OpperationLoggerT . liftIO instance (MonadOperation sm, MonadIO m) => MonadOperation s (OpperationLoggerT m) where perform op = do liftIO $ putStrLn $ show op lift (perform op) -- Result logger newtype ResultLoggerT ma = ResultLoggerT { runResultLoggerT :: ma } instance (Monad m) => Monad (ResultLoggerT m) where return = ResultLoggerT . return (ResultLoggerT m) >>= k = ResultLoggerT (m >>= \x -> runResultLoggerT (kx)) instance MonadTrans (ResultLoggerT) where lift = ResultLoggerT instance (MonadIO m) => MonadIO (ResultLoggerT m) where liftIO = ResultLoggerT . liftIO instance (MonadOperation sm, MonadIO m) => MonadOperation s (ResultLoggerT m) where perform op = do result <- lift (perform op) liftIO $ putStrLn $ "\t" ++ (show result) return result 

To create this example, you will need the mtl and containers libraries.

Answer 2

In Haskell, you do have to (and want to!) Keep abreast of everything IO does.

This is one of the strengths.

You can use a class like MonadIO to write functions that work in any monad that is allowed to perform I / O:

 myFunctionUsingIO :: (MonadIO m) => ... -> m someReturntype myFunctionUsingIO = do -- some code liftIO $ ... -- some IO code -- some other code 

Because many Haskell programming interfaces are expressed in monads, such functions can work in more contexts.

You can also use unsafePerformIO to secretly run I / O operations from clean code, but this is practically not recommended in almost all cases. Being clean allows you to immediately see if side effects are being used or not.

IO caching is a side effect, and you feel good if your types reflect this.