A huge amount of text data for parsing

Question

I am developing a ruby ​​parser that parses some uneven text data. Can someone tell me where I can get a lot of open text data?

Answer 1

Here you will get a list of many:

http://www.quora.com/Data/Where-can-I-get-large-datasets-open-to-the-public

And my fav:

http://ftp.sunet.se/mirror/archive/ftp.sunet.se/pub/tv+movies/imdb/

Answer 2

You can copy Wikipedia (or just run a bunch of it through lynx -dump ). It will also give you an extensive source of non-English text. The Gutenberg project will be another good source of a lot of plain text.