Python program is very slow

Question

This program generates letter combinations and checks to see if they are words, but the program is extremely slow, generating only a few words per second. please tell me why it is very slow and what I need to do faster.

import itertools for p1 in itertools.combinations('abcdefghijklmnopqrstuvwxyz', 4): with open('/Users/kyle/Documents/english words.txt') as word_file: english_words = set(word.strip().lower() for word in word_file) def is_english_word(word): return word.lower() in english_words print ''.join(p1),"is", is_english_word(''.join(p1)) 

Answer 1

This is slow because you are re-reading the file for each iteration of the loop and creating a new function object. None of these two things depend on a loop variable; move them out of the loop to run only once.

In addition, a simple function can be integrated; function call is relatively expensive. And don't call ''.join() twice. And you only use lowercase letters to generate words, so .lower() is redundant:

 with open('/Users/kyle/Documents/english words.txt') as word_file: english_words = set(word.strip().lower() for word in word_file) for p1 in itertools.combinations('abcdefghijklmnopqrstuvwxyz', 4): word = ''.join(p1) print '{} is {}'.format(word, word in english_words) 

Since you generate words of length 4, you can save yourself some memory by simply loading words of length 4 from your English word file:

 with open('/Users/kyle/Documents/english words.txt') as word_file: english_words = set(word.strip().lower() for word in word_file if len(word.strip()) == 4)