When I add utf-8 encoding, it uses about 3000 MB of memory (and freezes my computer) exmaple below works fine,
pool = ThreadPoolExecutor(max_workers=10)
with open("data.txt") as f:
for line in f:
pool.submit(thread, line)
pool.shutdown(wait=True)
but when I add utf-8 encoding, it basically causes my PC to crash
pool = ThreadPoolExecutor(max_workers=10)
with open("data.txt", encoding='utf-8') as f:
for line in f:
pool.submit(thread, line)
pool.shutdown(wait=True)
source
share