I have the following code that runs very slowly (6.5 sec for iterating over 57.390 lines):
import mysql.connector
cnx=mysql.connector.connect(host=***, ***)
cursorSQL = cnx.cursor()
for dt in date_vect:
cursorSQL.execute(query2, ((dt[0] + '-00:00:00'), (dt[0] + '-23:59:59')))
usr_list = list(cursorSQL.fetchall())
As recommended here: https://stackoverflow.com/a/2776262/2326321 , I tried:
usr_list= cursorSQL.fetchall()usr_list= list(cursorSQL.fetchall())
And as @postoronnim suggested, I also tried:
usr_list= cursorSQL.fetchmany(57390)
Without success.
However, there is some caching effect, since the same iteration takes only 0.5 seconds, when the iteration starts a second time, and then slows down to 6.5 seconds.
- Any idea where this might come from?
- Could you confirm that it has nothing to do with my database, since all import from MySQL is performed in a row
cursor.execute, and fetchall()slowness is only due to list processing? - Could you explain why there is a caching effect?
Thank.
: Python 3.5 | 8- i7 | 16go Ram