I am using the Tweepy package in Python to collect tweets. I track several users and collect their latest tweets. For some users, I get the error "Failed to parse the JSON payload:", for example. "Failed to parse the JSON payload:" Pending "," delimiter "or"} ": row 1 of column 694303 (char 694302)." I took note of the user id and tried to reproduce the error and debug the code. The second time I ran the code for this particular user, I got the results (i.e. Tweets) without any problems. I adjusted my code so that when I get this error, I will try to extract the tweets again. Thus, I can get this error once or twice for the user, but in the second or third attempt, the code returns tweets,as usual, without error. I get similar behavior for other users as well.
My question is: why does this error appear randomly? Nothing else has changed. I searched the Internet but could not find a similar report. A snapshot of my code follows
alltweets = []
ntries = 0
while True:
try:
if beforeid:
new_tweets = api.user_timeline(user_id = user,count=200, since_id=sinceid, max_id=beforeid)
else:
new_tweets = api.user_timeline(user_id = user,count=200, since_id=sinceid)
break
except tweepy.error.RateLimitError:
print "Rate limit error:", sys.exc_info()[0]
print("Timeout, retry in 5 minutes...\n")
time.sleep(60 * 5)
continue
except tweepy.error.TweepError as er:
print('TweepError: ' + er.message)
if er.message == 'Not authorized.':
new_tweets = []
break
else:
print(str(ntries))
ntries +=1
pass
except:
print "Unexpected error:", sys.exc_info()[0]
new_tweets = []
break
source
share