Elasticearch Python Reindex Timedout Client

Question

I am trying to reindex using the python Elasticsearch client using https://elasticsearch-py.readthedocs.org/en/master/helpers.html#elasticsearch.helpers.reindex . But I keep getting the following exception:elasticsearch.exceptions.ConnectionTimeout: ConnectionTimeout caused by - ReadTimeout

Stop error code

Traceback (most recent call last):
  File "~/es_test.py", line 33, in <module>
    main()
  File "~/es_test.py", line 30, in main
    target_index='users-2')
  File "~/ENV/lib/python2.7/site-packages/elasticsearch/helpers/__init__.py", line 306, in reindex
    chunk_size=chunk_size, **kwargs)
  File "~/ENV/lib/python2.7/site-packages/elasticsearch/helpers/__init__.py", line 182, in bulk
    for ok, item in streaming_bulk(client, actions, **kwargs):
  File "~/ENV/lib/python2.7/site-packages/elasticsearch/helpers/__init__.py", line 124, in streaming_bulk
    raise e
elasticsearch.exceptions.ConnectionTimeout: ConnectionTimeout caused by - ReadTimeout(HTTPSConnectionPool(host='myhost', port=9243): Read timed out. (read timeout=10))

Is there a way to prevent this exception other than increasing the timeout?

EDIT: python code

from elasticsearch import Elasticsearch, RequestsHttpConnection, helpers

es = Elasticsearch(connection_class=RequestsHttpConnection,
                   host='myhost',
                   port=9243,
                   http_auth=HTTPBasicAuth(username, password),
                   use_ssl=True,
                   verify_certs=True,
                   timeout=600)
helpers.reindex(es, source_index=old_index, target_index=new_index)

Answer 1

This may be due to the OutOfMemoryError heap space for Java, which means that you are not giving elasticsearch enough memory for what you want to do. Try looking at /var/log/elasticsearchif there are any exceptions.

https://github.com/elastic/elasticsearch/issues/2636

Answer 2

, request_timeout 30 ( 30 ) . , API- stream_bulk reindex elasticsearch.py ​​

chunk_size 500 ( 500 ) . 50, . .

def streaming_bulk (, , chunk_size = 50, raise_on_error = True,       expand_action_callback = expand_action, raise_on_exception = True,       ** kwargs):

def reindex (client, source_index, target_index, query = None, target_client = None,        chunk_size = 50, scroll = '5m', scan_kwargs = {}, bulk_kwargs = {}):