Connection Timeout with Elasticsearch
PythonPython 2.7ElasticsearchPython Problem Overview
from datetime import datetime
from elasticsearch import Elasticsearch
es = Elasticsearch()
doc = {
'author': 'kimchy',
'text': 'Elasticsearch: cool. bonsai cool.',
'timestamp': datetime(2010, 10, 10, 10, 10, 10)
}
res = es.index(index="test-index", doc_type='tweet', id=1, body=doc)
print(res['created'])
This simples code is returning the following error:
elasticsearch.exceptions.ConnectionTimeout: ConnectionTimeout caused by - ReadTimeoutError(HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=10))
Very strange, because the server is ready and set (http://localhost:9200/ is returning some json).
Python Solutions
Solution 1 - Python
By default, the timeout value is set to 10 secs. If one wants to change the global timeout value, this can be achieved by setting the flag timeout=your-time while creating the object.
If you have already created the object without specifying the timeout value, then you can set the timeout value for particular request by using request_timeout=your-time flag in the query.
es.search(index="my_index",
doc_type="document",
body=get_req_body(),
request_timeout=30)
Solution 2 - Python
Try setting timeout in Elasticsearch initialization:
es = Elasticsearch([{'host': HOST_ADDRESS, 'port': THE_PORT}], timeout=30)
You can even set retry_on_timeout to True and give the max_retries an optional number:
es = Elasticsearch([{'host': HOST_ADDRESS, 'port': THE_PORT}], timeout=30, max_retries=10, retry_on_timeout=True)
Solution 3 - Python
The connection timed out problem could occur if you are using Amazon Elastic Search service.
es = Elasticsearch([{'host': 'xxxxxx.us-east-1.es.amazonaws.com', 'port': 443, 'use_ssl': True}])
The above python code where you override the default port from 9200 to 443 and setting the SSL to true will resolve the issue.
If no port is specified, it is trying to connect to the port 9200 in the specified host and fails after time out
Solution 4 - Python
This is nothing to do with increasing your timeout to 30 seconds. Do people actually think that elastic search should need up to 30 seconds to return one tiny hit?
The way I fixed this problem was go to config/elasticsearch.yml uncomment the following
http.port: 9200
network.host: 'localhost'
Network.host might be set to 192.168.0.1 which might work But I just changed it to 'localhost'
Solution 5 - Python
Note that one of the common reasons for timeouts when doing es.search (or es.index) is large query size. For example, in my case of a pretty large ES index size (> 3M documents), doing a search for a query with 30 words took around 2 seconds, while doing a search for a query with 400 words took over 18 seconds. So for a sufficiently large query even timeout=30 won't save you. An easy solution is to crop the query to the size that can be answered below the timeout.
Increasing timeout or doing retries on timeout will help you if the cause was in traffic, otherwise this might be your culprit.
Solution 6 - Python
elasticsearch.exceptions.ConnectionTimeout: ConnectionTimeout caused by - ReadTimeoutError(HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=10))
mean the request didn't end in the specified time (by default, timeout=10).
This will work with 30 seconds :
res = es.index(index="test-index", doc_type='tweet', id=1, body=doc, timeout=30)
Solution 7 - Python
The reasons for the timeout could be many and it seems worth checking the logs on elasticsearch side (logs/elasticsearch.log) to see the detailed error. In our case, the error on ES was:
primary shard is not active Timeout: [1m]
As described in this post, this was because our disk was full. We had resized it (and the partition) a day ago to take care of that but ES needs to be restarted if the high/low watermark has been hit once (we are on 5.5.x) which we had not done.
Simply restarting the ES on production resolved the issue for us.
Solution 8 - Python
my personal problem was solved with (timeout = 10000) which was practically never reached because the entries on server were only 7.000 but it had heavy traffic and its resources were being hogged and that was why the connection was dropping
Solution 9 - Python
Two options that help:
1: increase the timeout
Setting a timeout solved this problem for me. Note that newer versions need a unit, e.g. timeout="60s":
es.index(index=index_name, doc_type="domains", id=domain.id, body=body, timeout="60s")
Without a unit, for example by setting timeout=60, you'll get
elasticsearch.exceptions.RequestError: RequestError(400, 'illegal_argument_exception', 'failed to parse setting [timeout] with value [60] as a time value: unit is missing or unrecognized')
2: reduce text length
It also helps to reduce the text length, e.g. by cutting of long texts, so elastic can store the text faster which will avoid timeouts, too:
es.index(index=index_name, doc_type="domains", id=domain.id, body=text[:5000], timeout="60s")
Solution 10 - Python
I got this timeout problem when trying out elasticsearch 7.11.1 (installed using apt install) at port 9202 (9200 is in use by an elasticsearch 6) in ubuntu.
I googled around for nearly two hours before finally fixing it. It turns out I nneed to set (comment out) one line in /etc/elasticsearch/elasticserch.yml
cluster.initial_master_nodes: ["node-1"] # earlier in the same file: node.name: node-1
I hope this may help someone.
Solution 11 - Python
extend the requestTimeout
client = new elasticsearch.Client({
host : 'http://localhost:9200',
requestTimeout: 60000
});
Solution 12 - Python
I got this error when specifying the fields option in my query for some reason. Removing the option made the query work (in OpenSearch at least):
Before:
elasticsearch.helpers.scan(es, index="my_index", query={"query": {"match_all": {}}, "fields": ["zip_code", "polygon"]}, size=5000, scroll="15m")
After:
elasticsearch.helpers.scan(es, index="my_index", query={"query": {"match_all": {}}}, size=5000, scroll="15m")
Solution 13 - Python
I had a similar problem.
> client.ping()
True
> client.info()
ObjectApiResponse({'name': .... })
Searching also worked, but indexing failed with an exception: ConnectionTimeout: Connection timed out.
The reason was that my disk was almost full. I freed up some space and then I could index without errors...