We have a fairly large cluster of Greenplum v4.3. 18 hosts, each host has 3 segments of nodes. Each host has about 40 cores and 60 GB of memory.
The table has a width of 30 columns, which contains 0.1 billion rows. The request we are testing has a response time of 3-10 seconds when there is no concurrency pressure. As the number of requests that we run in parallel increases, latency decreases with a period of 3 to 50 seconds, as expected.
But we found that no matter how many requests we run in parallel, we only have a very low QPS (request per second), almost only 3-5 requests / sec. We set max_memory = 60G, memory_limit = 800MB and active_statments = 100, hoping that the CPU and memory can be highly utilized, but they are still poorly used, for example 30% -40%.
I have a strong feeling, we tried to copy the cluster in parallel, hoping to get the most out of CPU and memory usage. But this does not work, as we expected. Is there something wrong with the settings? or is there something else I don’t know about?
source share