Best IT Recipes

What are some use cases for elasticsearch compared to standard sql queries?

I’m just starting out with Elasticsearch, and one of the main use cases I’ve seen is scalability when searching on large datasets, but other than that, when do you want to use it only to create SQL queries with traditional RDMS?

+42
elasticsearch use-case
Oct 22 '15 at 14:38
source share
2 answers

There are two main uses for Elasticsearch:

  • Text search

You want Elasticsearch when you do a lot of text search, where traditional RDBMS databases do not work very well (poor configuration, acts like a black box, low performance). Elasticsearch is very customizable, extensible through plugins. You can quickly create a reliable search without much knowledge.

  1. Recording and analysis

Another question is that many people use Elasticsearch to store logs from different sources (to centralize them), so they can analyze them and make sense out of it. In this case, Kibana becomes comfortable. It allows you to connect to the Elasticsearch cluster and immediately create a visualization. For example, Loggly is created using Elasticsearch and Kibana.

Keep in mind that you do not want to use Elasticsearch as your primary data store. The reasons are here: How reliable is ElasticSearch as the primary data warehouse against factors such as loss of record, data availability

Update

It seemed to me that the second part is no longer annoying, in fact the fact that Elastic, as a company, is doing very well last year. Thanks to the current DevOps movement, CI / CD pipelines, an increase in the number of indicators from different sources, ELK has become the choice of factual for monitoring infrastructure, it is no longer just a distributed text search engine RESTful. It has an amazing set of products:

  • Logstash (tons of data entry)
  • Beats
    • Filebeat
    • Metricbeat
    • Packetbeat
    • Winlogbeat
  • Kibana
    • Graph
    • Timelion
  • X-Pack (premium)
    • Alerts
    • Reports
    • Security
    • Machine learning
    • Cross data center metrics

A community-built ecosystem grows around the ELK stack, which extends current features, few of which are worth mentioning:

  • Elastlerler
  • Shield protector
+41
Oct 22 '15 at 16:08
source share

To add a different answer, logging is still a major use case, as well as a search, but now metrics and analytics are becoming more important.

I believe this post summarizes the changes in the market that bring new uses to big data. All you really need to know about open source databases

With the advent of Web 2.0, static web pages have become dynamic and the social media around us. Everyone tweet, publish, blogging, journal, photo sharing, chatting and commenting. Internet of Things (IoT) is emerging - a rapidly growing network that collects and exchanges data, such as sensors and smart devices. There are some great examples here.

All in all, it generates a huge amount of new data that businesses want to absorb and use to stay ahead to provide features such as product recommendations and best customer experience. Data can be analyzed by looking for patterns for applications such as analytics fraud detection and behavior. Most of the new data is unstructured, which means that it cannot be neatly stored in a table database.

Imagine trying to design a database to store data for your grocery stores — what you like, how often you buy it, whether you prefer milk or cream with coffee. New types of databases are needed to store new data, and they need to be non-relational and ideally cheap. The ring of bells? Not both in NoSQL and low cost, as in open source.

One of the Elasticsearch architects with whom I spoke said that 80% of Elasticsearch data works in companies, is unstructured, and 20% is structured. This is the unstructured data that companies are looking for to discover rare or unusual data patterns. They also use Elasticsearch to monitor data patterns. For example, a major retailer conducts real-time tracking using Elasticsearch to ensure adequate cash reserves in stores for people to cash checks on payday.

In my own experience with our search case, we not only use fuzzy search queries, but also turned into automatic and quick searches. From what I saw, as soon as you start working with Elasticsearch, you begin to develop into other use cases that complement what you already have. Now that we have created Elasticsearch as a fuzzy search engine in our company, now we have other teams that analyze the metrics for logging.

Here are some additional resources that are discussed in more detail in this topic:

+12
Apr 20 '17 at 20:18
source share


More articles:


All Articles