Articles about Elasticsearch

Share Button

Lire la version française

We have worked recently on search engines in different domains :

  • 2 multi-criterias front office search engines moteurs de recherche multi-critères front-office (for frontend users)
  • 2 multi-criterias back office search engines (for administrators)
  • on statistical engine (computing, agregations, sums, means, …)

Build these engines on a database would have been too costly (in development time but also for performance reasons) for approximate results (for both statistic and full-text search).

For these reasons, we have decided to use Elasticsearch : a search engine based on Lucene. Here is the result of our adventures : 11 articles (including this one):


Two articles to understand Elasticsearch whatever the language
Five for Symfony developers
Three for PHP developers in general,
One article to find the path,
One article to rull them all,
One article to find them,
One article to bring them all
And in the darkness, bind them.

These articles will explain the needs we had but most importantly, the solutions we brought. All the posts are not currently written, but don’t hesitate to contact us for the subjects we haven’t covered.
In order to make these articles more concrete, we are developing a sandbox project in which you can see every following subject in action. It’s a simple example, a blog, that allows us to test every feature we want to show you. Of course, since the articles are not all published yet, the project will evolve at the same time.

We hope these information will help you to better understand this wonderful tool called Elasticsearch.

1 – Install Elasticsearch with Symfony in 3 minutes.

This article presents the installation and the first Elasticsearch configuration, but also the installation of the FOSElasticaBundle and the library Elastica for PHP. That’s all, the goal is only to get a functional instance of Elasticsearch on you computer.
Read the article

2 – Indexing and simple search with Elasticsearch and Symfony

The basis ! Configure a mapping between a Symfony/Doctrine object and an Elasticsearch document.
Using the example of a blog, you can index your first objects (here some articles). We talk about the FOSElasticaBundle configuration, the creation of a search object and the associated form.
At the end of this article, you can perform a simple search :

  • Search by title (terms approximation)
  • Search by date
  • Search by status (published or not)

Read the article

3 – Paginate and sort your results with Elasticsearch and Symfony

Your search is working, but each good search engine comes with a pagination and a sorting (by date, by relevance, …).
This is the aim of this article : add a pagination and a sorting system to your search object, and use it when building the Elasticsearch query.
Read the article

4 – Efficient Elasticsearch indexing configuration

When using text-based search engines, two steps are important :

  • Indexing : The way the text is indexed will influence the relevance and the usefulness of the engine : cutting words, deleting irrelevant words, normalizing (unicode characters), synonyms indexing, …
  • Searching : In the same way, cleaning the “query string” is very important. Deleting the irrelevant characters increases the relevance of the results.

This article talks mainly about the text analyzers configuration when indexing and searching. We explain when and how to use the analyzers.

Read the article

5 – Elasticsearch and Symfony : hydrating objects

FOSElasticaBundle includes a default Transformer that converts the results sent by Elasticsearch into Doctrine objects.
The problem is that the joined objects are not fetched automatically (the Doctrine Lazy Loading).
This article talks about this (little) problem and explain how to improve substantially the transformation and to reduce the number of queries hydrating directly the joins.
Read the article

6 – Export to xls/csv with the Elasticsearch scroll function

How to export some search results to a csv or xls file, without overloading the server memory. The answer is in this article : use the Elasticsearch scroll function.
Read the article

7 – Advanced search (Object, Nested, hasParent, hasChild) with Elasticsearch

In non-relational databases, the lack of joins can be a difficulty. Fortunately, Elasticsearch proposes some solutions to meet these needs.
This article presents the different ways to link the objects, their advantages and disadvantages.
Read the article

8 – Elasticsearch aggregations, the futur of facets

Elasticsearch can index huge set of data, both documents and numbers.
In the versions older than v1.0.0, facets allowed to get statistical information about a list of documents (tag distribution, mean, standard deviation, …)
Over time, the use of facets has evolved. Developers has used them to realize more and more advanced statistics.
The great Elasticsearch team as reacted implementing the aggregations :

- Metrics : sum, minimum, maximum, mean, ….
- “Buckets” : terms, date or value distribution, …
These aggregations can contain “sub-aggregations”. So you can compute statistics in a terms distribution! How crazy is that!
The aim of this article is to describe one of the uses you can do with aggregations : statistics.
Read the article

9 – Create a ranking with Elasticsearch aggregations (coming soon…)

In this second article about aggregations, we’ll make a ranking. A bit different use.

10 – Advanced indexing : how to use ModelToElasticaTransformer

The simple mapping Doctrine <-> Elasticsearch is implemented automatically by the FOSElasticaBundle.
When you’ll use Elasticsearch in a more advanced way, you’ll maybe need to custom entirely the way you index (pre-computing datas, objects, nested, specific sql queries, …)
To do that, FOSElasticaBundle allows the customization of the Objet(orm/odm) -> Document(Elasticsearch) Transformer.
This article explains quickly how to use this feature.
Read the article

We hope to make you discover all the amazing Elasticsearch features throughout these articles!

Remind: the sample-projects sources used for these articles is available on Github. The project will evolve as we write more articles.

Share Button

6 thoughts on “Articles about Elasticsearch

  1. Wow, this is great. Perfect timing for me as I’ve just set up elastic search and am grappling with fine tuning search results

  2. Hi,
    Thanks ;-) !

    Elasticsearch is really a great tool, don’t hesitate to contact us for questions or comments. It will help us to improve our articles.

    François

  3. Hello!
    Point 7 is ready in French, we are translating it in english.
    The 8 will follow, but the subject is larger. Meanwhile, you can contact us for any question about aggregations ;-) We will try to help.

  4. Hi!

    First I want to thank you guys for your great articles. When will you publish the article about advanced indexing?

    Patrick

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Protected by WP Anti Spam