Elasticsearch Essentials - A book if I had completed 🤥

Aug, 2017

Mastering Elasticsearch fundamentals for reliable and high performance search

Current Status

Startup life (Pixibo) has put a hold on this now outdated book 🤐

Audience

This book is intended for users who are looking into using Elasticsearch as their primary search technology and would like to build a reliable and scalable Elasticsearch infrastructure along with appropriate index mappings and search techniques and queries. This books assumes the reader has some basic knowledge of using REST api's and lucene.

Mission

To provide a solid foundational understanding of various core elasticsearch concepts to help in avoiding irreversible changes during early planning stages of development and provide guidance for building a scalable, fault tolerant and distributed elasticsearch environment.

Objectives and achievements

The book aims to provide a good foundational understanding of core elasticsearch concepts such as mapping, filters, queries and search and the move on to more advanced search techniques such as scripting and indexing strategies to help build an efficient search platform. Besides providing understanding of core concepts the book also provides guidance for building a highly reliable, fault tolerant and scalable Elasticsearch cluster for production environments along with cautionary tips and gotchas that one must be aware off to avoid any unexpected behaviors.

General structure

Introduction To Elasticsearch Indexing, Retrieving and Search
Elasticsearch Document Tour – Guide to CRUD operations and Concurrency controls
Understanding Index and Property mappings – Analyzers, Types and Index Store
Mastering search with efficient Query and Filter combinations
Scripting when Queries aren’t enough
Distributed and Fault Tolerant Elasticsearch setup
Elasticsearch Gotchas and Cautionary tales
Example: Finding images based of their dominant color

Chapter 1: Introduction to Elasticsearch

Description

This chapter provides an introduction to elasticsearch core concepts such as creating indexes, mappings and performing simple searches. This level of detail will help guide the users to more advanced and core capabilities of Elasticsearch in further chapters

Topics covered

Introduction to Elasticsearch
Installing Elasticsearch
Indexing Documents
Retrieving Documents Get vs Post
Full-text vs Partial text search
Facets

Skills learned

Understand different components of elasticsearch
Understand how documents are stored and retrieved
Getting Elasticsearch up and running for quick development

Chapter 2: Elasticsearch Document Tour

Description

This chapters provides a gentle introduction to all core concepts, retrieval and storage of manipulating data a.k.a documents in elasticsearch

Topics covered

Indexing, Retrieving and Deleting documents
Updating documents - Partial and Full updates
Multi Get requests
Versioning documents for Concurrency control
Bulk inserting documents
Object relationships

Skills learned

On how to store and retrieve documents efficiently
On how to deal with conflicts in a distributed system environment
Modeling your data for efficient storage and retrieval

Chapter 3: Conquering Mappings

Description

This chapter providers the core knowledge of on how to effectively setup index and property mappings using built in and the most commonly used analysers and tokenizers along with dynamic templating for fields and it’s impact on search query performance and indexing as these are some of the foundational layers in building a performant search. As part of this chapter, more settings such using eager loading, global ordinals as well as setting doc values for saving memory and using the disk effectively.

Topics covered

Core Simple Field Types
Nested and inner object Mappings
Efficient map settings for memory and space consumption
Analysis and Analyzers
Updating and templating mappings
Reindexing

Skills learned

How to setup the right index mapping and property settings
How to use more advanced mapping feature to save memory usage and faster search
Templates for dynamic indexes
Understanding how to affectively use analysers for specific search purposes
How to perform a no downtime reindexing when mappings go wrong

Chapter 4: Performing Complex search in Elasticsearch

Description

This chapter providers an in-depth knowledge of how to affectively use a combination of queries and filters to perform valid and efficient search based on scoring, filtering and text.

Topics covered

All common Queries – Term, match, multi-match, Query String and many more
Filters – And, Bool, Exists and many more
Aggregations and facets
Introduction to Scoring and Boosting

Skills learned

Solid understanding on how to use queries along with Filters to produce better search results
Introduction to Scoring to manipulate the search behavior
Exposure to various API’s such as Validate, Explain, More Like This, Count etc
Aggregation/Facets Introduction and usage

Chapter 5: Scripting when Queries aren’t Enough

Description

This chapter provides an introduction to elasticsearch Scripting languages and its capabilies along with various scripting techniques for filtering, custom scoring and sorting options.

Topics covered

Introduction to Elasticsearch Scripting
Sorting using Scripting
Filtering using Scripting
Custom scoring thru Scripting

Skills learned

Understand the different use cases of using a scripting language
Understand how custom scoring affects different search results

Chapter 6: Distributed and Fault Tolerant Elasticsearch

Description

This chapter provides how Elasticsearch can be setup and configured in a clustered environment to be highly available, distributed and fault tolerant. Having a fault tolerant and recoverable system is the foundation of any distributed system which this chapter tends to address for any production deployment.

Topics covered

Introduction to Elasticsearch clustering
Zen, Multicast and Cloud based configuration
Fault detection and recovery setup
Logging setup and its implications
AWS Cloud formation Script for deploying to the cloud
Autoscaling Elasticsearch servers based on disk usage

Skills learned

Understand and implement different clustering configuration
Deploying Elasticsearch cluster both on private network and on cloud
Elasticsearch settings and it’s purposes

Chapter 7: Elasticsearch Gotchas, Cautionary tales and Backup-Restore

Description

This chapter provides an overview and cautionary alerts that a Elasticsearch user must be aware during the initial setup of Elasticsearch as well as settings and other data that one might take for granted could have serious side effects over the long run. As part of this chapter, one would learn how to secure Elasticsearch from malicious external requests.

Topics covered

Memory details and what they mean
Document count and disk space
Replication, Snapshots and Restore
Securing Elasticsearch by restricting http request

Skills learned

Setting up elasticsearch with knowing what the settings and its consequences
Snapshot, backup and restoring knowhow
Authentication and Authorization of http calls to Elasticsearch

Chapter 8: Finding images based of their dominant color

Description

This chapter provides an full-fledged example including mapping, index settings, queries and filters on how to perform a color based search of indexed images with indexed RGB data

Topics covered

Indexing and Mapping
Query, Filter, Scripting and Custom scoring

Skills learned

How to use some Elasticsearch filters and scoring to perform a real world implementation
Understanding Function Scoring feature alongside with document scoring