Dedicated Search Engines

In our last post, we saw how capable modern databases like PostgreSQL are at providing robust Full-Text Search. For many applications, this is the perfect solution. But what happens when your search needs grow? What if search isn't just a feature, but the core of your product? This is when you graduate to a dedicated search engine.

These are specialized, standalone systems designed from the ground up to do one thing exceptionally well: find relevant information at incredible speed. They take the principles of the inverted index and ranking algorithms we've discussed and scale them across distributed clusters, wrapped in rich, developer-friendly APIs.

Let's explore two popular examples: Elasticsearch, the powerful and established incumbent, and Meilisearch, the fast and developer-friendly challenger.

Elasticsearch

Elasticsearch is a distributed, open-source search and analytics engine built on the Apache Lucene library. It is the de facto standard for large-scale search applications, log analytics, and more.

Core Idea: It's designed for horizontal scalability. You can start with a single node and grow to a massive cluster of hundreds of nodes, handling petabytes of data and billions of documents.

Key Features:

Rich Query DSL
A flexible JSON-based language for constructing complex queries, aggregations, and filters.

Tunable Relevance
Deep control over scoring and ranking algorithms.

Aggregations
A powerful feature that allows you to perform complex analytics on your data, enabling features like faceted search.

Indexing a Document

// PUT /articles/_doc/1
{
  "title": "The Sleeping Dog",
  "body": "The lazy dog slept in the sun",
  "author": "jane_doe",
  "published_date": "2024-10-22"
}

Searching for a Document

// GET /articles/_search
{
  "query": {
    "match": {
      "body": "lazy dog"
    }
  }
}

Meilisearch

Meilisearch is a newer, open-source search engine that prioritizes speed, ease of use, and an exceptional out-of-the-box developer experience.

Core Idea: Make powerful search simple. It's designed to be set up in minutes and provide a great user experience with minimal configuration.

Key Features:

Typo Tolerance
Automatically handles typos in user queries without any setup.

Ultra-Fast
Written in Rust, it's optimized for near-instantaneous search results.

Simple API
A straightforward and intuitive RESTful API.

Indexing a Document

curl \
  -X POST 'http://localhost:7700/indexes/articles/documents' \
  -H 'Content-Type: application/json' \
  --data-binary '[
    { "id": 1, "title": "The Sleeping Dog", "body": "The lazy dog slept in the sun" },
    { "id": 2, "title": "The Fox", "body": "the quick brown fox jumped over the lazy dog" }
  ]'

Searching for a Document

curl \
  -X POST 'http://localhost:7700/indexes/articles/search' \
  -H 'Content-Type: application/json' \
  --data-binary '{ "q": "lazy dog" }'

When Should You Choose a Dedicated Engine

Moving from an integrated database FTS to a dedicated engine like Elasticsearch or Meilisearch is a significant architectural decision. It introduces more operational overhead but, in return, provides a level of performance, scalability, and feature richness that can transform your application and create a world-class user experience.