OpenSearch: Search, Logs & Vector Search Guide

OpenSearch

OpenSearch is an open-source search and analytics engine for full-text search, log and event analytics, observability, security analytics, and vector search. It began in 2021 as a fork of Elasticsearch 7.10 and Kibana, created after Elastic moved those projects away from the Apache 2.0 license. OpenSearch stayed Apache 2.0, and in 2024 the project moved from AWS stewardship to the OpenSearch Software Foundation under the Linux Foundation.

Architecturally, OpenSearch is still recognizably Elasticsearch-shaped: JSON documents stored in indices, split into shards built on Apache Lucene, queried with a JSON query DSL over a REST API. Since the fork, the two have diverged in features, plugins, clients, and managed offerings — see Elasticsearch vs OpenSearch for the comparison.

TL;DR

Documents live in indices; mappings define field types; shards and replicas spread data and load across nodes.
Search with the query DSL (match, bool, term, range, aggregations), plus SQL and PPL query languages.
OpenSearch Dashboards provides visualization, Discover, and plugin UIs.
The bundled security plugin provides TLS, authentication, roles, and document- and field-level security.
Index State Management (ISM) automates rollover, retention, and tiering for time-series indices.
The k-NN plugin and neural search support vector and hybrid search; Amazon OpenSearch Service and Serverless offer managed options.

Quick Example

Create an index with an explicit mapping, add a document, and search it — with the Python client:

text fields are analyzed for full-text matching; keyword fields are exact values for filters, sorting, and aggregations. Filters in bool.filter do not affect scoring and are cached.

Core Concepts

Cluster Building Blocks

Querying

The query DSL combines full-text queries (match, multi_match, match_phrase), term-level queries (term, terms, range, exists), and compound queries (bool, function_score). Aggregations compute facets, histograms, percentiles, and metrics alongside results. OpenSearch also offers SQL and PPL (Piped Processing Language) for analysts who prefer them, especially for log exploration.

Plugins as the Feature Model

Most capabilities ship as plugins included in the default distribution: security, alerting, anomaly detection, Index State Management, SQL/PPL, k-NN, ML Commons, neural search, cross-cluster replication, notifications, and observability. This is a major difference from Elasticsearch, where many equivalent features depend on license tier.

Security Plugin

The security plugin provides TLS between nodes and clients, authentication (internal users, LDAP, SAML, OpenID Connect), role-based access control, and document- and field-level security, plus audit logging. Production clusters should never run with security disabled or demo certificates.

Logs and Time-Series Data

Time-based data is written to a sequence of indices (or a data stream) rather than one ever-growing index. Index State Management moves each index through states:

Ingestion usually flows through Data Prepper, Fluent Bit, Logstash, or OpenTelemetry collectors, with ingest pipelines for parsing and enrichment. See Log Aggregation.

Vector and Hybrid Search

The k-NN plugin adds a knn_vector field type with approximate nearest-neighbor search using engines such as Faiss and Lucene (HNSW or IVF), with quantization options to reduce memory:

Neural search can generate embeddings at ingest and query time through connected models, and hybrid queries combine lexical (BM25) and vector scores with normalization in a search pipeline — a strong default for retrieval-augmented generation. See Hybrid Search and Vector Databases.

Deployment Options

Best Practices

Define Mappings Explicitly

Dynamic mapping guesses types from the first document it sees and can create thousands of fields from unpredictable data. Use explicit mappings, dynamic: strict where possible, and index templates.

Size Shards Deliberately

Aim for shards of roughly 10–50 GB, avoid thousands of tiny shards, and use rollover and ISM to keep time-series shards in that range.

Keep the Source of Truth Elsewhere

OpenSearch is usually a derived index of data owned by a primary database or event stream. Make reindexing from that source routine, through aliases, so mapping changes and recovery are boring.

Secure and Back Up Every Cluster

Enable TLS and authentication, apply least-privilege roles per application, and take regular snapshots to object storage — and test restoring them.

Common Mistakes

Exposing a Cluster Without Security

Text Fields Used for Filters and Aggregations

One Huge Index for All Logs

FAQ

What is OpenSearch?

An Apache 2.0-licensed search and analytics engine and visualization suite, forked from Elasticsearch and Kibana 7.10 in 2021, and now governed by the OpenSearch Software Foundation under the Linux Foundation.

Is OpenSearch compatible with Elasticsearch?

Largely at the level of Elasticsearch 7.10 APIs, but the projects have diverged since. Recent Elasticsearch clients do not work with OpenSearch, and newer features on each side have no equivalent on the other.

Can OpenSearch be used as a vector database?

Yes. The k-NN plugin stores and searches vector fields, and neural and hybrid search combine vectors with full-text relevance. It is a good fit when you also need text search, filtering, and aggregations in one system.

What is OpenSearch Dashboards?

The visualization and user interface layer forked from Kibana, used for exploring data, building dashboards, and accessing plugin features such as alerting, security administration, and anomaly detection.

Should I use Amazon OpenSearch Service or run it myself?

Managed service suits teams already on AWS that want upgrades, snapshots, and node failures handled for them. Self-managed clusters give more control over versions, plugins, and cost at the expense of operational work.