New 🔥

Hybrid Search is now available via the new Torus.hybrid/4 macro!

Hybrid search fuses several search strategies into a single ranked query using Reciprocal Rank Fusion - the industry-standard way to combine keyword and semantic search. Rows that rank high in several branches win; rows found by only one branch still compete.

search_vector = Torus.to_vector("magic school")

Post
|> Torus.hybrid([p], [
     full_text: {[p.title, p.body], "magic school"},
     semantic: {p.embedding, search_vector, weight: 2.0}
   ], limit: 10)
|> Repo.all()

Key features:

• Fuses any combination of full_text, bm25, similarity, and semantic branches (the same type can appear more than once)
• Per-branch :weight and :limit, tunable RRF :k
• Everything runs in a single SQL query - the result is a regular Ecto query you can keep piping select, preload, or pagination onto
• The fused score is available via the :torus_hybrid named binding or the :score_key option

See the Hybrid search guide for details.

Match highlighting is now available via the new Torus.highlight/3 macro!

Built on PostgreSQL's ts_headline, it wraps matches of the term in the selected text (in <b> tags by default) and can also return snippets instead of the full text. A type: :substring mode highlights matches inside words, pairing with ilike/like searches.

The simplest way to use it is the new :highlight option on full_text, bm25, similarity, ilike, like, and hybrid branches - just point it at the columns, the search's own term and options are reused:

Post
|> Torus.full_text([p], [p.title, p.body], "shocker", highlight: [title: p.title])
|> Repo.all()
# => [%Post{title: "Hogwarts <b>Shocker</b>", ...}]

Or standalone in any select/select_merge via Torus.highlight/3:

Post
|> Torus.full_text([p], [p.title, p.body], "shocker")
|> select([p], Torus.highlight(p.title, "shocker"))
|> Repo.all()
# => ["Hogwarts <b>Shocker</b>"]

Improvements

• Test suite now runs semantic (pgvector) integration tests on CI.

Breaking changes ⚠️

• Torus now requires ecto/ecto_sql ~> 3.10 (hybrid search relies on selected_as/2 and ordering by select aliases). Upgrade Ecto before bumping Torus.
• Torus.Embeddings.NebulexCache now targets Nebulex 3.0+. Nebulex 3.0 moved Nebulex.Adapters.Local (the default adapter) to the separate nebulex_local package - if you use the cache, upgrade nebulex to >= 3.0.0 and add nebulex_local to your dependencies.

Fixes

• Torus.QueryInspector.tap_sql/3 no longer crashes - it now prints the SQL and its parameters and returns the query.
• Torus.similarity/5 and Torus.full_text/5 with order: :desc (the default) now order with NULLS LAST, so rows whose search column is NULL no longer rank above actual matches. Hybrid branches rank the same way.
• Torus.Embeddings.NebulexCache.embedding_model/1 no longer raises - it now passes the options through to the underlying embedding module.
• Empty search terms contribute no rows in similarity and bm25 hybrid branches (matching full_text branches), instead of boosting arbitrary rows.

v0.6.0

New 🔥

BM25 Full-Text Search is now available via the new Torus.bm25/5 macro!

BM25 is a modern ranking algorithm that generally provides superior relevance scoring compared to traditional TF-IDF (used by full_text/5). This integration uses the pg_textsearch extension by Timescale.

See it in action on the demo page

Key features:

• State-of-the-art BM25 ranking with configurable index parameters (k1, b)
• Blazingly fast top-k queries via Block-Max WAND optimization (Torus.bm25/5 + limit)
• Simple syntax: Post |> Torus.bm25([p], p.body, "search term") |> limit(10)
• Score selection with :score_key and post-filtering with :score_threshold
• Language/stemming configured at index creation via text_config

Requirements:

• PostgreSQL 17+
• pg_textsearch extension installed
• BM25 index on the search column (with text_config for language)

See the BM25 Search Guide for detailed setup instructions and examples.

When to use BM25 vs full_text:

• Use bm25/5 for fast single-column search with modern relevance ranking
• Use full_text/5 for multi-column search with weights or when using stored tsvector columns

v0.5.3

Fixes

• Torus.Embeddings.OpenAI now correctly parses updated response structure from OpenAI API.
• exla nx backend is now correctly supported in Torus.Embeddings.LocalNxServing.

v0.5.2

New 🔥

• New demo page where you can explore different search types and their options. It also includes semantic search, so if you're hesitant - go check it out!
• Other documentation improvements

Fixes

• Correctly handles order: :none in Torus.semantic/5 search.
• Updates Torus.Embeddings.HuggingFace to point to the updated feature extraction endpoint.
• Suppresses warnings for missing ecto_sql dependency by adding it to the required dependencies. Most of us already had it, but now it'll be explicit.
• Correctly parses an array of integers in Torus.QueryInspector.substituted_sql/3 and Torus.QueryInspector.tap_substituted_sql/3. Now we should be able to handle all possible query variations.

v0.5.1

• Adds Torus.Embeddings.Gemini to support Gemini embeddings.
• Extends semantic search docs on how to stack embedders
• Adds :distance_key option to Torus.semantic/5 to allow selecting distance key to the result map. Later on we'll rely on this to support hybrid search.
• Correctly swaps > and < operators for pre-filtering when changing order in Torus.semantic/5 search.

v0.5.0

• Similarity search type now defaults to :word_similarity instead of similarity.
• Possible Torus.similarity/5 search types are updated to be prefixed with similarity to replicate 1-1 these in pg_trgm extension.
• Extended optimization section in the docs

v0.4.1

Minor doc updates

v0.4.0

Breaking changes:

• Torus.full_text/5 - now returns all results when search term contains a stop word or is empty instead of returning none.

Improvements:

• Torus.full_text/5 - now supports :empty_return option that controls if the query should return all results when search term contains a stop word or is empty.
• Torus.QueryInspector.tap_explain_analyze/3 - now correctly returns the query plan.
• Docs were grouped together by the search type.

New 🔥

Semantic search is finally here! Read more about it in the Semantic search with Torus guide. Shortly - it allows you to generate embeddings using a configurable adapters and use them to compare against the ones stored in your database.

Supported adapters (for now):

• Torus.Embeddings.OpenAI - uses OpenAI's API to generate embeddings.

• Torus.Embeddings.HuggingFace - uses HuggingFace's API to generate embeddings.

• Torus.Embeddings.LocalNxServing - generate embeddings on your local machine using a variety of models available on Hugging Face

• Torus.Embeddings.PostgresML - uses PostgreSQL PostgresML extension to generate embeddings

• Torus.Embeddings.Batcher - a long‑running GenServer that collects individual embedding calls, groups them into a single batch, and forwards the batch to the configured embedding_module (any from the above or your custom one).

• Torus.Embeddings.NebulexCache - a wrapper around Nebulex cache, allowing you to cache the embedding calls in memory, so you save the resources/cost of calling the embedding module multiple times for the same input.

And you can easily create your own adapter by implementing the Torus.Embedding behaviour.

v0.3.0

Breaking changes:

• full_text_dynamic/5 is renamed to full_text/5 and now supports stored columns.
• similarity/5 - limit option is removed, use Ecto's limit/2 instead.
• full_text/5 - :concat option is renamed to :coalesce.

Improvements:

• full_text/5 now supports stored tsvector columns.
• Torus.QueryInspector.substituted_sql/3 now correctly handles arrays substitutions.
• Docs are extended to guide through the performance and relevance.

And other minor performance/clearance improvements.

v0.2.2

• full_text_dynamic/5: Replaced :nullable_columns with :concat option
• similarity/5: Fixed a bug where you weren't able to pass variable as a term
• Torus.QueryInspector: now is not tied with Torus.Testing and serves as a separate standalone module.

And other minor performance/clearance improvements.

v0.2.1

similarity/5 search is now fully tested and customizable. full_text_dynamic/5 is up next.

Changelog for Torus v0.2.0

Torus now supports full text search, ilike, and similarity search.