We’ve been working with Elasticsearch since its inception, either with clients on consulting for Elasticsearch products and Elasticsearch production support, or by building our own hosted log management solution. For the last 4 years, we’ve also been sharing our knowledge through Elasticsearch training classes. In 2018, we had remote public training classes on a fixed quarterly schedule, so you can more easily plan your learning time and budget.

During our Activate presentation, we talked about how to do query expansion by dynamically generating synonyms. Instead of statically defining synonyms lists, we showed a demo of how you could use word2vec to derive synonyms from a dataset. Before we start, check out a useful Solr Cheat Sheets to guide you through Solr and help boost your productivity and save time.

Sematext provides a single pane of glass and machine learning powered alerts for logs, metrics, traces and digital user experience data. The new Sematext agent is fully Docker Engine and Kubernetes-aware. (Re)written in Go, it has a minimal memory and CPU footprint. It also collects Kubernetes metrics in the most optimal fashion possible.

Sematext provides a single pane of glass and machine learning powered alerts for logs, metrics, traces and user experience data. Sematext Cloud provides advanced monitoring, logging and tracing for all Docker platforms such as Docker EE, Kubernetes, GKE, AWS ECS, and IBM Cloud. Sematext’s new monitoring agent leverages the powerful eBPF Linux kernel observability functionality and uses the Kubernetes API to enrich the container and cluster level metrics.

Logagent is a modern, open-source, light-weight data shipper with out of the box and extensible log parsing, on-disk buffering, secure transport and bulk indexing to Elasticsearch and Sematext Cloud. Its low memory footprint and low CPU overhead make it suitable for deploying on edge nodes and devices, while its ability to parse and structure logs makes it a great Logstash alternative.

We’re excited to announce the release of the Field Stats API plugin for Elasticsearch. The Field Stats API used to be present from Elasticsearch 1.6 to 5.6, to provide efficient statistics for fields of each index. For example, the minimum and maximum values of a date field.

We recently had a presentation at Activate 2018 about entity extraction in the context of a product search. For example: how to tell, when the user typed in Activate 2018, that the intent was to run conference:Activate AND date:201

You can now create monitoring integrations for your technology stack easier than ever before. We’ve recently opened-sourced Sematext’s Monitoring Agent, reworked it, and made it fully pluggable, making it possible for you to collect metrics from a number of additional sources.

This recipe is similar to the previous rsyslog + Redis + Logstash one, except that we’ll use Kafka as a central buffer and connecting point instead of Redis. You’ll have more of the same advantages.

Starting from Elasticsearch 5.0, you’re able to define pipelines within it that process your data, in the same way you’d normally do it with something like Logstash. We decided to take it for a spin and see how this new functionality (called Ingest) compares with Logstash filters in both performance and functionality. Is it worth sending data directly to Elasticsearch or should we keep Logstash?