Elasticsearch

Elasticsearch is an important service that we use frequently in the organization. We use Elasticsearch as a logging service for our system logs, Once we have the logs in Elasticsearch, we connect to Kibana and start building dashboards and charts that help us track our system stability and availability in terms of System metrics. On the other hand, we use it to track new bugs and errors. The other usage for Elasticsearch in our system is as a search engine. Elasticsearch is a very fast and amazing search engine, where we store some fields and call Elasticsearch APIs to fetch these fields when needed.

• We're able to detect system incidents at early stage
• We can achieve 99.98% availability for searching service
• Data retrieval speed is less than 300 ms on avg

Sumo Logic, Splunk Log Observer and Prometheus

Splunk On-Call, ManageEngine Site24x7, MongoDB, Redis™*, Snyk, Cloudflare, Domo, Slack, Fullstory, NetSuite ERP, LaunchDarkly, Ghost Inspector

We use Elasticsearch to analyze and visualize logs from various Engineering workflows. We have clusters defined for providing Application Performance Monitoring for a variety of Engineering applications, utilizing Beats and other processes to populate the data required for monitoring and analysis. We also capture metrics (for both servers and applications).

• Elasticsearch provides a convenient way to analyze data from various data sources.
• Visualizations from data analysis provide easy guidance to management.
• Elasticsearch reduces overall IT/Administrative costs

Apache Druid, Splunk Enterprise and Grafana

Apache Druid, Apache Kafka, Apache Spark, Grafana

We use ECE platform and Elasticsearch for the delivery data to track delivery. And also use kibana for visualization of business analysis and KPI. We also ingest the log from different API and investigate when there is a trouble. We also use transform and machine learning feature to detect anomalies.

• The license is quite expensive
• The consultation and operation cost is also a high cost
• The performance during the peak period is not stable enough but there isn't good temporary solution

OpenSearch and Splunk Enterprise

AWS CloudTrail, Apache Kafka, AWS Lambda

• Data persistence, indexing and querying at high speed
• Scalability
• Building reporting over data thanks to Kibana

• Greatly reduced data-in-transit and at-rest overheads
• Provided us with a truly scalable solution for our data
• Kibana offers a reporting platform based on our custom queries. Extremely useful for reports from automated test executions.

Apache Solr and MongoDB

Microsoft Teams, Docker, Sonatype Nexus Platform

• Text Search (Natural Language Processing)
• Integration with Couchbase
• Integration with multiple platforms including dot net and nodejs
• Aggregations and search done together

• Most of our investment is in programming hours which is expensive
• Easy to set up nodes
• Free version has a lot of the great basic features

Apache Solr, Splunk Enterprise and Couchbase

Microsoft SQL Server, MySQL, Couchbase, MongoDB, Docker, ASP.NET, Visual Studio IDE, Microsoft Visual Studio Code, Windows Server, Global Relay Archive