Search on S3 Using AWS Lambda
Hi everyone, During the last 3 months, I worked on the Quickwit search engine to adapt it for AWS Lambda runtime. I wrote a blog post to announce the beta release [0] and two tutorials to quickstart: - Indexing and searching 20 million logs entries dataset [1]. - Implementing an E2E use case where an application generates data, uploads it in batches to S3. Users can then search through an HTTP API authenticated with an API Key [2]. On the performance side, I observed on a 20 million log dataset that search is sub-second, and analytics queries take from 1 to 4 second. On the cost side,…
Search on S3 Using AWS Lambda launched on January 21, 2024 with 6 votes, #2,120 of 4,437 launches that month and more than 50% of that year's launches. The site was not reachable when checked in September 2026.
What it does
In the maker’s words, at launch
Hi everyone, During the last 3 months, I worked on the Quickwit search engine to adapt it for AWS Lambda runtime. I wrote a blog post to announce the beta release [0] and two tutorials to quickstart: - Indexing and searching 20 million logs entries dataset [1]. - Implementing an E2E use case where an application…
Does a similar job
all alternatives →
Quickwit – OSS Alternative to Elasticsearch, Splunk, Datadog2024 · github.com · ▲145Hi folks, Quickwit cofounder here. We started Quickwit 3 years ago with a POC, "Searching the web for under $1000/month" (see HN discussions [0]), with the goal of making a robust OSS alternative to Elasticsearch / Splunk / Datadog. We have reached a significant milestone with our latest release (0.7) [1], as we have witnessed users of the nightly version of Quickwit deploy clusters with hundreds of nodes, ingest hundreds of terabytes of data daily, and enjoy considerable cost savings. To give you a concrete example, one company is ingesting hundreds of terabytes of logs daily…
Highly cost-efficient search engine in Rust2021 · github.com · ▲61Cloud-native OSS search engine for observability. Contribute to quickwit-oss/quickwit development by creating an account on GitHub.
s3-lambda – Lambda functions over S3 objects: each, map, reduce, filter2017 · github.com · ▲177Lambda functions over S3 objects with concurrency control (each, map, reduce, filter) - littlstar/s3-lambda
- IWI wrote a book about using Lambda with Go2021 · ▲103
Hi HN! During the last few years, I worked on a few applications built with Go, running on AWS Lambda. As I got to know the platform better, I started to find Go & Lambda to be a really productive combination. The applications were fast, and they ended up being much cheaper to run than what my team & I had built before. It’s probably not the best platform for _every_ application, but I was surprised at how much of our workload worked well on it. As we brought new engineers on to our team and helped them get up to speed with the stack, I found that we were covering a lot of the same topics…
Quickwit – OSS Alternative to Datadog, Elasticsearch2024 · github.com · ▲8Hi HN, Quickwit cofounder here! Quickwit is a cloud-native search engine for observability, and we have just released a significant upgrade with performance improvements, extended Elasticsearch API compatibility, and better integration with OpenTelemetry tools, Grafana, and Jaeger. This release is also a significant milestone as we have been witnessing users of the nightly version of Quickwit deploy clusters with hundreds of nodes, ingest hundreds of terabytes of data daily, and enjoy additional cost savings. We are very excited to put it in the hands of all of our users. One report…
- IBI built a service to help companies save on their AWS bills2020 · ▲73
Hey HN: I'm Kaveh, the founder of Usage (https://www.usage.ai/) We help companies drive down AWS costs. Why? Because the way it's done now is a pain. Stakeholders, especially engineers, are required to spend unnecessary time manually finding underutilized or overly expensive EC2s. We believe the optimization process should be done automatically through a series of sophisticated algorithms. At the moment, there are over 70,000 AWS EC2 prices - doing that manually just won't scale at most organizations. My background is in software engineering. Previous to founding Usage, I…
More dev tools this month
the category →
open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.
Dev tools · 28d ago · github.com
Contribute to reladraw/reladraw development by creating an account on GitHub.
Dev tools · 26d ago · github.com
OpenBot▲362Grok Bot alternative: free, local, open-source, multiplayer
Dev tools · 23d ago · openbot.run
Talanoa▲346Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…
Dev tools · 21d ago · talanoa.email
CoreSpeed▲332One MCP for everything your agents need: apps, memory, tools
Dev tools · 29d ago · corespeed.io
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, January 2024
the whole month →
Amie▲1,334Turn meeting notes into automated workflows. Handle meetings, summaries, todos and emails (soon) with your AI personal assistant.
Work · 2024 · amie.so
- IM
Sit in Shade - Discover the optimal seat to stay in the shade during your journey. Sit In Shade provides real-time sun exposure data and route visualization.
Life & fun · 2024 · sitinshade.com
