Sequin – A Kafka Alternative to REST APIs
Hey HN! We're Eric, Carter, and Anthony from Sequin (https://sequin.io). Sequin streams data from platforms like Stripe, AWS, and Salesforce into messaging systems like Kafka. Sequin loads historic data into topics and then streams new changes in real-time. You can provision a stream of API data with a single POST request. This means you can add third-party data to your apps and internal tools super fast. We support streaming to Kafka and NATS. If you don’t have a messaging system, you can use our serverless hosted one. To get your data at rest, you can stream to databases like…
What it does
In the maker’s words, at launch
Hey HN! We're Eric, Carter, and Anthony from Sequin (https://sequin.io). Sequin streams data from platforms like Stripe, AWS, and Salesforce into messaging systems like Kafka. Sequin loads historic data into topics and then streams new changes in real-time. You can provision a stream of API data with a single POST request. This means you can add third-party data to your apps and internal tools super fast. We support streaming to Kafka and NATS. If you don’t have a messaging system, you can use our serverless hosted one. To get your data at rest, you can stream to databases like Postgres. I (Anthony) joined IFTTT in the early 2010s. I worked on a lot of integrations there. At the time, it was clear the landscape was a mess. Yet, more than a decade later, nothing’s changed. A common misconception is that integrating is about making API requests. Or running background jobs that paginate endpoints. But that's just API extraction. After extraction is when the real integration starts. The integration is all the custom business logic that determines how your system should use that data or respond to it changing. [1] Big companies have to integrate hundreds of systems. They use message buses. That’s because message buses are the best way to integrate. [2] But what happens if the system is a 3rd-party API? How do you get data into your message bus? You need to extract. Unfortunately, most APIs stop at REST. But REST sucks for extraction (have you tried detecting deletes?) ETL tools like Fivetran and Airbyte help with extraction. But they're built for data engineers, not for developers. This is why ETL tools are not real-time, have poor support for messaging systems, are not programmable, and don’t support writes. Integration platforms like Zapier and Tray.io can help you wire up external systems. But this means writing business logic in their WYSWIGs or DSLs. Developers want to maintain business logic in their code and tools (e.g. git). The vision is to get developers to stop writing undifferentiated and tedious extraction code. We want every developer to be able to integrate with message buses – no matter the project. When a messaging system is your backbone, integrations throughout your system and across many external APIs all feel the same. Most of the differences between APIs are in frivolous details. The differences between ServiceNow and NetSuite's query parameters should not matter to you. When the API is just data, these differences disappear. While APIs haven’t evolved much since the early 2000s, message systems have. Companies like Upstash, WarpStream, and Synadia offer serverless message systems. They’re cheap to get started with and easy to scale. Along with partnering with these providers, we offer our own serverless hosted message bus [3]. This uses familiar message queue semantics. Sequin uses a number of strategies for each API to extract data in real-time. You don't need to stub out APIs in your tests or setup ngrok to receive webhooks. Both test and dev just need to read messages off a queue or records in a database. And we offer a uniform interface for writes. Writes are committed to both the API and your message stream simultaneously, so you can “read your writes.” Sequin's early, so it’s not perfect. We don't have any onboarding in our product. We want to add a lot more sources. We need better graphs and insights for observability and monitoring. But we pride ourselves on speed and consistency. We know that missing even a single event erodes trust in our event streams, so we work hard to eliminate that possibility. We'd love to hear your thoughts and questions! [1] https://martinfowler.com/articles/cant-buy-integration.html [2] Using message buses to integrate goes all the way back to 2003's classic "Enterprise Integration Patterns" (Hohpe and Woolf)! [3] https://sequin.io/docs/stream-apis/consume/overview - Uses NATS JetStream under the hood
Does the same job
all alternatives →
- SUSequin – Use Postgres/SQL to read from and write to APIs2023 · ▲7
Hi HN community, I'm Anthony, co-founder of Sequin (sequin.io), and am excited to show you what we've been working on. Sequin lets you skip HTTP APIs and integrate with Postgres and SQL instead. While we originally launched with a one-way sync from APIs to Postgres, as of today, you can now _write back_ to APIs using `insert`, `update`, and `delete` on your synced tables. We built Sequin because no one is happy with HTTP APIs today. For developers that need to build on top of third-party APIs, each integration is bespoke. There are few standards. I've personally seen over a dozen different…
- GDGoka – distributed stream processing library for Kafka written in Go2017 · github.com · ▲105
More dev tools this month
the category →
Open-source GTM skills for technical founders
Dev tools · 29d ago · gtmcofounder.com
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 1d ago · opentrailpaper.com
Launched alongside, April 2024
the whole month →
Supabase▲2,328The Postgres developer platform is now generally available
Dev tools · 2024 · supabase.com
Cal.com Platform▲1,602Build your pixel-perfect booking experience with Atoms
Dev tools · 2024 · cal.com
deco.cx 2.0▲1,080Build web apps 10x faster with Deno, JSX, TS & Tailwind
Dev tools · 2024 · decocms.com
