Ably AI Transport - a transport layer for agentic apps
Hey HN, Staff Engineer at Ably here. Over the past few months I've been speaking to engineers building AI assistants, copilots, and agentic workflows (over 40 companies at this point), with particular focus on cloud-hosted agents. I expected the hard problems to be in model selection, prompt engineering, and orchestration. Instead, the same infrastructure challenges kept coming up: realtime sync between agents and end clients is surprisingly painful to get right. - Managing and scaling WebSocket or SSE connections between agents and clients - Buffering messages server-side and implementing…
What it does
In the maker’s words, at launch
Hey HN, Staff Engineer at Ably here. Over the past few months I've been speaking to engineers building AI assistants, copilots, and agentic workflows (over 40 companies at this point), with particular focus on cloud-hosted agents. I expected the hard problems to be in model selection, prompt engineering, and orchestration. Instead, the same infrastructure challenges kept coming up: realtime sync between agents and end clients is surprisingly painful to get right. - Managing and scaling WebSocket or SSE connections between agents and clients - Buffering messages server-side and implementing replay logic for reconnecting clients - Tracking what each client has received across multiple devices - Ensuring continuity between historical and live responses on initial load - Routing reconnecting clients to the correct agent instance in distributed deployments Ably AI Transport solves this: it's a drop-in transport layer that sits between your agents and end-user devices. We discovered many companies who were already using Ably Pub/Sub to tackle these problems. The pub/sub pattern decouples agents from clients: agents publish to channels, clients subscribe, Ably handles delivery and replay. AI Transport makes this easier - for example, we've added message appends for efficient token streaming, and annotations for attaching metadata like citations. A few of the interesting technical pieces: - Channel-oriented architecture: In connection-oriented setups, the connection pokes the agent into life. If the connection drops, on reconnection the agent must figure out what state each client is missing. AI Transport uses channels instead: agents publish, clients subscribe, the channel handles replay. Presence events let agents detect when users go online/offline or connect from multiple devices. - Identity in decoupled pub/sub: Users authenticate with your server, which issues JWTs with embedded clientId and capabilities. Agents receive messages with cryptographically verified identity. User claims (ably.channel.* in the JWT) appear in message.extras.userClaim - useful for HITL workflows where you verify an approver's role before executing a tool call. - Token streaming with appends: New message.append operation lets you build a response incrementally. Clients joining mid-stream get a message.update with the complete response so far, then receive subsequent appends. Channel history contains one compacted message per response. - Annotations for citations: Attach citation metadata to responses without modifying content. Clients can subscribe to individual annotations or obtain them on demand via REST. Ably also automatically aggregates annotations into summaries (e.g. count by domain name), which are delivered to clients in realtime. - Messaging patterns: Docs include patterns for tool calls, human-in-the-loop approval flows, and chain-of-thought streaming. Docs: https://ably.com/docs/ai-transport If you're building AI UX, I'd love to hear what problems you've hit and what you've built in-house. Thanks! Mike Christensen
Does the same job
all alternatives →- UAUpsonic: An AI agent framework with client-server architecture2025 · github.com · ▲9
Three months ago, we started developing an open source agent framework. We previously tried existing frameworks in our enterprise product but faced challenges in certain areas. Problems we experienced: * We risked our stateless architecture when we wanted to add an agented feature to our existing system. Current frameworks lack server-client architecture, requiring significant effort to maintain statelessness when adding an agent framework to your application. * Scaling problem - needed to write Docker configurations as existing frameworks lack official Docker support. Each agent in my…
- AAAgentic AI Frameworks on AWS (LangGraph,Strands,CrewAI,Arize,Mem0)2025 · github.com · ▲7
We’ve published a set of open-source reference implementations on how to build production-grade Agentic AI applications on AWS. What’s in the repo: • Agentic RAG, memory, and planning workflows with LangGraph & CrewAI • Strands-based flows with observability using OTEL & Arize • Evaluation with LLM-as-judge and cost/performance regressions • Built with Bedrock, S3, Step Functions, and more GitHub: https://github.com/aws-samples/sample-agentic-frameworks-on-... Would love your thoughts — feedback, issues, and stars welcome!
- HGHybrid Groups – Agentic AI for human team collaboration [video]2025 · youtube.com · ▲7
Most AI applications are built for individuals but work happens in groups and humans want to collaborate with both agentic AI and other teammates in the same session. We created Hybrid Groups for that purpose. In Hybrid Groups, agents join group chats as virtual team members in Slack and GitHub. They participate in group conversations, proactively contribute when needed and perform actions on behalf of individual users, like managing your calendar for meeting suggestions or updating your todo list without sharing access to your private resources to the group. The project is open-source at…
- RARunning AI agents across environments needs a proper solutionMar 2026 · github.com · ▲8
Hi HN folks, I have been building AI agents for quite some time now. The shift has gone from LLM + Tools → LLM Workflows → Agent + Tools + Memory, and now we are finally seeing true agency emerge: agents as systems composed of tools, command-line access, fine-grained system capabilities, and memory. This way of building agents is powerful, and I believe it is here to stay. But the real question is: are the systems powering these agents ready for that future? I do not think so. Using Docker for a single agent is not going to scale well, because agents need to be lightweight and fast. LLMs…
- JAJoinly.ai – Build real-time interactive meeting agents using MCP2025 · github.com · ▲5
Hey HN, I'm one of the creators of joinly.ai, an open-source solution that makes browser-based video conferences accessible to AI agents, allowing you to interact with them in real time. Think of it as a connector layer that brings the functionality of your AI agents to your meetings, allowing you to essentially build your own custom meeting assistant. We didn't want to build yet another Python framework for creating agents with its own syntactic sugar that you have to learn. That's why we opted for a different approach, utilizing an MCP server. Our MCP server provides essential meeting…
- AAAgentServe – A open-source framework for hosting scalable AI agents2024 · github.com · ▲9
Hey HN! I am super excited (and slightly nervous) to introduce AgentServe! AgentServe is a framework to make hosting scalable AI agents as easy as possible. With 4 lines of code AS wraps your agent (any framework) in a FastAPI and connects it to a Task Queue (celery or redis). Why Should You Care? Standardized Communication Pattern: AgentServe proposes that all agents should communicate with each other and the outside world with “Tasks” that can be submitted in a sync or async way. This simple API wil enable Framework Agnostic: No favorites. OpenAI, LangChain, LlamaIndex, CrewAI are all…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 26d ago · cactuscompute.com
Launched alongside, January 2026
the whole month →- IN
Hey HN! I wanted to share something I built over the last few weeks: isometric.nyc is a massive isometric pixel art map of NYC, built with nano banana and coding agents. I didn't write a single line of code. Of course no-code doesn't mean no-engineering. This project took a lot more manual labor than I'd hoped! I wrote a deep dive on the workflow and some thoughts about the future of AI coding and creativity: http://cannoneyed.com/projects/isometric-nyc
AI · Jan 2026 · cannoneyed.com
Automatic AI-powered code reviews the moment you open a PR
Dev tools · Jan 2026 · kilo.ai
