Conversational search in less than 500 lines of Python

We brought conversational AI to Meilisearch

Democratizing the magic of ChatGPT with open models

I was not getting good cache utilization when including dynamic context in agent threads. After a lot of experimentation, I found a good pattern that minimizes how often long lived conversation history gets modified while still supporting dynamic context. It has flexible hooks for doing things like truncating or summarizing tool outputs when transitioning messages to the long term history. And I'm seeing >>90% of tokens hitting the cache for my agents despite including a lot of dynamic user context. There are a wide range of agent prompting strategies so I'd love to hear where this library…

I kept running into this annoying problem: I’d remember a really useful answer, but not where it was. ChatGPT? Claude? Gemini? No idea. So I’d end up digging through all of them or just rewriting the prompt. Built this to fix that. It’s a Chrome extension that indexes chats locally and lets you search across them all in one place. Once it’s indexed, search is basically instant. Still early. UIs change and break things sometimes, so it’s a bit fragile in places. Curious if other people have the same issue or if it’s just me jumping between tools too much.