Promptspot
Open-source 'unit testing' stack for your prompts and data
What it does
Promptspot is an open-source (Apache License 2.0) project that helps automate testing of large language model (LLM) prompts against an array of input data, automating the process of "teaching the bot to fish".
Does the same job
all alternatives →- IBI built an open-source “unit testing” suite for prompts ⮂ inputs2023 · github.com · ▲5
Hi HN, I'm pleased to share Promptspot, an open-source (Apache License 2.0) project that helps automate testing of large language model (LLM) prompts against an array of input data. Modern LLMs offer an enormous amount of leverage if you "teach the bot to fish" — i.e. simply prompt it with both a "system prompt" (which typically doesn't change often) and a dynamic input, which is often application state, search results, recent activity, user profile data, etc. Existing playgrounds and prompt management systems often lack the rigor and flexibility required for this dynamic approach — and as…
- POPromptTools – open-source tools for evaluating LLMs and vector DBs2023 · github.com · ▲211
Hey HN! We’re Kevin and Steve. We’re building PromptTools (https://github.com/hegelai/prompttools): open-source, self-hostable tools for experimenting with, testing, and evaluating LLMs, vector databases, and prompts. Evaluating prompts, LLMs, and vector databases is a painful, time-consuming but necessary part of the product engineering process. Our tools allow engineers to do this in a lot less time. By “evaluating” we mean checking the quality of a model's response for a given use case, which is a combination of testing and benchmarking. As examples: - For generated…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 27d ago · cactuscompute.com
Launched alongside, June 2023
the whole month →- NI
Heya HN, I've been working on a reddit-like platform as my primary side project for the last few years. Doing a (very) soft launch today, mainly because I want to use it to encourage discussion of alternatives. How non.io works: 1. Free to browse, paid to interact. 2. Minimum subscription is $2 (though you can choose more). I take $1 to run the servers, everything left gets split evenly between everything you upvote that month. It's a simple model, but I hope it's a better one than the freemium model we've been relying on for the last few years. Fundamentally I feel like any ad-supported…
Commerce · 2023 · non.io
Typefully 2.0▲931Effortlessly publish on Twitter and LinkedIn, now with AI
Growth · 2023 · typefully.com
SlashPage▲760Create websites in seconds to build your first 1K community
Growth · 2023 · slashpage.com
