Roundtable – Estimating survey results in seconds
Recent academic work ([1], [2]) has suggested that LLMs can effectively simulate different Internet subpopulations. For example, you may ask ChatGPT to emulate being a high school teacher explaining Newton’s laws of physics. Building upon this, we created Roundtable, a platform that uses LLMs to predict how people will respond to any arbitrary survey question. To do so, we needed to first reduce bias arising from GPT’s training procedure. Because these models are primarily trained on Internet data, they can be heavily skewed towards the demographics of heavy Internet users (e.g.,…
What it does
In the maker’s words, at launch
Recent academic work ([1], [2]) has suggested that LLMs can effectively simulate different Internet subpopulations. For example, you may ask ChatGPT to emulate being a high school teacher explaining Newton’s laws of physics. Building upon this, we created Roundtable, a platform that uses LLMs to predict how people will respond to any arbitrary survey question. To do so, we needed to first reduce bias arising from GPT’s training procedure. Because these models are primarily trained on Internet data, they can be heavily skewed towards the demographics of heavy Internet users (e.g., high-income, male). We addressed this by fine-tuning GPT on the GSS (General Social Survey) to ‘de-bias’ the model into emulating a more representative U.S. population. We allow users to ask any multiple-choice question and add conditioning questions and/or descriptions of their target population. Here are some examples: Simulation 1 (General Interest) Are you interested in buying an e-bike? Yes 28%, No 72% ([3]) Are you interested in buying an e-bike? conditioned on "Yes" to "Do you own a Tesla car?" Yes 40%, No 60% ([4]) Simulation 2 (reproducing the Stack Overflow Developer Survey; [5]) Where did you learn to code? conditioned on "Yes" to "Are you 45 years or older?" Books 55%, Online 45% ([6]) Where did you learn to code? conditioned on "No" to "Are you 45 years or older?" Books 26%, Online 74% ([7]) Simulation 3 (USA vs. Stack Overflow Developers vs. Hacker News Users) Do you code? Yes 24%, No 76% ([8]; USA) Do you code? Yes >99%, No 0% ([9]; Stack Overflow Developers) Do you code? Yes 83%, No 17% ([10]; Hacker News Users) — Of course, a natural question is whether we can trust these results. If you click ‘Investigate Results’, we report the most similar (in terms of cosine distance between LLM embeddings) GSS questions as a way of estimating how much extrapolation / interpolation is going on. This doesn’t quite address the accuracy of the subpopulations / conditioning questions (we are working on this), but we thought we are at a sufficiently advanced point to share what we’ve built with you all. Feedback would be greatly appreciated. ---- [1] https://arxiv.org/pdf/2209.06899.pdf [2] https://openreview.net/pdf?id=eYlLlvzngu [3] https://roundtable.ai/sandbox/e02e92a9ad20fdd517182788f4ae7e... [4] https://roundtable.ai/sandbox/6b4bf8740ad1945b08c0bf584c84c1... [5] https://survey.stackoverflow.co/2023/ [6] https://roundtable.ai/sandbox/d701556248385d05ce5d26ce7fc776... [7] https://roundtable.ai/sandbox/8bd80babad042cf60d500ca28c40f7... [8] https://roundtable.ai/sandbox/4a9d2fd6025459bd73b7798a8b2fdc... [9] https://roundtable.ai/sandbox/7e41ed16c01de48247bce02700c398... [10] https://roundtable.ai/sandbox/13aaa142e87337201601fb4b76d125...
Does the same job
all alternatives →
- BIBuzzngo.io – Real-time public polls using RethinkDB2018 · buzzngo.io · ▲60
- SCSurveySparrow – Conversational and Recurring Survey Software2017 · surveysparrow.com · ▲23
- RYRoundtable (YC S23) – Survey Quality Control API2024 · app.roundtable.ai · ▲8
- RARoundtable – A rubber duck that argues with itselfDec 2025 · roundtable.ovlo.ai · ▲6
Hey HN, I'm a founder at Ovlo a supply chain company.I had a problem. After every batch of customer interviews/research/feedback sessions, I'd run ideas through an LLM to help me decide what we should build next. Except it was obvious to my cofounder I wasn't really validating anything. LLMs are incredibly good at agreeing with you in subtle ways, especially when you feed them context that already reflects your thoughts. I'd ask "Does this make sense?" and get a beautifully worded essay about why yes, obviously, this is the best thing ever. I was using AI as an echo chamber without…
- SNSolving NYT Connections with ChatGPT2023 · github.com · ▲8
Just for fun I decided to see if I could use chatGPT to solve NYT Connections word puzzles. It uses a pretty straightforward BFS search in which the LLM is first prompted to generate several possible groupings of four related words, and then a different prompt is used to evaluate the soundness of each of those groupings. This approach seems to be able to produce the correct solution somewhat less than half the time. Some observations: * For whatever reason, chatGPT-4 seems to be a bit worse than 3.5 at generating Connections groupings. I haven’t tested systematically so maybe this is just…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 26d ago · cactuscompute.com
Launched alongside, July 2023
the whole month →
- WL
Hey everyone, I here is a small open-source project I've been working on lately. I'd love to hear your thoughts and improvement ideas :) GitHub: [github.com/Vincenius/workout-lol](https://github.com/Vincenius/workout-lol)
Dev tools · 2023 · workout.lol
- HN
I saw this [0] pretty cool thread by user revskill, and wanted a quicker way to search through it, but also to keep them all in one place so I can read them at my leisure whenever I get time. Right now is like 60 lines of Ruby using Nokogiri, but I will certainly look into it further down the line and improve the list. There's a cronjob checking the thread every 12 hours but I will eventually shut that down and it will become static after that. There are some really awesome blogs in there. I really recommend going through the list, it made my day. [0] "Could you share your personal blog…
Life & fun · 2023 · dm.hn
A safe space for your thoughts, private, local, p2p & open
Commerce · 2023 · anytype.io
