Alternatives
Products that do what Generate Fine-tunning dataset using deep research in terminal does
Just open-sourced a small terminal tool I’ve been working on. The idea came from wondering how useful it’d be if you could just describe the kind of dataset you need, and it would go out, do the deep research, and return something structured and usable. You give it a description, and it pulls relevant info from across the web, suggests a schema based on what it finds, and generates a clean dataset. The schema is editable, and it also adds a short explanation of what the dataset covers. In some cases, it even asks follow-up questions to make the structure more useful. Started off as a quick…
- 1IB
I was experimenting with building a local dataset generator with deep research workflow a while back and that got me thinking. what if the same workflow could run on my own files instead of the internet. being able to query pdfs, docs or notes and get back a structured report sounded useful. so I made a small terminal tool that does exactly that. I point it to local files like pdf, docx, txt or jpg. it extracts the text, splits it into chunks, runs semantic search, builds a structure from my query, and then writes out a markdown report section by section. it feels like having a lightweight…
2025 · github.com
- 2IB
2025 · github.com
- 3
Doing research with agents is fun until they blow way past budget, jumble the sources, and don't even give you the best possible answer, just sound confident. And if you want to run some research task on local data - you have no idea where your data ends up after the prompt consumes it. So I built this tool: a deep-research agent with an enforced budget, verified quotes, and a privacy boundary for local data. 1. Never spend more than you budgeted (measured overshoot is 0%). 2. Every claim carries a source 3. Data stays local (give a CSV, it'll analyze it without the data ever leaving your…
23d ago · github.com
- 4DD
We recently used DeepSeek V4 Flash as a teacher for finance tasks with GPT-OSS-120B. Distillation works well on this problem. At a constrained 8k token budget, our self-distilled 120B scores 83.61% on FinanceReasoning, above Kimi K3 (81.93%) and Inkling (65.13%). We released the 20B open weights. With V4 as the teacher though, we realized it would be timely to measure if the censorship characteristic of it transferred to the distilled version of the base model. tl;dr it didn't, the teacher answered politically sensitive questions 7 SDs differently than expected, but the distilled model's…
Jul 2026 · ctgt.ai
- 5PI
Hi HN! I’m Alex from Parabola (https://parabola.io). Parabola is a visual programming tool for creating functional data flows that everyone can use. It’s entirely drag-and-drop, handles data sizes much larger than a traditional spreadsheet, calculates everything live, and can run your flows on a schedule of your choosing. I used to work in strategy consulting, doing data analytics for SMBs and Fortune 500 companies. The amount of time wasted on menial tasks was astounding. Things like cleaning data, generating custom reports, creating human workflows to solve shortcomings in third…
2018 · parabola.io
- 6
- 7DB
I've been doing some data cleaning for my fine tuning projects using LLMs, and decided to just build a package for it as a side project. Check it out here: https://github.com/databonsai/databonsai Some features: - categorization (labelling), transformation and decomposition (text into structured format) - validates llm outputs - batch mode batches up the inputs/outputs so you don't send the prompt (schema, fewshot examples) for every row of data, saving a significant amount of tokens There are some similarities to the Instructor repo, but this is simpler and made for…
2024 · github.com
- 8SA
Hey there! This is Dan and Ustin (@uzarubin), and we want to share something cool we've been working on for the past year - an open-source `tail -f` for your data, with a UI. We call it "Streamdal" which is a word salad for streaming systems (because we love them) and DAL or data access layer (because we’re nerds). Here's the repo: https://github.com/streamdal/streamdal Here's the site: https://streamdal.com And here's a live demo: https://demo.streamdal.com (github repo has an explanation of the demo) — — — THE PROBLEM We built this because the…
2023 · github.com
- 9FA
Hey HN! We’re building FinetuneDB (https://finetunedb.com/), an LLM fine-tuning platform. It enables teams to easily create and manage high-quality datasets, and streamlines the entire workflow from fine-tuning to serving and evaluating models with domain experts. You can check out our docs here: (https://docs.finetunedb.com/) FinetuneDB exists because creating and managing high-quality datasets is a real bottleneck when fine-tuning LLMs. The quality of your data directly impacts the performance of your fine-tuned models, and existing tools didn’t offer an easy…
2024 · finetunedb.com
- 10SO
Built a tool for transforming unstructured data into structured outputs using language models (with 100% adherence). If you're facing problems getting GPT to adhere to a schema (JSON, XML, etc.) or regex, need to bulk process some unstructured data, or generate synthetic data, check it out. We run our own tuned model (you can self-host if you want), so, we're able to have incredibly fine grained control over text generation. Repository: https://github.com/automorphic-ai/trex Playground: https://automorphic.ai/playground
2023 · automorphic.ai
- 11DA
Dear HN, I am Riwaj, the cofounder of dstack.ai (https://github.com/dstackai). A few months ago, we built an online service that allows users to publish data visualizations from Python or R. The idea was to build a tool that did not require additional programming or front-end development for publishing data visualizations. Such a code can be invoked from either Jupyter notebook, RMarkdown, Python, or R scripts. Once the data is pushed, it can be accessed via a browser. Open-sourcing dstack: During our customer discovery phase, we realized that dstack.ai should integrate a lot…
2020
- 12
- 13DD
Gleb, Alex, Erez and Simon here – we are building an open-source tool for comparing data within and across databases at any scale. The repo is at https://github.com/datafold/data-diff, and our home page is https://datafold.com/. As a company, Datafold builds tools for data engineers to automate the most tedious and error-prone tasks falling through the cracks of the modern data stack, such as data testing and lineage. We launched two years ago with a tool for regression-testing changes to ETL code…
2022
- 14OS
Hey HN! A few months ago we shared our AI dataset generator as an open source repo, and the response was incredible (https://news.ycombinator.com/item?id=44388093). We got requests from folks who wanted to use it without the hosting overhead, so we created both options: a hosted version (https://www.metabase.com/ai-data-generator for instant use and the source code fully open (https://github.com/metabase/dataset-generator) for anyone who wants to self-host or contribute. Looking forward to seeing how you use it and what you build on top of…
Sep 2025 · metabase.com
- 15CA
Synthetic data generation is an essential step in training and evaluating LLMs/Agents/RAG pipelines, but tooling around this is still lacking. We're introducing Curator, an open-source library designed to streamline the data curation process. While there are many libraries to prompt LLMs, the semantics of generating synthetic data is different from prompting. For example, we need to process a large number of prompts (sometimes in millions or more) while accepting some failures, utilize several stages of prompting, incorporate human feedback, and filter out bad data using verifiers…
2025 · github.com
- 16FL
Hi HN community, I have been working on benchmarking publicly available LLMs these past couple of weeks. More precisely, I am interested on the finetuning piece since a lot of businesses are starting to entertain the idea of self-hosting LLMs trained on their proprietary data rather than relying on third party APIs. To this point, I am tracking the following 4 pillars of evaluation that businesses are typically look into: - Performance - Time to train an LLM - Cost to train an LLM - Inference (throughput / latency / cost per token) For each LLM, my aim is to benchmark them for…
2023 · github.com
- 17FG
We developed a new framework that enables flexible control of generated text in language models. By combining several models and/or system prompts in one mathematical formula, it lets you tweak your style and combine model outputs with ease. A handy tool for those working with LLMs, looking for more fine-grained control of stylistic output. More details in our paper: https://arxiv.org/abs/2311.14479. Feedback and potential applications are welcome.
2023 · github.com
- 18MA
I've been exploring the (not so=) amazing potential of AI in coding and have compiled a list of tools. From AI-powered IDEs to code generators, this resource is my contribution to the community. I'm still on the fence about including txt2sql projects, as their functionality seems too basic to me. And I'm personally maintaining this, so your feedback is wellcome.
2025 · aicode.danvoronov.com
- 19GA
Simon(sfarshid) and I spend a lot of time on GitHub. As data nerds we put together a quick tool to explore your repository’s data. How it works: - Data Loading: We use dlt to pull data (issues, PRs, commits, stars) from GitHub - Semantic Layer: Relta wraps the underlying dataset into a semantic layer so the LLM doesn’t hallucinate. - Text-to-SQL: A text-to-SQL agent transforms your plain-English question into a query using the semantic layer - Generative Charts: assistant-ui dynamically generates a chart based on the SQL query - Refinements: If the semantic layer can’t handle your question,…
2024 · github.com
- 20OD
Hello Hacker News! We are Rick & Yannick from Orchest (https://www.orchest.io - https://github.com/orchest/orchest). We're building a visual pipeline tool for data scientists. The tool can be considered to be high-code because you write your own Python/R notebooks and scripts, but we manage the underlying infrastructure to make it 'just work™'. You can think of it as a simplified version of Kubeflow. We created Orchest to free data scientists from the tedious engineering related tasks of their job. Similar to how companies like Netflix, Uber and Booking.com…
2020
- 21NA
Hey HN - our team wants to open source a project called NASTY (NASTY Abstract Syntax Tree thingY) that we built for ourselves. NASTY was built to maintain testable/composable data pipelines. Our team was ripping our hair out trying to maintain dbt/SQL scripts across different data warehouses (Redshift, BigQuery, Postgres, Snowflake) on top of ever shifting data foundations maintained by our customer's internal data teams. NASTY is the result of our learnings from field experience. We wanted to write abstractions so that we could reuse code. We wanted to bundle those abstractions…
2024 · getnasty.dev
- 22DT
I built DDL to Data after repeatedly pushing back on "just use production data and mask it" requests. Teams needed populated databases for testing, but pulling prod meant security reviews, PII scrubbing, and DevOps tickets. Hand-written seed scripts were the alternative slow, fragile, and out of sync the moment schemas changed. Paste your CREATE TABLE statements, get realistic test data back. It parses your schema, preserves foreign key relationships, and generates data that looks real, emails look like emails, timestamps are reasonable, uniqueness constraints are honored. No setup, no…
Jan 2026
- 23TW
I want to share a new dataset of 331 reward-hackable environments. These are real environments used in Terminal Bench and adjacent benchmarks. I first got interested in this because, as a reviewer of Terminal Bench, I noticed a lot of our tasks were hackable. I also noticed that many contributors to the benchmark do so because it provides credibility when selling environments to labs. Hence, TBench tasks are, in my opinion, held to a higher quality standard than those being used today for RL. No one is spending hours manually reviewing the $1B in tasks being purchased by major labs. As far…
Apr 2026 · github.com
- 24MO
Solo Dev. Couch Potato. Build a Standalone Open Source Deep research tool. And it Beats Google , Open ai and Perplexity in Multible Metrics : https://veritas-test.neocities.org/ ( pls translate it its german) Guys : lets get this to be used. Because KNOWLAGE Shouldnt be locked behind Paywalls
Feb 2026 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →