Alternatives
Products that do what Rag over all physics-related Wikipedia pages does
A few weeks ago, during the LK-99 hype, I made a demo app to explain superconductors in simple terms using RAG over relevant Wikipedia pages. I thought it would be cool to extend this to all physics-related Wikipedia pages (which turns out to be ~ 14K). After getting the text and splitting the pages, I ended up with around 100K chunks. I created all the embeddings using OpenAI’s embedding API, which cost around $7. I stored all the vectors in Pinecone.
- 1FB
Hey there HN! We’re Antonio, Luca, and Yuhang, and we’re excited to introduce Fast GraphRAG, an open-source RAG approach that leverages knowledge graphs and the 25 years old PageRank for better information retrieval and reasoning. Building a good RAG pipeline these days takes a lot of manual optimizations. Most engineers intuitively start from naive RAG: throw everything in a vector database and hope that semantic search is powerful enough. This can work for use cases where accuracy isn’t too important and hallucinations are tolerable, but it doesn’t work for more difficult queries that…
2024 · github.com
- 2WD
Connect two articles on Wikipedia, but do it the long way. I've always been a fan of the theory of six degree of separation, but it's an overused concept when exploring the Wiki-graph. Instead of showing the shortest path, which in my opinion is "boring" and ends up connecting super-important central articles, I came up with my own method: WikiBinge selects the smaller, less represented articles on Wikipedia. In a WikiBinge path, the underdogs are the kings! How does it work? It's pretty straightforward! Compute PageRank on the Wiki-graph and assign as weight of each edge the PageRank value…
2023 · wikibinge.com
- 3TA
I built this tool because I wanted a way to just take a bunch of URLs or domains, and query their content in RAG applications. It takes away the pain of crawling, extracting content, chunking, vectorizing, and updating periodically. I'm curious to see if it can be useful to others. I meant to launch this six months ago but life got in the way...
2024 · embedding.io
- 4IM
As a grad student (and an ADHDer), I had trouble doing literature review systematically. To combat this, I made a website that finds similar papers using the meaning of the thing I am looking for. I used MixedBread's [^1] embedding model to generate vectors from the abstracts. I store and search similar vectors using Milvus [^2] and finally use Gradio [^3] to serve the frontend. I update the vector database weekly by pulling the metadata dataset from Kaggle [^4]. To speed up the search process on my free oracle instance, I binarise the embeddings and use Hamming distance as a metric. I would…
2024 · papermatch.mitanshu.tech
- 5IM
Hey HN! I love finding new books to read on here. I wanted to gather the most mentioned books and recreate the serendipity of physical browsing. I scraped 20k comments from HN threads related to reading, extracted the references and opinions using GPT-4o mini, and visualised their embeddings as a map. - OpenAI's embeddings were processed using UMAP and HDBSCAN. A direct 2D projection from the text embeddings didn't yield visually interesting results. Instead, HDBSCAN is first applied on a high-dimensional projection. Those clusters tend to correspond to different genres. The genre…
2024 · hnbooks.pieterma.es
- 6AH
2018 · upvote.pub
- 7ET
2013 · github.com
- 8
- 9AB
Recently i stumbled on too many clickgates on the Medium blog Towards Data science. Considering that most people publish to share knowledge on Medium and are driven into putting their content behind a paywall, without actually getting paid for it, including myself. I felt like Medium is running the academic publishing scheme. Get free content and get paid for it. So I decided to create a small script to bypass the paywall on Medium, it turns out it also works on other newssites. Heres the website: https://sugoidesune.github.io/readium/ For the curious I will explain the…
2019
- 10PF
Introducing embeds.ai: an embedding playground to compare how embedding models work on a real world use case (retrieval augmented generation for Wikipedia articles + Elad Gil's High growth handbook) A few weeks ago, Shreyan and I were looking for an embedding model to use for RAG. We eventually came across the MTEB leaderboard, but we struggled to understand the benchmark scores. We wanted a tool to test various embedding models with example queries on real-world datasets. After unsuccessfully looking for such a “playground”, we decided to just build one ourselves! We embedded HuggingFace’s…
2023 · embeds.ai
- 11TA
Hey HN, I've been working on a project called ThreadQuilt, and I'm excited to share it with you all. ThreadQuilt is a community discussion aggregator that brings together the best threads from various forums and platforms into one convenient place. Whether you're interested in programming, tech trends, or just want to stay updated on niche topics, ThreadQuilt helps you find and follow the most relevant conversations without the clutter of AI-generated content. It's all about real human discussions, curated for quality and relevance. I built ThreadQuilt because I was tired of wading through…
2024 · threadquilt.com
- 12IB
Hey all! I wanted to share this project I've been working on that can maybe help you or your developer friends out. I built a RAG system for our product a while back and didn't realize how easy they were to get started. So I put together my learnings into this online course. It's not quite ready but if you sign up and mention HackerNews, I can get you early access. I'm looking to get feedback on the following: (1) materials — is it engaging & did you learn something? (2) UI/UX of the platform — did you have any issues that prevented you from starting or finishing the tutorial? (3)…
2024 · takehomes.com
- 13PS
We recently released an open source project that syncs wikipedia with a vector database : https://github.com/Piazza-tech/Piazza-Updater We used Verba, Weaviate and Docker for deployment We'd like to have some feedback on how to continue the project, which data sources would be interesting to vectorize. You can give feedback on our landing page http://piazza.tech Please leave a star !
2024 · piazza.tech
- 14AE
Hi all, Sharing a repo I was working on for a while. It’s open-source and includes many different strategies for RAG (currently 17), including tutorials, and visualizations. This is great learning and reference material. Open issues, suggest more strategies, and use as needed. Enjoy!
2024 · github.com
- 15OS
Hello. This is an easy-to-use application for exploring your own data with retrieval augmented generation (RAG) backed by txtai. txtai is an all-in-one embeddings database for semantic search, LLM orchestration and language model workflows. txtai has a feature to automatically create knowledge graphs using semantic similarity. This enables running Graph RAG queries with path traversals. This RAG application generates a visual network to illustrate the path traversals and help understand the context from which answers are generated from. Embeddings databases are used as the knowledge store.…
2024 · hub.docker.com
- 16ML
We’ve recently open-sourced Model2vec, a method to distill sentence transformers into static embeddings that outperform all previous approaches by a large margin on MTEB. Our new models set a new state-of-the-art for static embeddings. Main features: - Our best model (potion-base-8M) has only 8M parameters, which is ~30mb on disk - Inference is ~500x faster than the distilled base model (bge-base), on a CPU - New models can be distilled in 30 seconds on a CPU without requiring a dataset - just a vocabulary - Numpy-only inference: The packaged can be install the package with minimal…
2024 · github.com
- 17BS
I saw that a lot of interesting Wikipedia articles regularly crop up on HN. So, just made a quick hack which lets you browse such interesting articles in a Stumbleupon fashion. This may be seen as a replacement of Wikipedia's own random article function which usually outputs stubs or other non-interesting links. I have seeded the existing database with 100 odd links, please feel free to submit new ones (only accepts Wikipedia links) or vote on existing ones. Here is the link: http://wingify.com/arbid/ Even though this is super-simple at the moment, feedback and suggestions are welcome! PS:…
2010
- 18SV
Hi HN, I'm Daniel from Superlinked! We have built an open-source framework that improves vector search relevance and usefulness by combining structured metadata with unstructured data in your embeddings. We included self-hostable API server that sits between your data sources and vector database. Docs: https://docs.superlinked.com/ We're launching our cloud offering soon where you can use Superlinked to orchestrate high-performance retrieval for RAG, Search & Recommendation apps in your own cloud. Looking for feedback and happy to answer questions!
2024 · github.com
- 19PA
We're excited to release PaperQA2, an open source RAG library specialized to work with the scientific literature. We've seen some really compelling results with it (https://paper.wikicrow.ai), like superhuman performance at question answering and summarization when compared with expert scientists. PaperQA2 is a major overhaul of our prior PaperQA system, it includes automatically obtained rich metadata for each paper, a CLI to work with local papers directly, a local full-text search engine for keywords searches over PDF files, a state-of-the-art algorithm for LLM-based re-ranking…
2024 · github.com
- 20PR
Hi HN, While building RAG agents, I noticed a lot of token budget was wasted on formatting overhead (HTML tags, JSON structure, whitespace). Existing solutions felt too heavy (often requiring torch/transformers), so I wrote this lightweight, zero-dependency library to solve it. It includes strategies for context packing, PII redaction, and tool output compression. Benchmarks show it can save ~15% of tokens with negligible latency overhead (<0.5ms). Happy to answer any questions!
Dec 2025 · github.com
- 21RA
Hey HN! If you’re at all interested in LLMs/LLM-apps, you’ve probably heard of RAG: Retrieval-Assisted Generation, i.e. retrieving relevant documents to give to your LLM as context to answer user queries. Today, I’m releasing RAGatouille v0.0.1, whose aim is to make it as easy as can be to improve your RAG pipelines by leveraging state-of-the-art Information Retrieval research. As of right now, there’s quite a big gap between common everyday practice and the IR literature, and a lot of the gap is because there just aren’t good ways to quickly try out and leverage SotA IR techniques.…
2024 · github.com
- 22WC
Nov 2025 · myclone.is
- 23PT
Hey all, I built a quick PoC that scrapes a webpage, sends the content to Gemini Flash, and outputs a clean, structured JSON — ready for RAG workflows. In my case, I’ll use this structured data to enhance models by integrating external knowledge sources during the generation process. Curious if you think this has potential or if there are any use cases I might have missed. Happy to share more details if there's interest!
2025 · structured.pages.dev
- 24
Ranked by how close each launch is in meaning, then by votes. Refine with a description →