Simple CLI to aggregate repos, papers and docs for LLM ingestion
I created a tool that consolidates information from the following inputs: GitHub repository URL (e.g., https://github.com/jimmc414/onefilellm) arXiv abstract URL (e.g., https://arxiv.org/abs/2401.14295) Local folder path (e.g., C:\python\PipMyRide) Youtube video URL (e.g., https://www.youtube.com/watch?v=KZ_NlnmPQYk) Webpage URL (e.g., https://llm.datasette.io/en/stable/) It outputs the repo, web documentation, arXiv paper or YT transcript to a text file and the clipboard, displaying a token count. It also…
What it does
In the maker’s words, at launch
I created a tool that consolidates information from the following inputs: GitHub repository URL (e.g., https://github.com/jimmc414/onefilellm) arXiv abstract URL (e.g., https://arxiv.org/abs/2401.14295) Local folder path (e.g., C:\python\PipMyRide) Youtube video URL (e.g., https://www.youtube.com/watch?v=KZ_NlnmPQYk) Webpage URL (e.g., https://llm.datasette.io/en/stable/) It outputs the repo, web documentation, arXiv paper or YT transcript to a text file and the clipboard, displaying a token count. It also produces an additional file of the same content that reduces token usage by removing stopwords and converting to lowercase. The purpose of this tool is to make it easier to manually pass repos, papers, transcripts or web documentation into an LLM for inference. For accessing Github repos you just need to generate a github personal token as described in the readme. I needed this tool for myself and thought it might be of use to someone else.
Does the same job
all alternatives →
- IMI made a webapp which summarizes YT videos with LLMs2024 · github.com · ▲5
Live demo here: http://fonctionlabs.com:8000 Similarly to aka_sh (guess we were working parallelly on similar topics), I created with my brother a chainlit-based webapp, which summarizes Youtube videos in order to gain time. It works as an RAG-based LLM, and is very light in the sense that it does not use RAG libraries like langchain or llamaindex. You can use it with your own OpenAI API key. It also supports local models like Mistral, or Llamma. It is ofc open-source, and you can deploy with Docker if you choose. Some of the next steps are: - using whisper to be able to compute a…
- UEUithub – Easily Ask Your LLM Coding Questions2024 · uithub.com · ▲6
I've created uithub, a tool that allows developers to easily get LLM context for their coding questions and perform AI repo analysis at scale. Here's what it does: - Get Context: Simply change the 'g' in github.com to 'u' to access AI-powered insights on any GitHub repo. - Flexible Querying: Fetch entire repos, specific branches/subfolders, or filter by file type and size. - API for Developers: Power the next generation of development tools with our API. Key features: - Customizable token limits - File type filtering - Multiple response formats - Size-based file exclusion I built this…
- RSRenderarXiv – Search ArXiv from terminal, HTML to read/paste into LLMOct 2025 · github.com · ▲6
Terminal tool to search arXiv papers and render them as beautiful HTML for humans and LLMs
- IBI built a local Elixir/Python pipeline to curate 14,000 RAW photosApr 2026 · qwelian.com · ▲5
I had 14,000 photos sitting on a drive and wanted an excuse to play with local vision models and Elixir/Phoenix. I originally tried to get LLaVA to tell me if a photo was 'good' or matched my style, but quickly learned that LLMs have terrible taste. I ended up demoting the LLM to just extract metadata, and built a custom CLIP/Ridge Regression pipeline to actually learn my preferences based on how I rate things. The stack is Phoenix/Oban on the orchestrator side, and Python/FastAPI/Instructor for the AI workers. Happy to answer any questions about the architecture,…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 26d ago · cactuscompute.com
Launched alongside, February 2024
the whole month →
MarketingIdeas.com▲1,230Never run out of marketing ideas ever again
Growth · 2024 · marketingideas.com
