Vectara Ingest

What it does

vectara-ingest demonstrates how to crawl datasets and ingest them into Vectara. Includes several types of crawlers like website, RSS, Jira, Notion, or Docusarurus, and you can add your own crawler very easily to handle other data sources.

An open source framework to crawl data sources and ingest into Vectara - vectara/vectara-ingest

• 
  I built an open-source data copy tool called ingestr2024 · github.com · ▲156

  Hi there, Burak here. I built an open-source data copy tool called ingestr (https://github.com/bruin-data/ingestr) I did build quite a few data warehouses both for the companies I worked at, as well as for consultancy projects. One of the more common pain points I observed was that everyone had to rebuild the same data ingestion bit over and over again, and each in different ways: - some wrote code for the ingestion from scratch to various degrees - some used off-the-shelf data ingestion tools like Fivetran / Airbyte I have always disliked both of these approaches,…

• RH
  Replace "hub" by "ingest" in GitHub URLs for a prompt-friendly extract2024 · gitingest.com · ▲185

  Gitingest is a open-source micro dev-tool that I made over the last week. It turns any public Github repository into a text extract that you can give to your favourite LLM easily. Today I added this url trick to make it even easier to use! How I use it myself: - Quickly generate a README.md boilerplate for a project - Ask LLMs questions about an undocumented codebase It is still very much work in progress and I plan to add many more options (file size limits, exclude patterns..) and a public API I hope this tool can help you Your feedback is very valuable to help me prioritize And…

• 
  Turn any website into a knowledge base for LLMs2024 · embedding.io · ▲305

  I built this tool because I wanted a way to just take a bunch of URLs or domains, and query their content in RAG applications. It takes away the pain of crawling, extracting content, chunking, vectorizing, and updating periodically. I'm curious to see if it can be useful to others. I meant to launch this six months ago but life got in the way...

• WI
  Web Ingest – Convert websites to LLM-friendly text2025 · webingest.com · ▲8

  I really love Git Ingest, but I needed something more general. So I built this! It crawls websites and converts each page to Markdown. It’s great for developer documentation. And it’s free. I built it over the excellent Jina Reader. I hope you get some use out of it!

• 
  Vecta.io - Team diagramming with an advanced SVG editor2017 · vecta.io · ▲154

  Vecta is a real time, collaborative SVG editor with a powerful Javascript based plugin system, for teams. Easily extend and automate, or create flowcharts, UML, network, isometric and web diagrams with thousands of included symbols. Export to multiple formats and resolution, including PNG, JPG and m

• 
  LLM Scraper – turn any webpage into structured data2024 · github.com · ▲88

  Turn any webpage into structured data using LLMs. Contribute to mishushakov/llm-scraper development by creating an account on GitHub.

• 
  Spira Maxima▲578

  Video model that turns script into viral social media videos

  AI · 18d ago · spira.ai

• 
  Rill Browser▲540

  The browser where Claude Code and Codex work beside you

  AI · 18d ago · rill.love

• 
  Germany's new sovereign AI model Kolibri▲424

  Kolibri is Aleph Alpha

  AI · 20d ago · tej.as

• 
  Giving Opus 5.5 a simulated paint canvas▲392

  Paintings by AI models: programs run through a simulation of oil paint.

  AI · 21d ago · stillwet.art

• 
  PixVerse R2▲381

  A real-time world model you can explore and change

  AI · 30d ago · world.pixverse.video

• 
  Beam.win▲378

  Run AI agents on one Mac and use its windows on another

  AI · 5d ago · beam.win

• 
  Wordware▲9,848

  Your tool for building AI agents with natural language

  AI · 2024 · wordware.ai

• 
  If YouTube had actual channels▲2,741

  If Youtube had actual channels.

  Life & fun · 2024 · ytch.xyz

• 
  Me.bot▲1,627

  The inspiring companion for your life

  AI · 2024 · me.bot

• 
  Upmetrics AI▲1,197

  Business plan & financial forecast using AI

  Work · 2024 · upmetrics.co

• 
  Jupitrr AI▲1,195

  Jupitrr AI turns raw talking videos into scroll-stopping content for coaches and consultants. Research, script, record, edit, and publish in one app.

  AI · 2024 · jupitrr.com

• 
  InstantDB – A Modern Firebase▲1,145

  Hey there HN! We’re Joe and Stopa, and today we’re open sourcing InstantDB, a client-side database that makes it easy to build real-time and collaborative apps like Notion and Figma. Building modern apps these days involves a lot of schleps. For a basic CRUD app you need to spin up servers, wire up endpoints, integrate auth, add permissions, and then marshal data from the backend to the frontend and back again. If you want to deliver a buttery smooth user experience, you’ll need to add optimistic updates and rollbacks. We do these steps over and over for every feature we build, which can…

  Dev tools · 2024 · github.com