I made a VS Code extension where you can use Mixtral 8x7B for free

Hey HN! We’ve forked Jupyter Lab and added AI code generation features that feel native and have all the context about your notebook. You can see a demo video (2 min) here: https://www.tella.tv/video/clxt7ei4v00rr09i5gt1laop6/view Try a hosted version here: https://pretzelai.app Jupyter is by far the most used Data Science tool. Despite its popularity, it still lacks good code-generation extensions. The flagship AI extension jupyter-ai lags far behind in features and UX compared to modern AI code generation and understanding tools (like…

Hello HN, I don't post on here much, but wanted to get some eyes on a new project I'm just launching. I think we definitely need one more AI code agent.. I'm a long-term C++ dev, and over 30+ years I've created some successful audio dev tools (JUCE, the Tracktion DAW, the Cmajor DSP language). All of these came from me getting annoyed with something I had to use, and deciding to have a go at my own take on whatever it was. So Juggler is my attempt at an AI code agent, after spending too many hours loving what the models could do, but hating the CLI experience, and having some opinions of…

Try it out! https://glhf.chat/ Hey HN! We’ve been working for the past few months on a website to let you easily run (almost) any open-source LLM on autoscaling GPU clusters. It’s free for now while we figure out how to price it, but we expect to be cheaper than most GPU offerings since we can run the models multi-tenant. Unlike Together AI, Fireworks, etc, we’ll run any model that the open-source vLLM project supports: we don’t have a hardcoded list. If you want a specific model or finetune, you don’t have to ask us for it: you can just paste the Hugging Face link in and…

The fast, easy and cheap OpenAI alternative

I’d originally launched my app: Private LLM[1][2] on HN around 10 months ago, with a single RedPajama Chat 3B model. The app has come a long way since then. About a month ago, I added support for 4-bit OmniQuant quantized Mixtral 8x7B Instruct model, and it seems to outperform Q4 models at inference speed and Q8 models at text generation quality, while consuming only about 24GB of RAM[3] at 8k context length. The trick is: a) to use a better quantization algorithm and b) to use unquantized embeddings and the MoE gates (the overhead is quite small). Other notable features include many more…

Hey HN! When implementing an AI-powered feature for a project, we—and many people we've talked to—often reach a point where we have to choose an AI model but aren’t sure which one best fits our constraints or where to even start. Unfortunately, the advice to "just use chatgpt" is not always a good one. What if I want an open-source model? What languages does it support? What about context window size or the number of parameters? There are thousands of AI models already out there and many of them are perfect for certain problems. That’s why we’ve carved out this part of our product as a free…