dstack – an open-source tool to build data applications easily
Dear HN, I am Riwaj, the cofounder of dstack.ai (https://github.com/dstackai). A few months ago, we built an online service that allows users to publish data visualizations from Python or R. The idea was to build a tool that did not require additional programming or front-end development for publishing data visualizations. Such a code can be invoked from either Jupyter notebook, RMarkdown, Python, or R scripts. Once the data is pushed, it can be accessed via a browser. Open-sourcing dstack: During our customer discovery phase, we realized that dstack.ai should integrate a lot…
What it does
In the maker’s words, at launch
Dear HN, I am Riwaj, the cofounder of dstack.ai (https://github.com/dstackai). A few months ago, we built an online service that allows users to publish data visualizations from Python or R. The idea was to build a tool that did not require additional programming or front-end development for publishing data visualizations. Such a code can be invoked from either Jupyter notebook, RMarkdown, Python, or R scripts. Once the data is pushed, it can be accessed via a browser. Open-sourcing dstack: During our customer discovery phase, we realized that dstack.ai should integrate a lot more open source data science frameworks than we integrated ourselves. For example, as a user, I want to push a matplotlib plot, a Tensorflow model, a plotly chart, a pandas dataframe, and I expect the presentation layer to fully-support it. Supporting all types of artifacts and providing all the tools to work with them solely seems to be a very challenging task. With this, we open-sourced the framework. Now you can build dstack locally, and run it on your servers, or in a cloud of your choice if that’s needed. More details on the project, how to use it, and the source code of the server can be found at the https://github.com/dstackai/dstack repo. The client packages for Python and R are available at the https://github.com/dstackai/dstack-py and https://github.com/dstackai/dstack-r correspondingly. What’s next: User callbacks- so that application shows not just pre-calculated visualizations but also can fetch data from a store and process it in real-time. ML models- so that data scientists can publish a stack which binds together a pre-calculated ML model and user parameters Use cases- Support specific use cases that help data scientists to build data science models into data applications as fast as possible. We would be happy to get your feedback on the open-source framework and also get your opinion on what kind of use cases can be built on top of the framework? Thank you.
Does the same job
all alternatives →
- DAdstack.ai – publish, track, and share data visualizations2020 · dstack.ai · ▲20
- DADstack – a command-line utility to provision infra for ML workflows2022 · github.com · ▲5
Hi. :) I’m Andrey, the creator of dstack. I started this project while I was working at JetBrains where I helped the PyCharm team to improve support for Jupyter notebooks. As I was in close contact with many ML devs (who used PyCharm) I was able to see their struggle with running ML workflows. Unlike traditional dev workflows, ML workflows are difficult to run on a local machine (due to the lack of memory, more CPUs/GPUs, etc). This is why people often have to use remote machines (e.g. via SSH), or adopt one of the end-to-end MLOps platforms. Using remote machines is not difficult but…
- SPSelf-Published Book – “Data Science in Production”2020 · ▲172
Hi HN, Over the past 6 months I've been working on a technical book focused on helping aspiring data scientists to get hands-on experience with cloud computing environments using the Python ecosystem. The book is targeted at readers already familiar with libraries such as Pandas and scikit-learn that are looking to build out a portfolio of applied projects. To author the book, I used the Leanpub platform to provide drafts of the text as I completed each chapter. To typeset the book, I used the R bookdown package by Yihui Xie to translate my markdown into a PDF format. I also used Google docs…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 27d ago · cactuscompute.com
Launched alongside, August 2020
the whole month →
- IB
Life & fun · 2020 · storycreatorapp.com
