Orchest – Data Science Pipelines
Hello Hacker News! We are Rick & Yannick from Orchest (https://www.orchest.io - https://github.com/orchest/orchest). We're building a visual pipeline tool for data scientists. The tool can be considered to be high-code because you write your own Python/R notebooks and scripts, but we manage the underlying infrastructure to make it 'just work™'. You can think of it as a simplified version of Kubeflow. We created Orchest to free data scientists from the tedious engineering related tasks of their job. Similar to how companies like Netflix, Uber and Booking.com…
What it does
In the maker’s words, at launch
Hello Hacker News! We are Rick & Yannick from Orchest (https://www.orchest.io - https://github.com/orchest/orchest). We're building a visual pipeline tool for data scientists. The tool can be considered to be high-code because you write your own Python/R notebooks and scripts, but we manage the underlying infrastructure to make it 'just work™'. You can think of it as a simplified version of Kubeflow. We created Orchest to free data scientists from the tedious engineering related tasks of their job. Similar to how companies like Netflix, Uber and Booking.com support their data scientists with internal tooling and frameworks to increase productivity. When we worked as data scientists ourselves we noticed how heavily we had to depend on our software engineering skills to perform all kinds of tasks. From configuring cloud instances for distributed training, to optimizing the networking and storage for processing large amounts of data. We believe data scientists should be able to focus on the data and the domain specific challenges. Today we are just at the very beginning of making better tooling available for data science and are launching our GitHub project that will give enhanced pipelining abilities to data scientists using the PyData/R stack, with deep integration of Jupyter Notebooks. Currently Orchest supports: 1) visually and interactively editing a pipeline that is represented using a simple JSON schema; 2) running remote container based kernels through the Jupyter Enterprise Gateway integration; 3) scheduling experiments by launching parameterized pipelines on top of our Celery task scheduler; 4) configuring local and remote data sources to separate code versioning from the data passing through your pipelines. We are here to learn and get feedback from the community. As youngsters we don't have all the answers and are always looking to improve.
Does the same job
all alternatives →
- DAdstack – an open-source tool to build data applications easily2020 · ▲134
Dear HN, I am Riwaj, the cofounder of dstack.ai (https://github.com/dstackai). A few months ago, we built an online service that allows users to publish data visualizations from Python or R. The idea was to build a tool that did not require additional programming or front-end development for publishing data visualizations. Such a code can be invoked from either Jupyter notebook, RMarkdown, Python, or R scripts. Once the data is pushed, it can be accessed via a browser. Open-sourcing dstack: During our customer discovery phase, we realized that dstack.ai should integrate a lot…
- PIParabola.io – Automate your work with visual programming2018 · parabola.io · ▲291
Hi HN! I’m Alex from Parabola (https://parabola.io). Parabola is a visual programming tool for creating functional data flows that everyone can use. It’s entirely drag-and-drop, handles data sizes much larger than a traditional spreadsheet, calculates everything live, and can run your flows on a schedule of your choosing. I used to work in strategy consulting, doing data analytics for SMBs and Fortune 500 companies. The amount of time wasted on menial tasks was astounding. Things like cleaning data, generating custom reports, creating human workflows to solve shortcomings in third…
- SPSelf-Published Book – “Data Science in Production”2020 · ▲172
Hi HN, Over the past 6 months I've been working on a technical book focused on helping aspiring data scientists to get hands-on experience with cloud computing environments using the Python ecosystem. The book is targeted at readers already familiar with libraries such as Pandas and scikit-learn that are looking to build out a portfolio of applied projects. To author the book, I used the Leanpub platform to provide drafts of the text as I completed each chapter. To typeset the book, I used the R bookdown package by Yihui Xie to translate my markdown into a PDF format. I also used Google docs…
- PFPlatform for data scientists2013 · ▲7
Hi there Hacker News, I've started a side project http://datasourcehub.com which aims to be a platform for data scientists. The project is still in the idea phase so the UI/UX and functionality are all subject to change. Feel free to play around, below is a guest login, and make sure files are content type of 'text/csv'. All data is subject to deletion, it's just a sandbox right now! By reaching out to the Hacker News community I hope to reach expert data scientists and get their feedback. Below are some questions I'd like to answer and some proposed directions that this…
More dev tools this month
the category →
Open-source GTM skills for technical founders
Dev tools · 29d ago · gtmcofounder.com
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 1d ago · opentrailpaper.com
Launched alongside, August 2020
the whole month →
- IB
Life & fun · 2020 · storycreatorapp.com
