Data Engineering with Rust
Hi HN! A few months ago, EntICOnc shared a blog post I wrote about using Rust from the perspective of a Python developer [0] which was well received, with some positive as well as mixed feedback. One thing I didn’t foresee would happen, is this little post getting a lot of attention, especially from the data engineering crowd which I consider myself a part of. Looking at the feedback, it was clear that there are some resources missing for a specific use case for which I think Rust is a perfect candidate: Data Engineering. Currently, Data Engineering is dominated by languages like Java, Scala…
What it does
In the maker’s words, at launch
Hi HN! A few months ago, EntICOnc shared a blog post I wrote about using Rust from the perspective of a Python developer [0] which was well received, with some positive as well as mixed feedback. One thing I didn’t foresee would happen, is this little post getting a lot of attention, especially from the data engineering crowd which I consider myself a part of. Looking at the feedback, it was clear that there are some resources missing for a specific use case for which I think Rust is a perfect candidate: Data Engineering. Currently, Data Engineering is dominated by languages like Java, Scala and of course, Python. However as I explain and attempt to prove with prose and code in the first and second chapters, Rust is a very serious and good option for data engineering workloads. While I’m not an expert in Rust, and explicitly disclose it, I think that I now know “enough” having written and maintained a few Rust workloads and comparing them to Python based ones. I started working on this project a while ago and I held back from sharing it here until at least a few chapters are “released” and available for people to read through and perhaps constructively criticise. I already sort of launched it on LinkedIn and Twitter, where it was very well received [1], so I thought it’s time to share it here as well. Right now there are 2 chapters: - Why Rust is a good choice for data engineering - How to get started with Rust Most of the Rust code on the website can run in your browser (but do require Javascript), some of the code snippets are also editable. This is thanks to mdbook (written in Rust, btw :) ) - should be fine on mobile too. It’s free, you don’t have to subscribe however I’d appreciate that and any feedback that can help me make this into an ongoing resource. If I made any mistakes, don’t hold back and let me know, I’ll fix them right away and credit you as well. I'm a french guy living in Germany, I'm very used to criticism ^^ [0] Carefully exploring Rust as a Python developer: https://news.ycombinator.com/item?id=33581901 [1] Latest LinkedIn post: https://www.linkedin.com/feed/update/urn:li:activity:7059274...
Does the same job
all alternatives →- IMI'm making a dynamic language in Rust2022 · github.com · ▲184
An implementation of a dynamic programming language in Rust. Includes: Parser/Compiler, REPL, Virtual Machine, Bytecode Disassembler This started out as a learning project to teach myself Rust. It has grown into a decently substantial piece of software and I've learned quite a bit in the process! Some neat things: + A garbage collector that can store dynamically sized types without any double-indirection (i.e. I have my own Box implementation with manual alloc/dealloc) + The smart pointer used to reference GCed data is a thin pointer. The ptr metadata needed for DSTs is stored in…
- BBBaseten – Build ML-powered applications2022 · baseten.co · ▲112
Hi HN, Three months ago, I took a job at Baseten to help craft and document an application builder that lets data scientists build full-stack, production-ready applications around their ML models without worrying about containers, Flask, or React. From my first day, everyone was focused on what would happen today: opening up our public beta. I’m super excited to see what you build with Baseten. If you want to take Baseten for a full-speed test drive, follow along with this tutorial, where you can build and deploy an application in 20 minutes:…
- ASAn SQL Builder in Rust2022 · github.com · ▲15
Hi guys. I just write an SQL builder. I want to know what you think.
- RVRust vs. Go – What are your thoughts?2018 · ▲10
I personally haven't event used or looked at Rust so I don't have an opinion. So now I'm interested hearing what others think? I think the syntax of Rust isn't very appealing, but that's not a good reason to /not/ look at it. Convince me? :)
- IQI quit my job to build a Kubernetes GUI with Rust2023 · aptakube.com · ▲13
About a year ago I was working on a project with mirrored Kubernetes clusters deployed across various regions. Constantly switching contexts was a big pain for me and I badly needed something more practical to work with multiple clusters, ideally simultaneously. At that time everyone was talking about Rust and I really wanted to try it out. I then found Tauri, a framework to build GUI apps with Rust + JavaScript. I was sold. I learned Rust and built the app I always wanted to use. Fast, lightweight and with the ability to connect to multiple clusters simultaneously. Fast forward to today,…
- ABA book introducing Geospatial and Rust2022 · book.georust.org · ▲11
My friend and I have been collaborating on this lighthearted introduction to doing geospatial computation in rust. Don't know Rust? Don't know geospatial? I'd be interested to know what you think.
More dev tools this month
the category →
Open-source GTM skills for technical founders
Dev tools · 29d ago · gtmcofounder.com
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 1d ago · opentrailpaper.com
Launched alongside, May 2023
the whole month →
- BR
In today's world, catchy headlines and articles often distract readers from the facts and relevant information. By utilizing OpenAI's language models, Boring Report processes sensationalist news articles, transforms them into the content you see, and helps readers focus on the essential details. We recently updated our iOS app experience, so any and all feedback would be appreciated. App Link: https://apps.apple.com/us/app/boring-report-news-by-ai/id644...
AI · 2023 · boringreport.org
Momentum Page▲834Launch your website in seconds, get users in minutes
Dev tools · 2023 · page.mmntm.build