Alternatives
Products that do what Custom Data engineering services does
data engineering, Data Migration
- 1

- 2DE
Hi HN! I'm currently a Master's student at USTC (University of Science and Technology of China). I've been diving deep into Data Engineering, especially in the context of Large Language Models (LLMs). The Problem: I found that learning resources for modern data engineering are often fragmented and scattered across hundreds of medium articles or disjointed tutorials. It's hard to piece everything together into a coherent system. The Solution: I decided to open-source my learning notes and build them into a structured book. My goal is to help developers fast-track their learning curve. Key…
Feb 2026 · github.com
- 3

- 4

- 5

- 6

- 7
- 8

- 9PA
2019 · github.com
- 10

- 11IB
Every data pipeline job I had to tackle required quite a few components to set up: - One tool to ingest data - Another one to transform it - If you wanted to run Python, set up an orchestrator - If you need to check the data, a data quality tool Let alone this being hard to set up and taking time, it is also pretty high-maintenance. I had to do a lot of infra work, and while this being billable hours for me I didn’t enjoy the work at all. For some parts of it, there were nice solutions like dbt, but in the end for an end-to-end workflow, it didn’t work. That’s why I decided to build an…
2024 · github.com
- 12

- 13

- 14DA
2021 · github.com
- 15DE
2020 · awesomedataengineering.com
- 16SF
2020 · github.com
- 17

- 18

- 19

- 20

- 21DL
2014 · databaselabs.io
- 22OS
Hey HN, I am the founder of Tensorlake. Prototyping LLM applications have become a lot easier, building decision making LLM applications that work on constantly updating data is still very challenging in production settings. The systems engineering problems that we have seen people face are - 1. Reliably process ingested content in real time if the application is sensitive to freshness of information. 2. Being able to bring in any kind of model, and run different parts of the pipeline on GPUs and CPUs. 3. Fault Tolerance to ingestion spike, compute infrastructure failure. 4. Scaling compute,…
2024 · getindexify.ai
- 23PI
2019 · medium.com
- 24DM
2014 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →