nowfound

Dev tools · July 18, 2022

AJ

A just another Cron alternative but with much more capabilities

I have created a Cron alternative that runs DAGs (Directed acyclic graph) defined in a simple YAML format. Why not Airflow? Airflow and other similar tools are powerful and valuable, but in most cases, they require writing code to manage workflows. Our ETL pipeline is already hundreds of thousands of lines of complex code in Perl and shell scripts. Adding another layer of Python on top of this would make it difficult to maintain. Instead, we needed a more lightweight solution. So we developed Dagu, which requires no coding, and is easy-to-use and self-contained, making it ideal for smaller…

What it does

In the maker’s words, at launch

I have created a Cron alternative that runs DAGs (Directed acyclic graph) defined in a simple YAML format. Why not Airflow? Airflow and other similar tools are powerful and valuable, but in most cases, they require writing code to manage workflows. Our ETL pipeline is already hundreds of thousands of lines of complex code in Perl and shell scripts. Adding another layer of Python on top of this would make it difficult to maintain. Instead, we needed a more lightweight solution. So we developed Dagu, which requires no coding, and is easy-to-use and self-contained, making it ideal for smaller projects with fewer people.

Does the same job

all alternatives →
  • Apache Airflow on Astronomer2018 · ▲87

    Write and deploy custom ETL pipelines in Python

  • FH
    Free hosted open-source alternative to Zapier/Airflow2020 · cloud.titanoboa.io · ▲279
  • PI
    Parabola.io – Automate your work with visual programming2018 · parabola.io · ▲291

    Hi HN! I’m Alex from Parabola (https://parabola.io). Parabola is a visual programming tool for creating functional data flows that everyone can use. It’s entirely drag-and-drop, handles data sizes much larger than a traditional spreadsheet, calculates everything live, and can run your flows on a schedule of your choosing. I used to work in strategy consulting, doing data analytics for SMBs and Fortune 500 companies. The amount of time wasted on menial tasks was astounding. Things like cleaning data, generating custom reports, creating human workflows to solve shortcomings in third…

  • EA
    Eno – A lightning fast, user-friendly YAML/TOML alternative2018 · ▲102

    eno [1] - A modern plaintext language w/ libraries [2] for JavaScript, Python, Ruby & soon more! We migrated a big relational research database to a file-based solution - requirements were: - Super fast and easy editability for users - Highest performance for parsing/validating >10K documents on every user change. Our trials with YAML/TOML showed us that we wanted something both faster [3][4] and easier [4], something tailored for file-based content management ... and after months of research & development it's now publicly available (under MIT license) for everyone! Last but…

  • AI
  • PP
    Panda Patrol: Open-Source Data Pipeline Monitoring in < 5 Lines of Code2023 · panda-patrol.vercel.app · ▲9

    I’ve found that I don’t have any context about the data in my pipelines only know if the pipeline is successful or not. So I built panda-patrol which allows you to monitor each node in your DAG, use AI to generate data tests for your data, store data profiles, and more. All with this comes with dashboards and alerts. You can easily drop it into your Python-based data pipeline (i.e. Airflow, Dagster, Prefect, etc.) and just run your pipelines are you normally would — but with monitoring and more context. Hope its valuable to some people

More dev tools this month

the category →
  • Dograh592

    The open source VAPI alternative

    Dev tools · 25d ago · dograh.com

  • Meridian530

    Don't let your work go unnoticed. Get promoted!

    Dev tools · 20d ago · meridiona.com

  • x1516

    Lovable for iPhone apps go from idea to App Store

    Dev tools · 11d ago · x1.new

  • Open-source GTM skills for technical founders

    Dev tools · 29d ago · gtmcofounder.com

  • OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.

    Dev tools · 2d ago · opentrailpaper.com

  • Nuphos380

    The AI-Native DevOps Workspace.

    Dev tools · 24d ago · nuphos.ai

Launched alongside, July 2022

the whole month →
  • Locofy.ai720

    Turn Figma designs into code

    Dev tools · 2022 · locofy.ai

  • PO

    Life & fun · 2022 · justine.lol

  • PO

    Dev tools · 2022 · github.com

  • ruttl552

    Collect visual feedback directly on websites, PDFs & images

    Dev tools · 2022

  • PP

    Hello, HN - I wanted to share this puzzle game I made during my vacation. I'm rather fond of the pipes puzzle where your goal is to restore a scrambled network of connections by rotating tiles. It's usually played on a grid of squares and this all started when I decided to make a programmatic solver for that kind of puzzle. Then I realized that with some minor changes the solver could generate new puzzle instances. I thought about what kind of puzzle to make and someone suggested a hexagonal grid. Adapting the generator wasn't too hard but then I had to create a way to play this variant. So…

    Life & fun · 2022 · hexapipes.vercel.app

  • PD

    We're building Parsnip to create a "tech tree" of cooking skills that allows anyone to level up on the building blocks of cooking knowledge while tracking their progress over time. It took us a few iterations to figure out the right product; here's the story of our latest pivot: [https:&#x2F;&#x2F;parsnip.substack.com&#x2F;p&#x2F;a-new-hope] The goal is to create a personalized way to learn any recipe on the Internet, then use this as a springboard to help home cooks of all levels solve the problem of repeated meal planning in a 10x better way:…

    Work · 2022 · parsnip.ai