Visions – User defined data type systems
Visions is a python library for working with user defined data type systems. Out of the box, it provides type inference and automated data cleaning of sequence data with backend specific implementations for pandas, spark, python, and numpy. We often use it as a first pass cleaning step when working with tabular data and to simplify the backend logic of both pandas-profiling[1] and our tabular data compression library compressio[2]. Because data types are user defined, we can build user customizable libraries based around types without adding code complexity. In the case of compressio that…
What it does
In the maker’s words, at launch
Visions is a python library for working with user defined data type systems. Out of the box, it provides type inference and automated data cleaning of sequence data with backend specific implementations for pandas, spark, python, and numpy. We often use it as a first pass cleaning step when working with tabular data and to simplify the backend logic of both pandas-profiling[1] and our tabular data compression library compressio[2]. Because data types are user defined, we can build user customizable libraries based around types without adding code complexity. In the case of compressio that means offering users the ability to use any compression algorithm they want by simply passing a dictionary mapping `{type: compression algorithm}` or defining new compression algorithms for otherwise unsupported data types like shapely geometries, images, etc. We hope you like it and would really appreciate feedback about how to make the library more useful and easier to use. P.S. If you're interested in learning more about the project, the original paper is available on JOSS[3] you can also check out our Numpy Global 2020 talk[4] 1. https://github.com/pandas-profiling/pandas-profiling 2. https://github.com/dylan-profiler/compressio 3. https://joss.theoj.org/papers/10.21105/joss.02145 4. https://www.youtube.com/watch?v=h2w99XIKizY
Does the same job
all alternatives →
- APA Python with Hindley-Milner-like type annotations, compiling to C2015 · github.com · ▲134
- BABamboolib – A GUI for Pandas (Python Data Science)2019 · bamboolib.com · ▲119
- APA Python container for dataclasses with multi-indexing and vector opps2020 · github.com · ▲122
More dev tools this month
the category →
Open-source GTM skills for technical founders
Dev tools · 29d ago · gtmcofounder.com
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 1d ago · opentrailpaper.com
Launched alongside, February 2022
the whole month →
- S2
Life & fun · 2022 · sha256algorithm.com
- E1EdgeDB 1.0▲947
Life & fun · 2022 · edgedb.com
