Show HN: Lythonic – Compose Python functions into data-flow pipelines
hackernews
|
|
📦 오픈소스
#하드웨어/반도체
원문 출처: hackernews · Genesis Park에서 요약 및 분석
요약
파이썬 라이브러리인 Lythonic은 동기 및 비동기 파이썬 함수를 데이터 흐름 파이프라인으로 구성해 별도의 실행 환경 없이 어디서든 실행할 수 있습니다. ‘>>’ 연산자를 통해 노드를 연결하면 데이터가 노드 간에 흐르는 과정을 투명하게 추적할 수 있으며, 작업의 불투명성이 있는 기존 태스크 스케줄러와 달리 데이터 자체를 추적합니다. 또한 DAG 내부 중첩, 맵-리듀스, SQLite 기반의 출처 추적 및 캐싱 기능을 제공하여 코드를 재사용하고 생산 환경에서 효율적으로 관리할 수 있습니다.
본문
Compose Python logic into data-flow pipelines — sync or async, run anywhere. Write plain Python functions. Wire them with >> . Data flows visibly between nodes — you can see what went in, what came out, what failed. Unlike task schedulers where jobs are opaque units, lythonic tracks the data itself. uv add lythonic from lythonic.compose.namespace import Dag def fetch(url: str) -> dict: return {"source": url, "values": [1, 2, 3]} def double(data: dict) -> dict: return {**data, "values": [v * 2 for v in data["values"]]} dag = Dag() dag.node(fetch) >> dag.node(double) # Run it — sync or async, doesn't matter import asyncio result = asyncio.run(dag(url="https://example.com")) print(result.outputs) # {"double": {"source": "...", "values": [2, 4, 6]}} Data flow, not task flow. Each node receives typed data from upstream and passes results downstream. The DAG runner wires inputs to outputs by type — fan-out, fan-in, and map-reduce built in. Provenance tracking records what data flowed through each edge. Compose freely. DAGs nest inside DAGs. dag.node(sub_dag) runs a sub-DAG as a single step. dag.map(sub_dag) runs it on each element of a collection, concurrently. Build small, reuse everywhere. Run transparently. await dag() for a quick test. DagRunner with provenance for production. lyth start for a long-running engine with cron-triggered pipelines. Same code, different execution context. Sync and async — mixed freely. Write sync functions, async functions, or both in the same DAG. Sync nodes run in a thread executor automatically. - DAG composition — >> wiring, callable DAGs, MapNode, CallNode @dag_factory — define reusable DAG templates as decorated functions- Triggers — cron-scheduled or push-triggered execution via TriggerManager - Provenance — SQLite-backed tracking of runs, node executions, edge traversals - Caching — per-callable SQLite cache with probabilistic TTL refresh lyth CLI —start ,stop ,run ,fire ,status commands- State — Pydantic-based SQLite ORM with schema management and multi-tenant support Full documentation at walnutgeek.github.io/lythonic.
Genesis Park 편집팀이 AI를 활용하여 작성한 분석입니다. 원문은 출처 링크를 통해 확인할 수 있습니다.
공유