Adding one command to code makes it easy to move data across almost any source and destination
Airbyte, creators of the leading open-source data movement infrastructure, announced availability of PyAirbyte, an open-source Python library, that makes it easy to move data across API sources and destinations by enabling Airbyte resources to be created and managed using code, rather than the user interface (UI). Python users gain access to Airbyte’s more than 250 data connectors rather than having to build and maintain those themselves. Airbyte is the first to provide Python users this capability with availability of over 250 connectors.
Recommended: Top 10 CIO Influence News of Jan’24
“A majority of existing data pipelines are written in Python today”
“A majority of existing data pipelines are written in Python today,” said Michel Tricot, co-founder and CEO, Airbyte. “We’ve seen growing demand for this capability, which provides a natural fit into the coding workflow and only requires running one command.”
Python is a popular programming language for data engineers because of its simplicity, versatility, scalability, and the wide range of libraries and frameworks for data manipulation, exploration, and visualization. Python easily integrates with other tools commonly used in data analysis and data science, such as SQL databases, Hadoop, and Spark.
PyAirbyte is an addition to the Airbyte API and Terraform Provider, which enable Airbyte resources to be managed programmatically – streamlining workflows and integrating Airbyte configurations with existing data infrastructure. Other deployment models include Airbyte Open Source, Airbyte Self-Managed, Airbyte Cloud, and Powered by Airbyte.
Recommended: CIO Influence Interview with Neil Kole, Chief Information Officer at Boomi
Key features and benefits of PyAirbyte include the following.
- Programmatically manage data movement: Airbyte resources can now be created and managed through code, providing a more efficient and scalable approach for managing data pipelines.
- Integrate Airbyte configuration within existing data infrastructure: PyAirbyte enables users to merge changes to Airbyte configurations, enhancing workflow efficiency and simplifying the deployment process.
- Automate Airbyte configuration management: Leveraging automation minimizes the likelihood of errors and improves efficiency, saving engineering time.
- Compatibility: Works with open-source orchestration tools, like Airflow and Dagster, as well as LangChain for creation of applications using large language models (LLMs).
There is a Getting Started guide available and go here to register for the webinar on March 14 to learn more.
Airbyte makes moving data easy and affordable across almost any source and destination, helping enterprises provide their users with access to the right data for analysis and decision-making. Airbyte has the largest data engineering contributor community – with more than 800 contributors – and the best tooling to build and maintain connectors.
Recommended: Neal Singh, Industry Veteran from Microsoft, GE, and Icertis, Joins Zenoti as President & COO
[To share your insights with us as part of editorial or sponsored content, please write to sghosh@martechseries.com]
