IData Corporation released an open-source no-code Pipeline for AWS that can save companies hundreds of hours of development time. The Pipeline can easily transform and load data files into a data lake or into a data warehouse such as Snowflake or Redshift, all with no coding.
The Pipeline can perform data quality checks, data deduplication, transformations and load data directly into its destination (S3, Snowflake, or Redshift). Notifications are automatically sent when data lands, allowing downstream users or systems to process it immediately.
CIO INFLUENCE: Apprentice Now Joins Amazon Web Services Training Partner Program to Deliver AWS Cloud Skills Training
“By working with the open-source Pipeline, companies can move their expensive programming talent to focus on AI and machine learning tasks, instead of spending months building a data pipeline first”, explains Todd Fearn, CEO of IData. “Our customers can have this Pipeline up and running in their own AWS account in just a couple of hours, streaming data downstream to internal and external systems”.
Additionally, the Pipeline employs Apache Iceberg technology. Apache Iceberg is an open-source project that enables building a Lakehouse architecture on top of data lakes. Apache Iceberg provides ACID transactions, scalable metadata handling, and unifies streaming and batch data processing on top of existing data lakes, such as S3.
CIO INFLUENCE: PlainID Launches The PlainID Technology Network to Enable Identity Aware Security for Advanced Access Control
[To share your insights with us, please write to sghosh@martechseries.com]
