The Trifacta integration with dbt Core will provide low code/no code options for visual data profiling and orchestration of SQL pipelines.
Trifacta, the Data Engineering Cloud, announced a strategic integration with dbt Core, the popular open-source analytics engineering tool maintained by dbt Labs. The combination of these two technologies creates a unique blend of low-code and code-based tools to support data pipeline development and collaboration among data engineers, data analysts, data scientists, and business analysts.
“The dbt Core open source project is an increasingly popular standard for stitching together SQL-centric data engineering pipelines. By embracing open source and integrating dbt Core repositories into Trifacta’s data preparation platform, we’re continuing to ensure that there are no walls in our Data Engineering Cloud platform.”
“Trifacta is committed to data engineering for any cloud environment,” said Joe Hellerstein, Trifacta Co-Founder and Chief Strategy Officer. “The dbt Core open source project is an increasingly popular standard for stitching together SQL-centric data engineering pipelines. By embracing open source and integrating dbt Core repositories into Trifacta’s data preparation platform, we’re continuing to ensure that there are no walls in our Data Engineering Cloud platform.”
Top iTechnology Cloud News: Lucid Launches Cloud Visualization Solution Lucidscale, Expanding its Visual Collaboration Suite
The initial integration enables dbt CoreTM users to import datasets produced from dbt CoreTM models into Trifacta for visual data profiling during model development. Data professionals can use these profiles to understand data quality, validate overall data shape and breadth, identify outliers, and serve as a guide while defining test conditions for their models. The solution also helps users by configuring a workspace where other stakeholders can be invited to leverage their data warehouse assets to build self-service data pipelines.
With this new integration, dbt Core users can take advantage of one of Trifacta’s most popular features, visual data profiling, to understand key statistics and data patterns generated by their dbt models and dependent models before operationalizing them. Once dbt model datasets are imported and visually profiled, they can be leveraged by users across the organization.
“We are very excited to hear about Trifacta’s integrations with dbt Core,” said Shweta Gandhi, Data Analyst at My Muscle Chef. “Having the ability to profile and orchestrate dbt model outputs in combination with Trifacta Flows that we have developed using their visual interface will give us additional flexibility to address a variety of use cases.”
Top iTechnology Networking News: Opsani Ignite to Analyze Kubernetes-Orchestrated Applications and Identify Optimization Opportunities
“At Brooklyn Data, we build infrastructure to unlock value for a wide range of data practitioners and stakeholders,” said Scott Breitenother, Founder, Brooklyn Data. “Trifacta’s native integration with dbt Core is exciting because it allows companies to build and orchestrate end-to-end data pipelines while providing code and low-code options to empower every data practitioner.”
Looking ahead, Trifacta will continue to innovate so that data professionals can maximize their work across both Trifacta and dbt Core. Future integration points include the ability to orchestrate and monitor dependencies between dbt jobs, and Trifacta flows on cloud data warehouses in a single, integrated environment. Further, users will be able to profile data and orchestrate dbt Core tasks, including support for authoring and auto-generating model files.
Integration with dbt Core is another key pillar of Trifacta’s self-service platform for modern data workers to improve quality and streamline operations as they onboard, assess, and refine raw data.
Top iTechnology IT and DevOps News: Hewlett Packard Enterprise Chooses Clarivate as IP Lifecycle Management Partner
[To share your insights with us, please write to sghosh@martechseries.com]
