#1 most frequent
United States of America
51.4%8,077 installations of 15,709
What is dbt?
dbt is an SQL development environment, developed by Fishtown Analytics, now known as dbt Labs. The vendor states that with dbt, analysts take ownership of the entire analytics engineering workflow, from writing data transformation code to deployment and documentation. dbt Core is distributed under the Apache 2.0 license, and paid Teams and Enterprise editions are available.
Categories & Use Cases
Product Demos
Top Performing Features
Simple transformations
Simple data transformations are calculations, data type conversions, aggregations and search and replace operations
Category average: 8.1
Collaboration
Collaboration is enabled by a shared repository of project information and metadata
Category average: 7.3
Data model creation
Ability to create and maintain data models using a graphical tool to define relationships between data
Category average: 8.4
Areas for Improvement
Business rules and workflow
Ability to define and manage business rules and workflows
Category average: 8.3
Metadata management
Automated discovery of metadata with ability to synchronize and share metadata with other tools like Master Data Management
Category average: 7.9
Testing and debugging
Tool to debug and tune for optimal performance
Category average: 7.1
Who Buys & Uses dbt
#2 most frequent
United Kingdom of Great Britain and Northern Ireland
6.8%1,075 installations of 15,709
#3 most frequent
France
4.2%667 installations of 15,709
dbt - a great data transformation tool in data pipelines
Use Cases and Deployment Scope
At [...], dbt (Data Build Tool) is used for data transformation in the ELT processes. As [...] is a data rich company, there are lot of instances where the data needs to be transformed after it is loaded into the data warehouse and dbt handles this perfectly. dbt helps our company to maintain data quality with its transformation capabilities using the SQL queries.
Pros
- dbt supports version control through GIT, this allows teams to collaborate and track the data transformation logic.
- dbt allows us to build data models which helps to break complex transformation logic into simple and smaller logic.
- dbt is completely based on SQL which allows data analyst and data engineers to build the transformation logic.
- dbt can be easily integrated with snowflake.
Cons
- dbt can improve their debugging and error messaging.
- dbt does not support python based transformation which are needed in advanced cases like machine learning.
- dbt should provide the feature of query cost estimation and usage reports to reduce high compute cost.
Return on Investment
- With dbt the data transformation is now faster which ultimately improves the time to insights.
- dbt has reduced the cost compared to other traditional ETL tools.
- Data quality and reliability of [...] has improved with dbt.
Usability
Other Software Used
Atlassian Confluence, Numerator TruView, Apache Airflow
dbt GREAT for Most Data Orgs
Use Cases and Deployment Scope
We used dbt to transform source data into data tables, push these data tables into our data warehouse, establish sources of truth for data, track data lineage / dependencies / downstream impacts, and as a source of truth for business logic (metric definitions, what data can be used for what, and so forth).
Pros
- Easy to create data tables (analysts can do what used to require engineer)
- Documentation & lineage is built in
- It can run tests to make sure data is being transformed correctly
Cons
- You have to use a separate tool for job orchestration
- You need a separate tool for ingestion
- Table optimization work is manual (no automations there)
Return on Investment
- Analytics team members can now do what required more expensive engineers
- Can save on other tools for lineage, unit tests, etc.
Usability
Alternatives Considered
Snowflake and Google Cloud Dataform
Other Software Used
Snowflake, Looker, Apache Airflow
Manage your data transformations with engineering practices.
Use Cases and Deployment Scope
We use dbt to manage all the data transformation logic in our data warehouse, all the way from raw data to modeled data ready for analysis. This allows us to harmonize and clean our data and create models combining data from multiple sources. Our scope contains billing and payment data, CRM data, marketing, and lead pipeline data, etc.
Pros
- Automation
- Version control.
- Automated generation of lineage graphs.
Cons
- Tried hard, but cannot think of anything.
Return on Investment
- Ability to analyze marketing campaign effectiveness.
- Ability to calculate a customer health score used to reduce churn.
- Ability to created trusted financial reporting, e.g. billings and recurring revenue.
Usability
Alternatives Considered
Matillion
Other Software Used
Snowflake, Fivetran, Tableau Desktop, Tableau Cloud, Tableau Prep
dbt - an excellent transformation tool for the masses
Use Cases and Deployment Scope
We use dbt to transform source data into meaningful report data, so it can be easily consumed in dashboards, allowing our management insights and the ability to steer the company. We use Fivetran and other tools to land the data in our Snowflake data warehouse, and then dbt to transform and utilize that data.
Pros
- Text based integration with github - it's very easy to see changes to code over time.
- Leverages SQL which makes it a fast learning curve for most developers.
- Removes complexity of deployment to multiple environments.
- Adds powerful templating, making dynamic sql easy.
- Data lineage and documentation.
- Easy to add automated testing for data quality.
- Easy to switch output between tables and views by setting a flag.
- Excellent documentation, slack app, training, and support.
- Packages (libraries) exist with helpful code readily available.
- Failsafe - dbt core is open source so our investment in code is sound even if they hike the prices.
Cons
- Field-level lineage (currently at table level)
- Documentation inheritance - if a field is documented the downstream field of the same name could inherit the doc info
- Adding python model support (in beta now)
Most Important Features
- SQL-based (can hire and scale quickly)
- Github integration (can see changes)
- dbt core is open source - solid investment in business logic in case it gets pricey$$$
- Powerful - ability to do dynamic work easily (enhances SQL)
- Data lineage visibility
Return on Investment
- In 3 months we re-wrote the data warehouse (15-20 sources) in dbt with 3 developers.
- We are using it continually for the past year with no issues.
- Sorry, I don't have ROI numbers but the impact was huge.
Alternatives Considered
SnapLogic
dbt - bang for your buck for transformation in ETL stack
Use Cases and Deployment Scope
I use dbt on the T part of ETL in my data tech stack. It is amazing and also does the change control quite well. So, to summarize - ingest, do a multitude of transforms, spit out to data mart. Allows various business logic applications to data to happen simultaneously and well tracked for the data marts.
Pros
- Transformation
- Change Control
- Organisation
- Flow build
Cons
- Learning Curve is steep
- Better documentation
- Better YouTube tutorials
Most Important Features
- Change Control
- Transformation
- Developer vibe for data analysts
Return on Investment
- Quicker insights
- Quicker data
- Quicker solves
Alternatives Considered
Hevo Data and Fivetran
