dbt vs. HCL DataConnect

dbt (Data Build Tool) is best suited for doing the data transformation. dbt is just a transformation tool and it is not suitable for building a data pipeline which requires extraction of data and loading. dbt is well suited for SQL based transformation logic and it is less appropriate when transformation logic requires python.

Incentivized

I wouldn't recommend Data Integrator over SSIS, but it may be a better tool than other ETL offerings. Having a background in the Microsoft Stack, I am hesitant to recommend any other products especially in a Microsoft-centric company. If this was the only option to use, I think it does a good job at integrating systems after taking time to learn the approach. "Maps" in Data Integrator is definitely different than "process flow" philosophy in SSIS.

Incentivized

• user experience makes it easy to work with SQL and version control
• customer success team and the dbt (data build tool) community help establish best practices
• thorough and clear documentation

• Map multiple source and destination tables - what would be done in many steps in SSIS can be done in one or two "maps" in Data Integrator.
• Platform independent, cloud based, no client software.
• Conversion of data types, write flat files - this helps us with integrating systems.
• Fairly straightforward to use if the user has ETL experience.

Incentivized

• Slow load times of the dbt cloud environment (they're working on it via a new UI though)
• More out-of-the-box solutions for managing procedures, functions, etc would be nice to have, but honestly, it's pretty easy to figure out how to adapt dbt macros

Incentivized

• Integration Manager needs better scheduling improvements. Like dependencies, and not running a process again if the prior run is still running.
• New connector development does not seem a priority.

Incentivized

dbt is very easy to use. Basically if you can write SQL, you will be able to use dbt to get what you need done. Of course more advanced users with more technical skills can do more things.

Incentivized

Matillion is graphical versus dbt, which is SQL code-based (that, of course, is a matter of personal preference and not an objective advantage). The integrated testing, documentation generation, lineage, etc., were additional criteria that led us to choose dbt.

Incentivized

As I stated earlier, DataConnect makes a head-to-head run against the others. It has advantages and disadvantages compared to the competitors. The choice really comes down to pricing and specific requirement needs for the project you are planning. Questions such as On-premise or Cloud, Security, and Specific Connectors available will drive your decision.

Incentivized

• In 3 months we re-wrote the data warehouse (15-20 sources) in dbt with 3 developers.
• We are using it continually for the past year with no issues.
• Sorry, I don't have ROI numbers but the impact was huge.

Incentivized

• Our customer likes the consistent reliable integrations we have built so we [got] a definite return on investment.
• Problems are fairly easy to handle and rework.

Incentivized