AWS Glue vs. Dataiku

When the data which requires ETL has different formats, schema, and volume, this service suits them best. So, when the volume is not consistent (typical use-case of healthcare and online shopping), AWS Glue can be the prime choice. When the data is available in both batch and streaming mode, the developer needs to generate a separate codebase. This increases the source code management efforts. So, prefer to go with Glue when the nature of the data is the same (either batched or streamed).

Incentivized

I would recommend it because it's an amazing tool for different levels of users. From Business Analysts to Data Scientists to Managers, various employees can make use of this tool to make data-driven decisions. I'm not sure about where it would be less appropriate as I'm using it as Data Scientist and so far it pretty much caters to my need.

Incentivized

• After data cleansing, the team also implemented the best practices for using AWS platform services as a Data Lake, such as job bookmarking for AWS Glue jobs, proper delimiter for the AWS Glue crawlers, partitioning in AWS S3, and transformation to parquet file for compression and faster querying time in Amazon Athena.
• Data modernization through combining data from multiple sources into a functioning datasets, rebuilding DW, and resctructuring data sources.
• Aims to lessen customer complaints, eliminate manual data extraction requests via SR from different data sources, and Increase accuracy, consistency and speed up reconciliation process.

Incentivized

• Very intuitive and easy to use UI, making a lot of types of users can collaborate with each other easily, by visualizing the same workflow.
• Many building blocks can be reused immediately, avoid a lot of non-standard boiler plate implementation.
• Data pre-analysis and feature engineering assistance increase the productivity as well as the efficiency of data scientists.
• Many data connectors support wide range of data storage, from SQL, TeraData, Hadoop Hive, etc.
• Support from research till final MaaS solution deployment.

Incentivized

• It’s integration with other cloud vendors is bit difficult
• If it can support non SQL based databases as well, it would be powerful.
• Real time data synchronisation in data source is missing

Incentivized

• Its community support is very limited at the moment
• Complex to integrate with automation tools such as Blue Prism

Incentivized

I personally found it very usable for a data engineer's day job, particularly for performing ETL and managing the data pipelines.

Incentivized

As I have described earlier, the intuitiveness of this tool makes it great as well as the variety of users that can use this tool. Also, the plugins available in their repository provide solutions to various data science problems.

Incentivized

Amazon responds in good time once the ticket has been generated but needs to generate tickets frequent because very few sample codes are available, and it's not cover all the scenarios.

Incentivized

The open source user community is friendly, helpful, and responsive, at times even outdoing commercial software vendors. Documentation is also top notch, and usually resolves issues without the need for human interactions. Great product design, with a focus on user experience, also makes platform use intuitive, thus reducing the need for explicit support.

Incentivized

The cataloging of data objects is the best in the case of AWS Glue. We use AWS Glue in all of our data pipelines to sync external and internal data sources and to automatically produce SQL-based ETL based on AWS Glue catalog objects. Integration with Amazon products is the other advantage.

Incentivized

Strictly for Data Science operations, Anaconda can be considered as a subset of Dataiku DSS. While Anaconda supports Python and R programming languages, Dataiku also provides this facility, but also provides GUI to creates models with just a click of a button. This provides the flexibility to users who do not wish to alter the model hyperparameters in greater depths. Writing codes to extract meaningful data is time consuming compared to Dataiku's ability to perform feature engineering and data transformation through click of a button.

Incentivized

• Positive Impact :- after ETL we can able to do some kind of automation
• Negative :- At some point of time it can hamper the cost but not really

Incentivized

• Given its open source status, only cost is the learning curve, which is minimal compared to time savings for data exploration.
• Platform also ease tracking of data processing workflow, unlike Excel.
• Build-in data visualizations covers many use cases with minimal customization; time saver.

Incentivized