AWS Glue vs. Microsoft Fabric

When the data which requires ETL has different formats, schema, and volume, this service suits them best. So, when the volume is not consistent (typical use-case of healthcare and online shopping), AWS Glue can be the prime choice. When the data is available in both batch and streaming mode, the developer needs to generate a separate codebase. This increases the source code management efforts. So, prefer to go with Glue when the nature of the data is the same (either batched or streamed).

Incentivized

I would highly recommend Microsoft Fabric, especially for medium to large enterprises aiming to build a robust, scalable, and secure data analytics platform. It effectively unifies various data workloads, streamlining data integration, engineering, and particularly enhancing our ability to create and share reliable Power BI dashboards. The deep integration with Azure AD for features like Row-Level Security is a significant advantage for data governance.

Incentivized

• After data cleansing, the team also implemented the best practices for using AWS platform services as a Data Lake, such as job bookmarking for AWS Glue jobs, proper delimiter for the AWS Glue crawlers, partitioning in AWS S3, and transformation to parquet file for compression and faster querying time in Amazon Athena.
• Data modernization through combining data from multiple sources into a functioning datasets, rebuilding DW, and resctructuring data sources.
• Aims to lessen customer complaints, eliminate manual data extraction requests via SR from different data sources, and Increase accuracy, consistency and speed up reconciliation process.

Incentivized

• Access Control / Security & Governance
• Collaborative Work up to Visualization
• Data Ingestion

Incentivized

• It’s integration with other cloud vendors is bit difficult
• If it can support non SQL based databases as well, it would be powerful.
• Real time data synchronisation in data source is missing

Incentivized

• Clarity and Documentation on Power BI Integration Changes and Best Practices
• More Flexible and Advanced Scheduled Report Delivery Options
• Enhanced Data Export Control Based on Sensitivity Labels

Incentivized

I personally found it very usable for a data engineer's day job, particularly for performing ETL and managing the data pipelines.

Incentivized

I've rated Microsoft Fabric's overall usability as a 4, primarily due to its extensive and multifaceted feature set, which can make it challenging to navigate and determine the optimal functionality for a given task.While the breadth of capabilities is a core strength for large enterprises, it often leads to a sense of being "lost" or overwhelmed for teams like ours that do not have highly formalized roles or dedicated specialists for each Fabric "experience" (e.g., Data Engineering, Data Warehousing, Data Science).

Incentivized

Amazon responds in good time once the ticket has been generated but needs to generate tickets frequent because very few sample codes are available, and it's not cover all the scenarios.

Incentivized

The cataloging of data objects is the best in the case of AWS Glue. We use AWS Glue in all of our data pipelines to sync external and internal data sources and to automatically produce SQL-based ETL based on AWS Glue catalog objects. Integration with Amazon products is the other advantage.

Incentivized

Microsoft Fabric integrates data ingestion, engineering, warehousing, and Power BI visualization into one cohesive environment. This "one-stop shop" approach dramatically reduces complexity, minimizes operational overhead, and eliminates the need to integrate disparate tools and manage data across multiple systems. It provides superior scalability for large datasets, supports open data formats, and offers a much broader suite of data engineering and data science capabilities.In essence, Fabric's integrated ecosystem and streamlined operational management were key differentiators, providing a more cohesive, scalable, and efficient solution for our evolving data strategy than combining specialized tools.

Incentivized

• Positive Impact :- after ETL we can able to do some kind of automation
• Negative :- At some point of time it can hamper the cost but not really

Incentivized

• Transformation from Monthly Reports to Daily Insights, Accelerating Decision-Making
• Streamlined Reporting: Enhanced Data Consistency and 30% Reduction in Reports Provided
• Empowered Self-Service Analytics Through Elevated Employee Data Literacy

Incentivized