Azure Synapse Analytics is described as the former Azure SQL Data Warehouse, evolved, and as a limitless analytics service that brings together enterprise data warehousing and Big Data analytics. It gives users the freedom to query data using either serverless or provisioned resources, at scale. Azure Synapse brings these two worlds together with a unified experience to ingest, prepare, manage, and serve data for immediate BI and machine learning needs.
$4,700
per month 5,000 Synapse Commit Units (SCUs)
Databricks Data Intelligence Platform
Score 8.5 out of 10
N/A
Databricks in San Francisco offers the Databricks Lakehouse Platform (formerly the Unified Analytics Platform), a data science platform and Apache Spark cluster manager. The Databricks Unified Data Service aims to provide a reliable and scalable platform for data pipelines, data lakes, and data platforms. Users can manage full data journey, to ingest, process, store, and expose data throughout an organization. Its Data Science Workspace is a collaborative environment for practitioners to run…
In terms of a well-suited scenario - the Azure Synapse can be used to capture data from multiple sources (especially from onPrem sources apart from Dataverse) and update the transformed data based on the given conditions (eg: refresh data based on the specified date/time ranges). Also, the transformed data can simply be transferred to Azure Data Lake for further processing by utilizing other analytics tools such as PowerBI.
If you need a managed big data megastore, which has native integration with highly optimized Apache Spark Engine and native integration with MLflow, go for Databricks Lakehouse Platform. The Databricks Lakehouse Platform is a breeze to use and analytics capabilities are supported out of the box. You will find it a bit difficult to manage code in notebooks but you will get used to it soon.
Keeping things "complicated, but simple"; [heterogeneous] data formats seen as just SQL tables to business experts used to use Power BI, Excel, and any other traditional SQL-oriented BI tools
Integration options using "Synapse pipelines", the application of ADFs
The greatly integrated solution of independent things (Spark MPP cluster, MPP SQL Servers, ADFs) - all sitting under one roof. Great job!
Integration with super-fast, globally replicated data. I really appreciate the integration of NoSQL databases (namely Core API and Mongo API under Cosmos DB) with purely batch-processed BI data
There is databricks community, which is a free version. It is available for beginners to have an easy start with a big data platform. It does not have every feature of the full version but is still adequate for extremely new coders.
There are many resourceful training elements that are available to developers, data scientists, data engineers and other IT professionals to learn Apache Spark.
With Azure, it's always the same issue, too many moving parts doing similar things with no specialisation. ADF, Fabric Data Factory and Synapse pipeline serve the same purpose. Same goes for Fabric Warehouse and Synapse SQL pools.
Could do better with serverless workloads considering the competition from databricks and its own fabric warehouse
Synapse pipelines is a replica of Azure Data Factory with no tight integration with Synapse and to a surprise, with missing features from ADF. Integration of warehouse can be improved with in environment ETl tools
Connect my local code in Visual code to my Databricks Lakehouse Platform cluster so I can run the code on the cluster. The old databricks-connect approach has many bugs and is hard to set up. The new Databricks Lakehouse Platform extension on Visual Code, doesn't allow the developers to debug their code line by line (only we can run the code).
Maybe have a specific Databricks Lakehouse Platform IDE that can be used by Databricks Lakehouse Platform users to develop locally.
Visualization in MLFLOW experiment can be enhanced
The data warehouse portion is very much like old style on-prem SQL server, so most SQL skills one has mastered carry over easily. Azure Data Factory has an easy drag and drop system which allows quick building of pipelines with minimal coding. The Spark portion is the only really complex portion, but if there's an in-house python expert, then the Spark portion is also quiet useable.
Because it is an amazing platform for designing experiments and delivering a deep dive analysis that requires execution of highly complex queries, as well as it allows to share the information and insights across the company with their shared workspaces, while keeping it secured.
in terms of graph generation and interaction it could improve their UI and UX
Microsoft does its best to support Synapse. More and more articles are being added to the documentation, providing more useful information on best utilizing its features. The examples provided work well for basic knowledge, but more complex examples should be added to further assist in discovering the vast abilities that the system has.
One of the best customer and technology support that I have ever experienced in my career. You pay for what you get and you get the Rolls Royce. It reminds me of the customer support of SAS in the 2000s when the tools were reaching some limits and their engineer wanted to know more about what we were doing, long before "data science" was even a name. Databricks truly embraces the partnership with their customer and help them on any given challenge.
They're all part of the Microsoft Azure family, so they are not exactly competitors. They overlap in functionality, but they're targeted at different levels of customers. Azure Data Factory is an excellent stand-alone PaaS (included in Synapse Analytics) for writing, scheduling, and monitoring pipelines. Azure SQL Database (and all the Azure SQL family) is excellent for traditional, SQL-based data warehouses, especially if you're migrating from on-premises. Combined with Azure Data Factory (that can run SSIS packages), it's a perfect solution for a simple path to the cloud. Azure Databricks is effectively the only internal "competitor" to Synapse Analytics but targeted more to a "platform-agnostic" audience. On the other hand, Synapse is more of a proprietary mix of products that are more tightly related to Microsoft technologies.
Databricks is a true all-in-one platform, and at the time of implementation, it had more features available to us, making it a clear choice over Snowflake. Moving our workloads from local computing to the servers in Databricks gave our start-up staff a great quality of life boost.
It definitely has a positive impact on ROI. We are able to use it to generate MORE revenue through predictive analytics and pricing optimization.
Because of the SQL Data Warehouse design, we're able to set up some self service reporting tools which allow our users to generate reports ad hoc instead of having a full time employee creating these by hand.
Having visibility into the data is very useful for management to make good business decisions.
