Hadoop as a Service without vendor lock-in
Pros
- From a UI perspective, I find Qubole's closest comparison to Cloudera's HUE; it provides a one-stop shop for all data browsing and querying needs.
- Auto scaling groups and auto-terminating clusters provides cost savings for idle resources.
- Qubole fits itself well into the open-source data science market by providing a choice of tools that aren't tied to a specific cloud vendor.
Cons
- Providing an open selection of all cloud provider instance types with no explanation as to their ideal use cases causes too much confusion for new users setting up a new cluster. For example, not everyone knows that Amazon's R or X-series models are memory optimized, while the C and M-series are for general computation.
- I would like to see more ETL tools provided other than DistCP that allow one to move data between Hadoop Filesystems.
- From the cluster administration side, onboarding of new users for large companies seems troublesome, especially when trying to create individual cluster per team within the company. Having the ability to debug and share code/queries between users of other teams / clusters should also be possible.
Return on Investment
- We like to say that Qubole has allowed for "data democratization", meaning that each team is responsible for their own set of tooling and use cases rather than being limited by versions established by products such as Hortonworks HDP or Cloudera CDH
- One negative impact is that users have over-provisioned clusters without realizing it, and end up paying for it. When setting up a new cluster, there are too many choices to pick from, and data scientists may not understand the instance types or hardware specs for the datasets they need to operate on.
Alternatives Considered
Cloudera Altus, Cloudera Data Science Workbench, Hortonworks Data Platform and Databricks