Hadoop as a Service without vendor lock-in
Use Cases and Deployment Scope
Qubole was used across the company to ease migration strategies from an on-premise Hadoop environment into the cloud. The end place for the data was between Amazon services such as S3 and RDS, but the initial goal was to use multiple clouds, as some parts of the company were using Google's BigQuery.
From what I've seen, Qubole abstracts away the setup, scalability, and installation of many Hadoop services by providing an a la carte offering of big data processing services from query engines of Hive, Spark, and Presto to useful UI tools of the query editors and Zeppelin Notebooks.
Pros
- From a UI perspective, I find Qubole's closest comparison to Cloudera's HUE; it provides a one-stop shop for all data browsing and querying needs.
- Auto scaling groups and auto-terminating clusters provides cost savings for idle resources.
- Qubole fits itself well into the open-source data science market by providing a choice of tools that aren't tied to a specific cloud vendor.
Cons
- Providing an open selection of all cloud provider instance types with no explanation as to their ideal use cases causes too much confusion for new users setting up a new cluster. For example, not everyone knows that Amazon's R or X-series models are memory optimized, while the C and M-series are for general computation.
- I would like to see more ETL tools provided other than DistCP that allow one to move data between Hadoop Filesystems.
- From the cluster administration side, onboarding of new users for large companies seems troublesome, especially when trying to create individual cluster per team within the company. Having the ability to debug and share code/queries between users of other teams / clusters should also be possible.
Likelihood to Recommend
I find Qubole is well suited for getting started analyzing data in the cloud without being locked in to a specific cloud vendor's tooling other than the underlying filesystem. Since the data itself is not isolated to any Qubole cluster, it can be easily be collected back into a cloud-vendor's specific tools for further analysis, therefore I find it complementary to any offerings such as Amazon EMR or Google DataProc.