Hortonworks Data Platform (HDP) is an open source framework for distributed storage and processing of large, multi-source data sets. HDP modernizes IT infrastructure and keeps data secure—in the cloud or on-premises—while helping to drive new revenue streams, improve customer experience, and control costs.
Hortonworks merged with Cloudera in eary 2019.
I recommend [Hortonworks Data Platform] as Big Data platform in order to start your developments. It's free. It's easy to use. You can install in more server or use a sandbox with you favorite virtualization platform ( vmware or oracle virtualbox). There is also a containerized version.
Manage our data in hdfs is simple; you can interact with server with REST API.
As an open source project collection, it relies strongly on community activity. You still have the option to contract premium consulting or training services.
Altough it is quickly evolving into Data Science tools availability (eg. Tensorflow incorporate in HDP 3), it can be cumbersome from a developer transitioning from a traditional IDE, into the notebook vs. datalake metaphore.
As expected for a big data infranstructure, the resource requirements base line is rather high. This means that if used on premise, you need to think of about 10 machines for a minimal reasonable deploy.
While Apache Hadoop is completely open sourced, Hortonworks Data Platform offers support as well as keeps pace with the open source versions. Also, the HDP open sources its own products, thus giving back to the community. I find using the Hortonworks Data Platform more intuitive than Cloudera or MapR versions.
It provides a convenient way of quickly setting up a big data environment, easily setting up clusters with different configurations. It provides several security architectures that can be used as well. Since it provides a big list of components and packaged together, it is a great tool for companies to get set and utilize it for their use cases.
Since it uses Ambari extensively to install, upgrade and manage software, it is very convenient and easy to support and operationalize the components. Alerting and notifications, ability to create custom alerts give you the capability to add any number of alerts to meet your custom needs. It provides a great way to maintain other software by creating mpacks and the ability to add custom code, and you can add other software to be managed in a centralized tool.
The use and support of popular and useful open source software and the company's contribution to the community makes HDP a very useful tool that enables a quick, secure, easily maintainable suite of components that can help companies meet the needs of the business. What is great is that new components keep getting added based on any new useful tool that comes available, like Druid, and made available as part of the suite of components. That helps businesses keep up with new capabilities as they become available, and use them to solve their problems.
