Hortonworks Data Platform (HDP) is an open source framework for distributed storage and processing of large, multi-source data sets. HDP modernizes IT infrastructure and keeps data secure—in the cloud or on-premises—while helping to drive new revenue streams, improve customer experience, and control costs.
Hortonworks merged with Cloudera in eary 2019.
N/A
PostgreSQL
Score 8.3 out of 10
N/A
PostgreSQL (alternately Postgres) is a free and open source object-relational database system boasting over 30 years of active development, reliability, feature robustness, and performance. It supports SQL and is designed to support various workloads flexibly.
I recommend [Hortonworks Data Platform] as Big Data platform in order to start your developments. It's free. It's easy to use. You can install in more server or use a sandbox with you favorite virtualization platform ( vmware or oracle virtualbox). There is also a containerized version.
Manage our data in hdfs is simple; you can interact with server with REST API.
PostgreSQL is ideal for handling databases that contain large volumes of information due to its efficiency, speed and above all because of the good management it makes of our resources, it also behaves very well in distributed environments of high demand, if you want a database of stable data and excellent performance PostgreSQL is one of the best.
As an open source project collection, it relies strongly on community activity. You still have the option to contract premium consulting or training services.
Altough it is quickly evolving into Data Science tools availability (eg. Tensorflow incorporate in HDP 3), it can be cumbersome from a developer transitioning from a traditional IDE, into the notebook vs. datalake metaphore.
As expected for a big data infranstructure, the resource requirements base line is rather high. This means that if used on premise, you need to think of about 10 machines for a minimal reasonable deploy.
The performance of PostgreSQL has been enhanced through the years, but always is better to have as much performance as we can.
The replication services could be done directly within the database, and more easily.
The Object Orientation of the Database could be extended, and albeit it manages inheritance of tables, and accepts XML and JSON as primary types, it would be wonderful if one could attach methods more easily to tables (to make them more like classes), and instances (rows for example).
Postgresql is the best tool out there for relational data so I have to give it a high rating when it comes to analytics, data availability and consistency, so on and so forth. SQL is also a relatively consistent language so when it comes to building new tables and loading data in from the OLTP database, there are enough tools where we can perform ETL on a scalable basis.
The data queries are relatively quick for a small to medium sized table. With complex joins, and a wide and deep table however, the performance of the query has room for improvement.
AWS, Heroku, and Digital Ocean all provide Postgres-as-a-service, where you pretty much never need to administrate it yourself but they do it for you. The Postgres community also has developed awesome and reasonably priced extensions, such as Citus DB and CockroachDB in case you need additional support for running it. If you need documentation, Postgres's docs are super thorough and their official forms are active.
The online training is request based. Had there been recorded videos available online for potential users to benefit from, I could have rated it higher. The online documentation however is very helpful. The online documentation PDF is downloadable and allows users to pace their own learning. With examples and code snippets, the documentation is great starting point.
While Apache Hadoop is completely open sourced, Hortonworks Data Platform offers support as well as keeps pace with the open source versions. Also, the HDP open sources its own products, thus giving back to the community. I find using the Hortonworks Data Platform more intuitive than Cloudera or MapR versions.
In this case, Postgres is preferred because it handles large data sets and requires fewer hardware resources than its competitor, MySQL. Compared to PostgreSQL, Microsoft products are excellent, but the installation process for MS SQL is lengthy. PostgreSQL has an advantage over its competitors in that it can adapt or configure third-party programs, applications, or settings.
It provides a convenient way of quickly setting up a big data environment, easily setting up clusters with different configurations. It provides several security architectures that can be used as well. Since it provides a big list of components and packaged together, it is a great tool for companies to get set and utilize it for their use cases.
Since it uses Ambari extensively to install, upgrade and manage software, it is very convenient and easy to support and operationalize the components. Alerting and notifications, ability to create custom alerts give you the capability to add any number of alerts to meet your custom needs. It provides a great way to maintain other software by creating mpacks and the ability to add custom code, and you can add other software to be managed in a centralized tool.
The use and support of popular and useful open source software and the company's contribution to the community makes HDP a very useful tool that enables a quick, secure, easily maintainable suite of components that can help companies meet the needs of the business. What is great is that new components keep getting added based on any new useful tool that comes available, like Druid, and made available as part of the suite of components. That helps businesses keep up with new capabilities as they become available, and use them to solve their problems.
Easy to administer so our DevOps team has only ever used minimal time to setup, tune, and maintain.
Easy to interface with so our Engineering team has only ever used minimal time to query or modify the database. Getting the data is straightforward, what we do with it is the bigger concern.
