Apache Pig Reviews & Insights

Score8.4 out of 10

22 Reviews and Ratings

Community insights

TrustRadius Insights for Apache Pig are summaries of user sentiment data from TrustRadius reviews and, when necessary, third party data sources.

Business Problems Solved

Apache Pig has proven to be an invaluable tool for data engineers working with large datasets in the Apache Hadoop ecosystem. Users have found it to be an excellent high-level scripting language that simplifies the process of working with big data. With Apache Pig, data engineers can easily build pipelines for advanced analysis and machine learning purposes, allowing them to transform and optimize data operations into MapReduce.

One of the key advantages of Apache Pig is its ability to write complex map-reduce or Spark jobs without requiring deep knowledge of Java, Python, or Groovy. This feature has been highly appreciated by users who value the efficiency and simplicity it brings to their work. Additionally, Apache Pig's query language, Pig Latin, provides users with a straightforward way to build data pipelines, eliminating redundant data and supporting user-defined functions UDFs.

The software also gives users control over task execution, which is crucial in maintaining control in a distributed processing system. This control allows users to efficiently handle transportation problems and manage large volumes of data including data streaming from multiple sources and performing joins. Users have utilized Apache Pig to explore and process large datasets in big data analytics projects, performing various operations within a single Java Virtual Machine.

Another key use case for Apache Pig is the generation of aggregate statistics, running refinement and filtering on logs, as well as generating reports for both internal use and customer deliveries. Data science and data engineering teams also utilize Apache Pig for building big data workflows pipelines for ETL and analytics. The software simplifies the creation of these pipelines by providing native language support with Pig Latin, combining features from various database systems like Hive, DBMS, and Spark-SQL.

Overall, Apache Pig offers a versatile solution for handling big data tasks in a simple yet efficient manner. Its user-friendly query language and extensive capabilities make it a valuable tool for data engineers working in the Apache Hadoop ecosystem.

Apache Pig Reviews

3 Reviews

Engineering

Apache pig - the easier to implement map reducer

Rating: 8 out of 10

Incentivized

October 8, 2018

Use Cases and Deployment Scope

Apache Pig is being used as a map-reduce platform. It is used to handle transportation problems and use large volume of data. It can handle data streaming from multiple sources and join them. This can be used to extract key findings, aggregate results and finally process output which is used for different types of visualizations.

Pros

Fast
Easy to implement
Can process data of almost any size
Easy to learn schema

Cons

It can only work on trivial arithmetic problems.
No or very difficult provision of looping across data
Sequential checks are almost impossible to implement

Likelihood to Recommend

It is well suited when you are aggregating data but really difficult if you want to aggregate based upon line by line. Apache Pig can be picked up in a few days with a few demonstrations. Codes can be written quickly, however, it becomes difficult to take up complicated tasks using it.

Subhadipto Poddar

Research Assistant in Engineering at Iowa State University (Higher Education, 5001-10,000 employees)

Vetted Review

2 years of experience

View profile

Apache Pig - Is it the tool for the job? Maybe, but probably not.

Rating: 7 out of 10

Incentivized

January 18, 2018

Use Cases and Deployment Scope

Apache Pig is one of the distributed processing technologies we are using within the engineering department as a whole and we are currently using it mainly to generate aggregate statistics from logs, run additional refinement and filtering on certain logs, and to generate reports for both internal use and customer deliveries.

Pros

Provides a decent abstraction for Map-Reduce jobs, allowing for a faster result than creating your own MR jobs
Good documentation and resources for learning Pig Latin (the Domain Specific Language of the Apache Pig platform)
Large community allows for easy learning, support, and feature improvements/updates

Cons

May not fit every need and a SQL-like abstraction may be more effective for some tasks (look at Spark-SQL, Hive, or even an actual DBMS)
All Pig jobs are written in a Domain Specific Language so not a lot of transferable knowledge
Writing your own User Defined Functions (UDFS) is a nice feature but can be painful to implement in practice

Likelihood to Recommend

Apache Pig is well suited as part of an ongoing data pipeline where there is already a team of engineers in place that are familiar with the technology since at this point I would consider it relatively depreciated since there are more suitable technologies that have more robust and flexible APIs with the added benefit of being easier to learn and apply. For ad-hoc needs, I would recommend Hive or Spark-SQL if a SQL-esque language makes sense otherwise to make use of Spark + a Notebook technology such as Apache Zeppelin. For production data pipelines I would recommend Apache Spark over Apache Pig for its performance, ease of use, and its libraries.

Verified User

Engineer in Engineering (Computer Software company, 51-200 employees)

Vetted Review

2 years of experience

Apache Pig - a good toolkit to have in your hadoop ETL toolbox

Rating: 8 out of 10

Incentivized

July 21, 2016

Use Cases and Deployment Scope

Yes, it is used by our data science and data engineering orgs. It is being used to build big data workflows (pipelines) for ETL and analytics. It provides easy and better alternatives to writing Java map-reduce code.

Pros

Apache pig DSL provides a better alternative to Java map reduce code and the instruction set is very easy to learn and master.
It has many advanced features built-in such as joins, secondary sort, many optimizations, predicate push-down, etc.
When Hive was not very advanced (extremely slow) few years ago, pig has always been the go to solution. Now with Spark and Hive (after significant updates), the need to learn apache pig may be questionable.

Cons

Improve Spark support and compatibility
Spark and Hive are already being used main-stream, both of them have an instruction set that is easier to learn and master in a matter of days. While apache pig used to be a great alternative to writing java map reduce, Hive after significant updates is now either equal or better than pig.

Likelihood to Recommend

- Custom load, store, filter functionalities are needed and writing Java map reduce code is not an option due susceptible to bugs.
- Chain multiple MR jobs into one pig job.

Verified User

Team Lead in Engineering (Retail company, 10,001+ employees)

Vetted Review

4 years of experience