AWS Glue is a managed extract, transform, and load (ETL) service designed to make it easy for customers to prepare and load data for analytics. With it, users can create and run an ETL job in the AWS Management Console. Users point AWS Glue to data stored on AWS, and AWS Glue discovers data and stores the associated metadata (e.g. table definition and schema) in the AWS Glue Data Catalog. Once cataloged, data is immediately searchable, queryable, and available for ETL.
$0.44
billed per second, 1 minute minimum
AWS Lambda
Score 8.7 out of 10
N/A
AWS Lambda is a serverless computing platform that lets users run code without provisioning or managing servers. With Lambda, users can run code for virtually any type of app or backend service—all with zero administration. It takes of requirements to run and scale code with high availability.
$NaN
Per 1 ms
Pricing
AWS Glue
AWS Lambda
Editions & Modules
per DPU-Hour
$0.44
billed per second, 1 minute minimum
128 MB
$0.0000000021
Per 1 ms
1024 MB
$0.0000000167
Per 1 ms
10240 MB
$0.0000001667
Per 1 ms
Offerings
Pricing Offerings
AWS Glue
AWS Lambda
Free Trial
No
No
Free/Freemium Version
No
No
Premium Consulting/Integration Services
No
No
Entry-level Setup Fee
No setup fee
No setup fee
Additional Details
—
—
More Pricing Information
Community Pulse
AWS Glue
AWS Lambda
Features
AWS Glue
AWS Lambda
Access Control and Security
Comparison of Access Control and Security features of Product A and Product B
When the data which requires ETL has different formats, schema, and volume, this service suits them best. So, when the volume is not consistent (typical use-case of healthcare and online shopping), AWS Glue can be the prime choice. When the data is available in both batch and streaming mode, the developer needs to generate a separate codebase. This increases the source code management efforts. So, prefer to go with Glue when the nature of the data is the same (either batched or streamed).
Scenarios where AWS Lambda is well suited: 1. When we need to run a periodic task few times in a day or every hour, we may deploy it on AWS Lambda so it would not increase load on our server which is handling client requests and at the same time we don't have to pay for AWS Lambda when it is not running. So, overall we only pay for few function invocations. 2. When some compute intensive processing is to be done but the number of requests per unit of time fluctuates. For example, we had deployed an AWS Lambda for processing images into different sizes and storing them on AWS S3 once user uploads them. Now, this is something that may happen few times every hour on a particular day or may not happen even once on other days. To handle this kind of tasks AWS Lambda is a better choice as we don't have to pay for the idle time of the server and also we don't have to worry about scaling when the load is high. Scenarios where AWS Lambda is not appropriate to use: 1. When we expect a large request volume continuously on the server. 2. When we don't want latency even in case of concurrent requests.
After data cleansing, the team also implemented the best practices for using AWS platform services as a Data Lake, such as job bookmarking for AWS Glue jobs, proper delimiter for the AWS Glue crawlers, partitioning in AWS S3, and transformation to parquet file for compression and faster querying time in Amazon Athena.
Data modernization through combining data from multiple sources into a functioning datasets, rebuilding DW, and resctructuring data sources.
Aims to lessen customer complaints, eliminate manual data extraction requests via SR from different data sources, and Increase accuracy, consistency and speed up reconciliation process.
AWS Lambda is a welcoming platform, supporting several languages, including Java, Go, PowerShell, Node.js, C#, Python, and Ruby. And if you need to deploy a Lambda function in another language, AWS offers a Runtime API for integration.
We really appreciate how AWS Lambda is always-on for our functions, with only a brief "cold-start" waiting period the first time a function is called after being dormant.
In addition to only generating costs when it's actually being used, AWS Lambda really puts the "serverless" in serverless architecture, offering turnkey scaleability and high availability for our code with zero effort on our part.
The UI and Developer experience is not so great. IF you use an abstraction like Serverless Application Model (SAM), things get pretty easy, but it's still AWS UI/DX you're working with after that (which is to say, not their strength).
Documentation is always a mixed bag. Sometimes it's just easier to google your specific problem and see how others have solved it. This can be much faster than trying to find an example that may or may not be there in the documentation (which oftentimes has multiple versions and revisions).
It is very easy to get started with AWS Lambda and create your first function. The user interface makes it easy to add AWS services to be inputs or outputs to the function, meaning it can be configured in many different ways for different needs. This makes it ideal for various scenarios in AWS.
Amazon responds in good time once the ticket has been generated but needs to generate tickets frequent because very few sample codes are available, and it's not cover all the scenarios.
As this is a product where a great part of errors can be at the source code level, AWS support team doesn't dive that further. I mean they don't evaluate problems more complex related to your code, [which] is totally understandable, but this make[s] debug process more tough and painful.
The cataloging of data objects is the best in the case of AWS Glue. We use AWS Glue in all of our data pipelines to sync external and internal data sources and to automatically produce SQL-based ETL based on AWS Glue catalog objects. Integration with Amazon products is the other advantage.
It's fine, it works as the others would have, except EC2. We are migrating back to EC2 for dedicated compute because we have scaled to a point where we have consistent traffic. The tradeoff of maintaining infrastructure in-house outweighs the benefits of moving quickly through our roadmap.
We have simplified log fiie ingestion using Lambda functions. The return has been less time worrying about getting logs from source to ingestion; one the process is in place the team is nearly 100% hands off.
We have begun taking a more API focused approach by using API Gateway as the interface to business processes and Lambda as the back end compute. Moving away from server based back ends places us on a path to reducing overall spend in compute costs.
Lambda functions allow us to easily interface with third party services through APIs. This simplifies access management since the function can be granted permissions and access to the function can be gated with API keys and other authentication methods.
