AWS Device Farm is a mobile application performance testing application that provides real-time automated testing and reproduction of issues, simulating and testing issues that may occur on a variety of platforms (e.g. iPhone or Samsung mobile device, or multiple operations systems, etc).
$0.01
per instance minute
Firebase
Score 8.4 out of 10
N/A
Google offers the Firebase suite of application development tools, available free or at cost for higher degree of usages, priced flexibly accorded to features needed. The suite includes A/B testing and Crashlytics, Cloud Messaging (FCM) and in-app messaging, cloud storage and NoSQL storage (Cloud Firestore and Firestore Realtime Database), and other features supporting developers with flexible mobile application development.
As long as you have the right scenarios in mind where you can use AWS Device Farm, you will be very happy. It's a cloud platform for testing automation support, where you can use real devices in multiple configs to validate your Android or iOS use cases. It does easily integrate with your CI pipeline, but reporting and UI are not perfect. A pain point is also the JUnit4 implementation, which could be more mature.
Firebase should be your first choice if your platform is mobile first. Firebase's mobile platform support for client-side applications is second to none, and I cannot think of a comparable cross-platform toolkit. Firebase also integrates well with your server-side solution, meaning that you can plug Firebase into your existing app architecture with minimal effort.
Firebase lags behind on the desktop, however. Although macOS support is rapidly catching up, full Windows support is a glaring omission for most Firebase features. This means that if your platform targets Windows, you will need to implement the client functionality manually using Firebase's web APIs and wrappers, or look for another solution.
Extremely robust. Has about any tool you can think of under one roof making it extremely useful as a backup platform for data analytics or small teams that need something quickly.
Intuitive and easy UI/UX. Being made and owned by Google, you expect nothing less. Very easy to use for anyone that has any marketing or analytical experience especially in Google Analytics (which I just assume all marketers do).
Safe, secure, and sturdy. Never need to worry about downtimes or misinformation as it's as clean and safe as it is being run by Google.
FREE! What else is there to say. Unless you're an extremely large application handling hundreds of thousands to millions of users, this pay as you go plan will stay free.
Firebase/Firestore has very limited support for querying more complicated items; for example, performing a simple string search is not possible.
While upfront costs are low, costs can grow quickly if you're not careful about what you are being billed for.
Dashboards have at times shown different information to what is billed, and support from Google is less than stellar and not as effective as that from Amazon or Microsoft.
Firebase functions are more difficult to use, there are no concepts of triggers or cascading deletes without the use of Firebase functions. Firebase functions can run forever if not written correctly and cause billing nightmares. While this hasn't happened to us specifically it is a thing that happens more than one realizes.
We had the enterprise support with AWS, so overall support experience was good with great engineers on the back providing answers. As you may know, overall AWS support is different and this is not different. Responses through the regular web support channel came easily, fast and accurate. We had questions/issues which were solved fast. Documentation is good as well, especially around the test automation pieces.
Our analytics folks handled the majority of the communication when it came to customer service, but as far as I was aware, the support we got was pretty good. When we had an issue, we were able to reach out and get support in a timely fashion. Firebase was easy to reach and reasonably available to assist when needed.
We haven't used anything like AWS Device Farm before. I am familiar with Amazon Web Services and when we had our MVP ready to test, we turned to AWS for a solution. AWS Device Farm was exactly what we were looking for as we have a really small team and limit resources.
Before using Firebase, we exclusively used self hosted database services. Using Firebase has allowed us to reduce reliance on single points of failure and systems that are difficult to scale. Additionally, Firebase is much easier to set up and use than any sort of self hosted database. This simplicity has allowed us to try features that we might not have based on the amount of work they required in the past.
Firebase has been able to help us understand reliably, the drop-off in our user flows with their funnel feature. This has made it easy for us to be able to pinpoint weaknesses in our funnel and test and optimize with data as the dependent variable.
From an economic standpoint, we don't pay for Firebase which is great, but as the saying goes "You get what you pay for" also holds true in this context. As we looked to grow and scale, we looked for a paid solution.
From a developer resource standpoint, Firebase has been extremely easy to integrate into our app. Whether it be the event tracking, dynamic links or crash reporting we have not had to waste too much developer time thanks to their well-organized developer docs.
