Why I Wouldn’t Use Firebase Analytics In My Next Project

If I were looking for an analytics service, I would not use Firebase Analytics. It has too much cognitive overhead, too many surprises, too many things you have to do manually that it should have done by itself. (I would use Firebase’s BaaS tools, and perhaps use Firebase Analytics in that context, but not by itself, if I were just looking for an analytics service.)

⦿ Firebase Analytics does not distinguish between production and development versions of your product. As you develop your app, you’ll run it locally, and as you do this, analytics data from your local runs will get mixed with production data, corrupting it. To fix these, you need to detect such builds and call setAnalyticsCollectionEnabled(false). There are multiple kinds of builds to detect: running from Xcode or Android studio, running on the simulator, debug mode builds, and testing production builds before launch using TestFlight or other similar tool. You need to detect all these cases and disable Firebase Analytics.

And you should disable Firebase Analytics before it automatically logs any events! [1] I’d call this before initialising it with FirebaseApp.configure(). According to the Firebase documentation, you should not call any Firebase functions before configure(), but I have to go against what the documentation says to get Firebase to do something it should have done by itself.

⦿ But disabling analytics doesn’t let you test it. For example, if you add a new event, you need to run the app, open the Firebase console, and verify that it is indeed being logged. But when you do this, your production data is again corrupted by development data. To fix this, you need to create a second “dev” Firebase project, and remember to keep reconfiguring your app to switch between the dev project for development and the production project each time you run or release your app.

⦿ If you have a second Firebase project, when you open the Firebase console in your browser, it shows data from both the dev and production versions together. You should change it to production. But it doesn’t remember your choice when you open it again next week. I got wrong conclusions from Firebase Analytics because of this.

⦿ This problem of the UI not remembering your choices extends further: it doesn’t remember your choices even as you navigate from screen to screen. For example, if you sort events in decreasing order, then click an event to see more details, and press back, events are no longer in order, so you’ve lost your place and perhaps train of thought.

⦿ Some screens in the Firebase console don’t let you Ctrl-click to open in a new tab. You have to click, see the data, go back, sort again, verify that it didn’t screw up your filters, re-apply them if it did, click something else, wait for the page to load, and see the data. This makes it hard to compare data. I often ended up forgetting what I was comparing by the time I navigated the Firebase UI.

⦿ Firebase supports what it calls user properties. They’re independent of any specific event. Think of them as global variables logged to the server. For example, in a camera app, you could have one numPhotosTaken. But this doesn’t work — user properties can only be of string type, so are unsuitable for numeric data.

⦿ The Firebase Analytics console doesn’t let you see a graph filtered by event parameters. For example, say you have an e-commerce site, and an order_placed event, and you want a graph of people who placed orders above some threshold. But you can’t do this. Firebase Analytics doesn’t let you see a custom graph. You have to switch to another tool (BigQuery), wrestle with its cognitive overhead (see below), and do a query SELECT * FROM order_placed WHERE amount > 100. Then you need to export the data to a Google Sheet. Then draw a graph using Sheets. This friction is significant enough that you often end up not doing it, so you make decisions without data or with stale data, which defeats the point of using analytics in the first place. Not to mention that our designer, who doesn’t know SQL, was disempowered from getting the data he needs to design better.

⦿ Not only can’t you define custom graphs, but you can’t customse the home screen of Firebase Analytics to show graphs that would otherwise require multiple steps of navigation to reach. For example, for a camera app, the north star metric is photos taken. I had a photo_taken event, and wanted to add its graph to the home screen of Firebase Analytics, so it’s available at a glance. But it’s not customisable. I also wanted to remove some graphs from the home screen, like number of users, since it’s a vanity metric, and putting vanity metrics in your face draws your attention to them, which is the exact opposite of what you should be doing. Unfortunately, neither adding graphs to the home screen nor removing them is supported.

⦿ Firebase Analytics’s retention metrics are useless for the same reason — for a camera app, retention should be defined as what fraction of people who took a photo in one month took a photo in the next. For a messaging app, retention is what fraction of people who sent or received a message this month did so the next. For no product should retention be measured based on people who merely opened the app, as Firebase does.

⦿ The Firebase console lets you define custom audiences based on user properties, such as paying users. But only data logged after the audience is defined is counted towards the audience, which produces the wrong answers when you do your analysis. And you don’t even know this unless you suspect it and go around digging for it. This also makes custom audiences useless for a situation when you’re doing some analysis, and want to define a custom audience and see how many are in that audience. Why else would you define a custom audience?

⦿ Firebase Analytics is buggy. For example, a field that’s supposed to represent the first time the user opened the app had two different values for 0.6% of my users. How can anyone open the app twice, both the first time? This doesn’t inspire confidence that it’s processing my data correctly.

⦿ Firebase is a Google product and so using it is a business risk. In my case, Google’s systems couldn’t charge my card, that too for 1 paise (< 1 cent). Using a card from another bank didn’t work, neither did paying manually using the console. Google blamed me for them not being able to charge any of my cards, and threatened to delete my account, with nobody available to talk to to sort it out, of course.

Problems with BigQuery

⦿ Firebase Analytics exports to BigQuery, which offers a powerful SQL interface, but querying it is too complex, because the data is not modeled as you’d expect in SQL, with each field becoming a column, so that you can do, for example, WHERE amount > 1000 AND delivery_type = “Express”. Instead, multiple (key, value) pairs are shoved into one value as an array type. Yes, SQL apparently supports an ARRAY datatype. To work around this, you have to do complex, unintuitive stuff like UNNEST. I ended up having to write some 300-line SQL queries, which are a complete waste of time. Instead, Firebase Analytics should export its data to a weakly typed NoSQL database that lets different rows have different columns. Then we’ll be able to write intuitive, straightforward SQL queries like amount > 1000 AND delivery_type = “Express”.

⦿ The export schema also means that if you mis-spell a field name, like amuont instead of amount, you won’t get an error. You’ll get zero results, leading you to the wrong conclusion. With normal SQL, you’ll get an error saying that there’s no column named amuont.

⦿ If you want to query with a comparison, like WHERE amount > 1000 in the previous example, you can’t do it directly. You have to figure out what data type it is and write something like amount.int_value > 1000 or amount.double_value > 1000. This is not ideal — you should focus on your logic, not technical trivia. This has also resulted in wrong results, say when a data type accidentally changed in the code but we forgot to update our query. I realised only after a week that I was acting on incorrect data. If Firebase had just let me write amount > 1000, this wouldn’t have happened. While we’re at it, the logged data has a field float_value but even if you log a float, it gets logged as a double, so any query of the form amount.float_value > 1000 will have zero results.

⦿ BigQuery’s export options don’t work. You are locked into Google.

Conclusion

I would not use Firebase Analytics for my next project [2]. It’s a tool built without care, without sufficient respect for users’ needs and workflows. I’d look for another analytics service. I don’t mind using Firebase’s BaaS, and I might use Firebase Analytics in the context of that, but I would not use Firebase Analytics if I were looking for an analytics service.

[1] Another way to disable Firebase in local builds is to change the bundle ID in Git to something else. For example, if your app is com.mycompany.Product, change the bundle ID to com.mycompany.DevProduct and commit the change to Git. That way, when anyone checks out the code from Git and runs locally, Firebase Analytics will disable itself.

This approach isn’t enough, because you can test in-app purchases only with the correct bundle ID. And, even otherwise, you can’t test analytics changes you make.

[2] There are a couple more issues that aren’t Firebase’s fault, but which you still need to keep in mind to get the right results:

Firebase Analytics tries to log the LTV, but on iOS, it doesn’t include paid up front, because iOS offers no API to query how much the user paid before installing the app. If you have a paid up front app, the LTV is logged as zero. Or in a paymium app where the user buys the app for $7 and then makes an in-app purchase for $2, the LTV is logged as $2.

BigQuery does offer less than and greater than operators for strings, but they don’t work for version numbers like 1.19, because lexicographically it’s < 1.2.

These aren’t Firebase’s fault, but something that can trip you, so you have to keep these in mind.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store