Meesho’s engineering efforts to create a single interface for all analytics requirements.
By Arjit Agarwal, Pratik Kumar, Joel Fernandes, Vibin Reddy, Sweekriti Dhawan and Ashok Lingadurai
Meesho is built on the premise of analyzing and understanding consumer expectations and satisfaction. By adhering to this principle, we continuously improve our product (app).
Always start with the user.
Our core value, “User First,” requires all teams to take the user’s psychological profile into account when designing new features.
In recent years, Meesho has grown exponentially, with millions of users regularly interacting with it. Every day, millions of interactions (or events) take place, which gives us the opportunity to better analyse user behavior and optimize their shopping experiences on our mobile app.
Capturing user interactions poses a variety of challenges. To help us overcome these challenges, we built our own in-house analytics tool called Meshlytics.🤩
What is Meshlytics?
Meshlytics is a single interface for all your analytics needs. It is a pure Kotlin library that allows your app to interact with all of your analytics platforms (like Mixpanel or Clevertap) through one single interface.
![](https://admin.meesho.io/content/images/2021/09/image-47.png)
Sounds simple! But Meshlytics is quite powerful and is packed with many features:
- It is a pure Kotlin library with kotlin-multiplatform, and you can use the same library on Android, iOS, and the web.
- It makes it super easy to integrate a new analytics platform or remove an existing analytics platform.
- It allows you to define platform-specific events.
- It makes onboarding existing events to a new platform very easy.
- It allows you to define a regex for your keys and validate if events and property names follow the regex.
- It allows you to have different naming patterns for different platforms without worrying about conversion while sending the event.
Why did we build Meshlytics?
As we were integrating new analytics platforms in our application, we faced various problems that made our development cycle more challenging and painful. For example, when we decided to integrate Clevertap and send an existing event App Open
to it, we had to rewrite the same repetitive code that we had written for Mixpanel earlier. This problem was more serious when the new platform followed a different naming convention like Snake Case (app_open
)
Let’s look at a code comparison for the following task: App Open
event being sent to 2 different analytics platforms which follows different key conventions.
![](https://admin.meesho.io/content/images/2021/09/image-48.png)
One the older code (left), if we decide to send the App Open
event to a new platform we have to write the same repetitive code once again. In short, we were facing four major problems:
- Code pollution.
- Difficulty in integrating a new platform or removing an existing platform.
- Slow development cycle to onboard existing events to a new platform.
- Erroneously sending different data to different platforms
Hence, we decided to create an aggregating layer that solves all the above problems and provides us a neat and extensible way to solve our analytics challenges. The idea came into realisation as Meshlytics.
In the above code snippet (right), the App Open
event is still being sent to the same three platforms using the same key conventions as described earlier. Clearly, the code looks neater now, and no code addition is required here even if you decide to send this event to a new platform in the future.
How did we build Meshlytics?
We formulated the idea of Meshlytics to solve several problems that were specific to Meesho. However, we wanted to extend this idea so that others can leverage this library in solving their product-specific use cases.
Hence we decided to build Meshlytics in iterations, with each iteration focussing on a subset of problems. Ultimately, we created a pure Kotlin library that is extensible and can be customised heavily to solve product-specific problems. Our first principle was to make sure:
If a problem is generic then it should be part of the library, otherwise, it should be solved through extensibility.
For example, when we allowed our users to experience the app without logging in, we decided to prefix certain events with Anonymous
. We felt that it was not a generic requirement, and hence we did not provide any provision to prefix events in the library when the user is not logged in.
To sum up, Meshlytics went through the following iterations:
Iteration 1
In its first iteration, we studied various analytics platforms, selected a common API to support via Meshlytics, and exposed a single interface to the app to interact with Meshlytics.
- Since the actual platforms depended on the consumer, we created an abstract implementation for a platform. For every platform, you need to write a concrete implementation of
Dispatcher
and register it toAnalyticsManager
. Dispatcher
supports generic common API. Hence the platform-specific API can be exposed via concrete implementation ofDispatcher
.- For defining dispatchers (or platforms) on which an event is going to be sent, we created
DispatcherResolver
which is again abstract, and the library’s consumer must provide the concrete implementation. See concrete implementation here.
Dispatcher Resolver makes onboarding events to a new platform very easy. For example, the following snippet sends App Open
event to Mixpanel, Clevertap, and Meesho.
![](https://admin.meesho.io/content/images/2021/09/image-49.png)
Suppose, if you’ve written a new dispatcher for FIREBASE
and you want to push the same event to it. Change the above code to:
![](https://admin.meesho.io/content/images/2021/09/image-50.png)
Automatically, the library will start sending this event to Firebase too.
Iteration 2
In the second iteration, we mainly focused on the naming strategy. We used the title case (App Open
) for naming events and properties related to Mixpanel & Clevertap; and snake case (app_open
) while sending events to our own analytics platforms (denoted by MEESHO
).
Earlier, we decided to support only the above two naming conventions. We felt that down the line, someone may require support for app-open
or maybe weirder app*open
formats too. So, we took inspiration from Glide and decided to use transformers known as KeyTransformer. A key transformer takes a string and outputs a string after applying the transformation.
The library itself provides specific concrete implementations of KeyTransformer
like LowerCaseTransformer to convert the key into lower case. As with Glide, you can use a combination of KeyTransformer
. For example, to convert App Open
to app_open
you can use LowerCaseTransformer
and PatternTransformer like below:
![](https://admin.meesho.io/content/images/2021/09/image-51.png)
You can register these transformations with the concrete implementation of Dispatcher
and these will be applied automatically before sending events to the associated platform.
Iteration 3
In the third phase, we built validation over event data. Our default naming convention for events are title cases containing only alphanumeric characters, but during development, we didn’t follow this pattern strictly and ended up using strings like Button Clicked
or Video Play/Pause Clicked
.Although this works just fine, special characters might be a problem while building data pipelines, and it is recommended to follow a naming convention strictly.
So, we created a mechanism to validate incoming data. First, you need to register an acceptable regex with AnalyticsManager
. Incoming keys are expected to honour this convention. For example,
![](https://admin.meesho.io/content/images/2021/09/image-52.png)
You can try to experiment here, and check if your key name is valid for the above regex.
All the incoming data is checked against this regex, and you can configure the validation level as NONE
, LOG_ERROR
and ABORT
as per your requirement. Since validation may slow down the actual process of sending the events, we recommend using LOG_ERROR
in the debug variant and NONE
in production. Now, even if you’ve accidentally broken the naming convention, the library will report an error, and you can fix it during development.
Our Learnings During the Implementation
Despite starting with a strong design foundation, we did face many challenges while implementing; so we kept revisiting the design again and again to make modifications wherever necessary.
Transforming Keys
Initially, we decided to support only two naming conventions; title case and snake case. However, over time we needed other transformations for reserved properties (for example, mixpanel accepts email under $email
key) and requirement to add Anonymous
prefix with certain events if the user is not logged into the app. Since these requirements were unexpected, this led to fragmentation in transformations and the Dispatcher
API started to look more complex.
We solved this problem with KeyTransformer
and now we can add any new transformation just by adding a transformer, and we don’t need to worry about where the Dispatcher actually uses it.
Dispatcher Resolution
Earlier, we used String
as dispatcher id and so DispatcherResolver
maintained a list of String
for each event name to store dispatcher information. This was consuming a decent amount of memory. We reduced it to less than half, by making the dispatcher id as Byte
. Since there is just one set bit in each dispatcher id (power of 2), we can use the bitwise OR
operation to store the same dispatcher information in a single Byte
.
![](https://admin.meesho.io/content/images/2021/09/image-53.png)
Optimising Dispatcher Resolution
At first, we resolved dispatchers by doing bitwise AND
with each registered dispatcher (and sending events only if the bitwise AND
of resolved byte and dispatcher id is non-zero). Though shouldn’t be a problem, if you are using 64 dispatchers and sending most of the events only to 3 platforms. However, now we do it only for the set bits in resolved Byte. So, if an event is meant to be sent only to 3 platforms, its resolved Byte will have only 3 set bits. While you may have 64 registered dispatchers, the library will send the event to those 3 dispatchers without making any comparisons.
We made good use of bit-magic to do so.
Several rounds of learnings and iterations made made Meshlytics one of our super complex projects that required days of brainstorming andcollaboration. Considering the fact that social e-commerce is on the rise, our engineering projects like Meshlytics will serve as a backbone of Meesho in the upcoming months (even years !)
Interested in working with Meesho’s Android Engineering Team and changing the way India shops online? Take a look at our careers page for opportunities.