The framework that may allow you to ace the Success Metrics Questions and standout
As I equipped for my Product Knowledge Scientist interviews, I scoured the net for suggestions and frameworks on dealing with the “Success Metrics” interview query. Regardless of discovering bits and items, a whole, end-to-end information was lacking. That’s why I’m excited to share the final word framework I crafted throughout my preparation, which landed me a proposal from Meta! Dive in, and I hope it should give you the results you want too!
Framework — Assume you’re a part of the DS group for Fb Teams, how would you outline success metrics?
Clarifying Questions — At all times begin by asking clarifying questions. Just remember to flesh out each phrase of the query and most significantly the product in scope. In case you don’t ask any questions, that’s undoubtedly a crimson flag, so please do!
Let me ask a number of clarifying questions. Are we taking concerning the Teams within the Fb core app? Is my understanding appropriate that Teams will be non-public or public?
As soon as the query is obvious, then take an enormous breath and begin your reply by taking an enormous detour. Sure, you learn that proper — don’t simply soar into answering the query. It’s paramount, and most significantly anticipated, to speak concerning the product, the mission of the corporate and the way these two tie collectively earlier than the rest. So please just remember to discuss concerning the factors under.
Begin your reply by taking an enormous detour — don’t simply soar into answering the query immediately however discuss concerning the product, the mission of the corporate and the way these two tie collectively earlier than the rest.
Firm’s Mission
To start with, Meta’s mission is to deliver folks nearer collectively and provides them the facility to construct communities.
Purpose of the Product + the way it ties to Firm’s Mission
FB Teams aim is to deliver folks with frequent pursuits nearer collectively and it’s a crucial product of Fb since its aim ties with the general mission of Meta to deliver folks nearer collectively.
Customers — At all times discuss concerning the customers. Nearly for each product there are 2 sides of customers: the producers and the customers. It is extremely vital to speak concerning the consumer journey of each and much more vital to present metrics later in your reply that cowl either side.
For FB Teams we now have two sides of customers, the admins of the teams who’re the producers and the members of the teams which are the customers.
The admins create a bunch, determine whether or not it is going to be pubic/non-public, ship invitation for customers to hitch, submit hyperlinks/media/information and get the dialog began within the group.
The members both see on feed, search or get invited to a bunch (relying on if it’s public or non-public) and as soon as joined, they’ll have interaction with the group via posting, commenting, liking, sharing and so forth.
Advantages+Prices (each for the Customers and Firm) — earlier than leaping into the metrics is nice to briefly discuss the advantages and prices of the product each for the customers and the corporate.
One of many principal advantages of FB Teams is that customers with frequent pursuits can get collectively, which ties with Meta’s mission and contributes to FB app total engagement. FB Teams additionally enable Meta to get a greater understanding of the customers’ pursuits by trying on the teams they’re members of, which in flip can assist in making higher suggestions and extra partaking feed for the customers.
On the flip facet, FB Teams may probably make customers to interact much less with FB’s Newsfeed, which is the “coronary heart” of the FB app and the place the place income is generated via advertisements. One other potential con is when teams don’t get ample members or engagement, which may discourage the admins and create “empty shell” teams.
Sorts of Metrics to give attention to — Now it’s time to start out taking about metrics. Select 2 out of Acquisition/Activation/Retention/Engagement/Monetization to give attention to.
Now going to metrics, since FB Teams is a mature product, I consider it is smart to give attention to Engagement + Retention.
Point out Firm’s NSM (North Star Metric) + reporting numbers — so as to select the first metric, it’s vital to have the NSM and reporting numbers in the back of our thoughts.
Earlier than leaping into the success metrics of the product, let me shortly discuss Meta’s NSM, which is the variety of periods per consumer per day. On prime of that, Meta reviews to Wall Avenue DAUs and MAUS. Therefore after we discuss success metrics, and particularly when choosing the first metric, it’s vital to maintain the above in thoughts.
Metrics — Now it’s lastly time to present the metrics. We’ll give metrics from the 2 areas we talked about above we’d give attention to and it’s vital to not over do it — 2 or 3 to-the-point metrics per space are greater than sufficient. Please notice how we be certain to have metrics that cowl each the producer and client sides of customers.
It’s vital to have metrics that cowl each the producer and client sides of customers.
Engagement:
# of teams created per week with at the least 3 members per consumer
# of interplay inside Teams per consumer per week
# of periods that concerned Teams per consumer per week
Retention:
# of energetic days per consumer throughout the previous 7 days (Lively = used FB Teams)
2nd week retention = # of customers that had been energetic at the least as soon as every week for two consecutive weeks / # of customers energetic solely the primary week
** I selected the week as timeframe since I consider FB Teams will not be meant for use each day.
Select Driver/Secondary/Guardrail metrics — Now it’s time to decide on the first, secondary and guardrail metrics from our record of metrics above. Don’t overlook to speak concerning the trade-offs!
Driver Metric:
# of periods that concerned group per week
[# of sessions is easy to explain, captures users behavior and ties with Meta’s NSM. If sessions involving groups are up then more interaction among users → creating communities.]
Secondary Metrics:
# of teams created per week with at the least 3 members per consumer (to seize provide)
# of interplay inside Teams per consumer per week (to seize demand)
# of customers energetic 2 weeks in a row (this + final week) / # of customers energetic final week (2nd week retention)
Guardrail Metrics:
# of customers faraway from teams
# of teams shut down or reported
# of offensive/inappropriate posts posted on Teams per week
time spent on Newsfeed (we don’t need customers to cease utilizing Newsfeed as a lot)
Wrapping up — At all times a good suggestion to shortly undergo the story you simply put collectively and present how/why it solutions the query.
In order that’s it — that is the framework that labored for me and helped me land a proposal from Meta! The identical framework can be utilized for any Product/Success Metrics questions requested for Knowledge Science positions. Having a well-structured response that completely scopes out all of the vital parts and takes the interviewer alongside your thought course of is vital. Hope you loved it and would love to listen to your suggestions within the feedback under!
