🚀 Add to Chrome – It’s Free - YouTube Summarizer
Category: Interview Tips
00:00
we are going to answer one of the most common product interview questions and they're called debugging or root cause analysis questions and they sound like uber cancellations have gone up what's going on we're going to give you a
00:15
fullprof framework to help you Ace this question and get the job let's go so today we're going to use the example of Uber's cancellations have gone up what is going on and we're going to go through a five-part framework starting
00:32
with let's clarify the question with some questions the first thing I want to clarify is Define the actual metric that's going up and this is really going to help me come up with specific hypotheses so make sure that if the
00:49
metric they give you is ambiguous ask further questions so in this case you said cancellations have gone up is that writer cancellations or driver's cancellations and also are we talking about the cancellation percentage or the
01:06
total number of cancellations on the second question I asked because if total number of cancellations are going up but the number of rides booked are also going up well that might be expected so let's assume the interviewer tells us it's the
01:22
rider cancellation percentage great that helps me focus on one area by the way a proy here is that when you ask questions share your thinking behind those questions through all of these
01:37
interviews interviewers want to understand how you're thinking so sometimes they may not necessarily agree with you but when you share your thinking there's a higher inclination that they'll understand that you have logical thinking another thing I want to understand is the time period that this
01:55
drop came under and this also helps me filter out hypotheses that might not be likely so I'll ask a question like so cancellations have gone up but over what time period has this been a steady decline over the last 6 months or is
02:12
this something happening within the last couple of weeks month and have we seen a similar pattern before let's assume they tell us that it's over the last 2 weeks only and there was no similar pattern before next I want to ask so cancellation rate for Riders have gone
02:28
up but what is the change in percentage that it's going up by versus our Benchmark the answer to this also helps me filter out hypotheses of course something like a 10% increase is going to be much bigger than a 1% increase so
02:44
imagine they say the average of our cancellations was 5% and then it doubled to 10% and the last clarifying question I would ask the interviewer is did you want me to get towards a specific answer that you have in mind or did you you
02:59
want me to generally explore and share my thinking this is important to realize because a lot of people answering these questions think oh there is one right answer but whenever I ask this question in interviews I don't have a specific answer I want them to get towards I just want to understand how they diagnose
03:18
problems so the second part of the framework is jumping into identifying the levers that are going to help us understand and come up with hypotheses so the first thing I want to cover here is is who are the users for this product so in this case we have the drivers and
03:35
the writers so we're realizing supply versus demand and not we did call out that the writer's percentage of cancellations have gone up but this doesn't necessarily mean it's a rider specific problem and I'll show you what I'm talking about in a bit next I want
03:51
to cover the user flow so how does this product work so on the user side they'll first open the app then type in the destination of where they're going and where they're at right now then a list of prices and different services or different rides would come up they
04:07
choose one there's a process of matchmaking where a driver is being found so it's called a processing part and this is a big area where people first have the opportunity to cancel next a driver accepts or declines then
04:22
the ride is confirmed and you get a page that shows you the ETA your driver is away from picking you up and how long it's going to take for the ride to be completed and how much it costs and here is another area where a customer can cancel the next thing I could do here is
04:39
some segmentation to understand if I break this out by let's say region are we seeing the cancellations coming from a specific region a specific city or country Etc because that could tell us maybe there's Regional specific things
04:55
that are leading to this problem and it's more of an external effect then it is something internal another thing I can cut it by is the day so maybe there are specific days over the last two weeks that led our total number of
05:10
cancellations or the cancellation rate to be pulled or maybe there was a couple days that led our cancellation rate to go all the way up whether something to do with weather or some event that was going on here you want to give a
05:26
rationale why you would segment by specific factors like I just shared so let's say the interviewer tells us that there was nothing specific on region or day so how would I proceed so tip here is I'm going to take a second to
05:43
summarize what I know so far so that helps me think of hypothesis so what I know so far is that Rider cancellation percentages have gone up over the last 2 weeks and it has doubled from 5% to 10%
05:59
so pretty darn drastic there is no Regional or day specific segmentation so it seems like a cross Regional problem so my third step is to start hypothesizing and figuring out ways to validate my hypothesis or invalidate it
06:14
here I'm going to use the user flow to help me drive some hypotheses on why cancellations have gone up my first hypothesis might be our cancellations attributed to the section where you type
06:30
in your destination where you're at today because if people are typing in the wrong address whether it's their starting address or the ending address that might lead to an immediate cancellation and a followup question I might be asking data for is are we seeing that these people that are
06:45
canceling are then booking another ride right after and maybe my hypothesis is that in the last 2 weeks we changed something in the user interface let's say the drop pin button where you place where you're at today is is kind of
07:00
Chang in how it looks or maybe the algorithm to determine where you are especially when you're in taller buildings was shifted a bit and hence when people drop inin where they are it wasn't actually where they were so that messed up and people had to then cancel
07:17
their rides because they noticed the address was wrong so how I would want to validate or invalidate is so then search how many of these cancellations led to a reorder with an updated address that was nearby I could also validate invalidate by
07:33
maybe calling some of these customers who have canceled and asked them another way of validating invalidating instead of guessing we can basically survey some of these customers and bring up a quick form to ask them well why did you cancel and give them a couple reasons and give
07:50
them other slot to see what they say so another tip for you here when you're calling out hypotheses make sure they're specific to the question or the product that you're talking about so for example a lot of people like to give the
08:05
hypothesis oh well there's competition so that's generic but if you say something like oh you might have competition like Lyft and Uber always fighting each other to get the best price for customers and that is what could be leading people to cancel
08:21
because the second narrative shows that you actually thought about how the competitive pressures would impact this metric we're talking about versus just throwing in something on a checklist like competition the next hypothesis I have is related to pricing over the last
08:36
2 weeks did we see our competitors reduce prices because I know for me I'm usually shopping around between Lyft and Uber to get a cheaper price and sometimes after I've ordered the Uber ride I go check lift and see if there's a cheaper prices and then I will cancel
08:52
the Uber ride so maybe Lyft in the last 2 weeks decided to give extra discounts or to offer cheaper pricing and how would I validate this by go using Lyft versus Uber in specific locations and
09:07
see if the price is cheaper my third hypothesis is you know that section where the ride is being processed so you've made your request and the page is showing finding drivers so that's the first opportunity where someone can cancel so first I
09:24
would want some data to understand are the cancellations this 10% mostly in this phase or after a driver is matched because if it's before the match happens the cancellations could be because that processing time in finding a driver is
09:40
taking too long and hence over the last 2 weeks have we seen a change in supplier demand so if we saw less drivers that may make it harder for existing Riders to find a match with a driver but if we've seen increased
09:56
demand even with the same number of drivers that could also sort of throw the equilibrium off where it makes it harder for us Riders to find a ride that's nearby so how would I validate or invalidate this I would go look at the data to check the average time the
10:13
average processing or matchmaking time of the rider being matched to a driver and compare this to the time before the 2 weeks to see if we've seen an increase and I would also go check the number of active drivers and the number of active
10:28
order ERS to see if that has gone up lastly another hypothesis is that when rides are confirmed and people see the ETA it is way too long so people cancel I know usually for me if it takes a driver more than 10 or 15 minutes to get
10:44
to me there's a higher chance that I'm going to cancel the ride either because I found an alternative or in that time I've had time to go look at lift for a cheaper price so I might check before the 2 weeks versus during the two weeks
10:59
where our cancellations have gone up have we seen a difference in the time it takes a driver to come to the rider and again this is another example where it's not just a rider specific problem but it could be a driver specific problem where there might be a lack of drivers and
11:17
that leads to wait times to be longer because drivers have to come from far away to support certain writers step number four I'm going to prioritize which of my hypotheses I want to spend more time validating invalidating and coming up with solutions for I'm going
11:33
to narrow them down based on some of the data that I shared before on that this happened within the last 2 weeks it was a 10% it is now a 10% cancellation rate versus a 5% so that leads me to two key hypotheses the first is on the pricing
11:49
bit and the second is that the weit time has increased for Riders whether before they got matched to a driver or after they got matched to a driver and they're waiting for the driver to pick them up I'm deep prioritizing my first hypothesis the drop pin hypothesis
12:06
because I don't believe that that's going to double cancellation rates because when people check their destination and where pickup is they're usually pretty Savvy about it or they'll double check but that's just me so I might prioritize that hypothesis last to validate or invalidate so step number
12:22
five based on some of these hypotheses how would I fix this so for looking at the price ing problem basically that our competitors are offering cheaper prices I might do simple things like price parody with lift basically matching the prices that we're seeing with them or
12:39
marketing ourselves to better emphasize that we're better on certain features for example maybe we're faster at getting the driver to the rider or maybe we have features that make people feel more safe in an Uber ride than a lift ride for the problem of matching drivers
12:55
and Riders taking too long I may first have to find out it like is the problem because there's not enough drivers or there's way too much Supply either way a couple solutions for that I might increase the incentives for drivers to
13:11
get more of them to go back to the platform or I might send out a bunch of notifications for drivers that are not active to get them back on the platform to let them know there's a whole host of demand where they can earn more or maybe I might change the algorithm that's used
13:26
in matching drivers and riders in a different way way that'll make it a bit more efficient or maybe relax some of the constraints on the algorithm so that writers are always able to find a driver even if it's a bit more inconvenient for the driver so what are the five things
13:42
we covered to answer debugging root cuse analysis questions you also want to know what are the things you should not do for these type of debugging questions so take a look at this video which shows you what typical candidates do
13:59
and big no no that do not show thinking and I will see you guys in the next video