Episode 162: Product Roadmap: Building a Platform for the Next Decade with Craig Saldanha, Chief Product Officer at Yelp
In this episode of Product Thinking, Craig Saldanha, Chief Product Officer at Yelp, joins Melissa Perri to explore user research, two-sided marketplaces, and the concept of the flywheel. They discuss future-proofing the business, building customer trust, and Yelp’s importance as a platform for reviews and recommendations.
You’ll hear them talk about:
[02:44] - Customer obsession is one of the key differentiating factors between Yelp and its competitors. Craig says that what separates Yelp from their peers is the quality of the reviews. It is the depth and breadth of these reviews that build customer trust, and mean Yelp is the go-to when someone is looking for a local business, restaurant, or nightlife spot. Craig emphasizes that every review is equal, and that this ethos drives the “road map” for Yelp’s evolution.
[08:26] - Craig decided to codify the product ethos and product culture of Yelp into five tenets. Tenet one is delighting a two-sided marketplace. Tenet two is regarding decision making, and how to differentiate between ‘one-way doors’ (like a price change) and ‘two-way doors’ (like UI/UX changes). Tenet three is being consistent and scalable. Tenet four is being diverse and inclusive in product thinking. Tenet five is “think big and learn fast”.
[13:40] - While driving revenue is, naturally, very important, Craig says that simply developing products that “delight the customers” is vital. Whether or not something moves a metric is almost by the by in the ideas stage, since the driving force behind product development should be the bigger picture, aka the customer experience. Craig says that if you build things people love, “you're definitely going to be OK in the long term”.
[20:13] - Working across teams, especially in product management, can be very difficult. Craig says that although each team does their best to work out their own dependencies, they will most likely only get 60% of them. One solution is to have a small product operations team that “owns the overall level road map”, can reconcile differences between other teams, and “see where some of the gaps are”.
Episode Resources:
Other Resources:
Intro - 00:00:01: Creating great products isn't just about product managers and their day-to-day interactions with developers. It's about how an organization supports products as a whole. The systems, the processes, and cultures in place that help companies deliver value to their customers. With the help of some boundary-pushing guests and inspiration from your most pressing product questions, we'll dive into this system from every angle and help you think like a great product leader. This is the Product Thinking Podcast. Here's your host, Melissa Perri.
Hello, and welcome to another episode of the Product Thinking Podcast. Today, we're joined by Craig Saldanha, who's a Chief Product Officer at Yelp, responsible for bringing in huge investments in the implementation of AI and LLMs to their services. Craig spent almost nine years working his way up the ranks at Amazon, from an internship all the way through to being Director of Product and Engineering for Prime Video International. I'm excited to talk to him about how he leads product management teams at scale. But before we talk to Craig, it's time for Dear Melissa. So in this segment, you can ask me all of your burning product management questions. And this week, we have a caller. So let's go to the phones.
Caller - 00:01:15: I'm Lisa, so I'm a biomedical technician and I'm hurrying on to pursue product management in the health tech industries to deal with any emphasis on how to achieve that position, the first steps I need to achieve. And also, can you recommend a book or a person that I need to follow, some companies that have product management in the health tech industry? Thank you very much.
Melissa - 00:01:42: So there's two different ways to think about entering product management and what kind of companies you might want to target. There are actually a lot of software providers out there for biomedical things. I've worked with two of them, Medidata and Cytel are two that do biomedical data. And they build software for people like you. So you might want to look at what kind of software you use and see if there's any openings in those companies for you to come in as a junior product manager and learn. Or you might want to start in a different role where your expertise and your clinical knowledge could really help and then try to transition your way into product management. So that's a start. So you can go to a pure software company and look for where your experience as a biomedical technician is really going to help. There's a lot of companies out there who really need those domain expertise. So that could be a good starting point. The other way to look at it is to go to a place that has software as a component.
Maybe they don't sell software but use software in the way that they run their business and express that you want to learn product management. You have the expertise already in your domain. You want want to learn product management. Im not exactly sure where you work right now but lets say you work for a pharmaceutical pharmacy. Now in pharmaceutical companies. They do have a huge engineering department they have a bunch of product managers. I work with a couple of them they have thousands of product managers and might be an opportunity for you to switch inside your company and say hey im really interested on it. If you want to do a switch internally, what you really need to look at, though, first is making that known. Make it known to your boss. Go find the people who are doing product management.
Find the leaders. Tell them that you're really interested in these things. A lot of companies have a way to develop people and help them transition into product management. That might be a good opportunity for you to get your foot in the door there, too. So I would look at those two options. If you want to go pure software, try to leverage your expertise to get hired by somebody who builds software for people like you. Or internally, look at where the software department is and see if you can make a move internally, express that you really want to learn this, and then do that. You're going to have to learn other things, though, besides just your domain experience. So I would look at trying to understand what product management is, what some of the tools that trade are, and make sure that you really want to do this first because a lot of people get into this role and then they hate it. So it involves a lot of people dynamics, involves working with engineers and tech. So I would really get up to speed on some tech concepts.
We do have a course at Product Institute called Tech Fundamentals for Product Managers. That might be a good place if you want to learn more about technology. We also do have a basics for product foundations there, too, that you could take. If you want to read a book, Inspired is a great place to start with Marty Cagan. I'll plug my own book, Escaping the Build Trap, for you to read as well. And there's lots of product management books out there for you to start to get up to speed on what that role actually entails. And I think if you walk in the door knowing some of those things, knowing what it entails, that's going to make it a little bit easier for you to transition. So I hope that helps. And good luck with your search.
For those of you listening, if you have a question for me, please go to DearMelissa.com, drop me a line. We do prioritize these voicemails, so I love to hear all your pretty voices. Let me know what your question is and I will answer them on an upcoming episode. Now it's time to talk to Craig.
Did you know I have a course for product managers that you could take? It's called Product Institute. Over the past seven years, I've been working with individuals, teams, and companies to upscale their product chops through my fully online school. We have an ever-growing list of courses to help you work through your current product dilemma. Visit productinstitute.com and learn to think like a great product manager. Use code THINKING to save $200 at checkout on our premier course, Product Management Foundations. Welcome, Craig. It's great to have you here.
Craig - 00:05:33: Thank you for having me. It's good to be here.
Melissa - 00:05:35: So it is 20th anniversary of Yelp being around. And I think everybody out there knows Yelp. That's where we get our reviews. That's where we figure out where to go and what restaurants to eat at, what service places to visit. Can you tell us a little bit about what Yelp looked like when you walked into the organization?
Craig - 00:05:52: Yeah, so first, let me tell you just a little bit about my history with Yelp, which is a good segue to why I joined in the org that I am inherited. So I first started using Yelp, I think, in 2008. And I remember because I was dating someone who's now my wife, who was a foodie. And she really prided herself on finding the hot and new restaurants and what it always got at the time. I was a very nerdy engineer who really viewed food as a vehicle to get through the day. And I spent a lot of time with chains. And so when we were dating, Yelp was my lifeline. I would go to Yelp to essentially impress her and pick all of these nice restaurants. And even more than that, when we went to a restaurant, there was often times that I would look at the menu and have no idea what these fancy dishes were. And so she would make fun of me because I'd pull up Yelp and look at the pictures and kind of decide what food I wanted to order. And so the opportunity to shape a product that I had loved for so long and used for so long and such a key part of my life was what essentially brought me here. And so I was already a fan. And so I inherited a product and a team that was in very good shape. And, you know, the focus was really looking ahead. How do we future-proof Yelp? How do we make Yelp appealing for the next decade?
Melissa - 00:07:14: That's awesome. I use Yelp for the same reasons and I like that. It has such a unique part of your relationship there too. When you talk about future-proofing Yelp, can you tell us a little bit about what's the situation look like for Yelp? Now we've got Google reviews out there. There's other places for reviews. I know Yelp was one of the first ones on the market. Tell us a little bit about what it looks like from your perspective and how you think about differentiation.
Craig - 00:07:36: I think what makes Yelp special is its consumers. We have such passionate folks who write the most in-depth reviews about their experiences, about the dishes they've had, about the ambience. And so I think what separates us from some of our peers is the depth and the breadth of the reviews that they have. I think it's why customers trust us when they want to pick a local business to connect with. And that's really the focus.
Melissa - 00:08:06: I had a friend in New York City who was one of the Yelp top users and we got invited to all these restaurants and lunch parties and all these different things all the time. And I know she was very passionate about writing her Yelp reviews. So I know you've got like a collection of people who are just a backbone there. Like, do you think about like how to engage those people all the time? Tell me a little bit about the customer obsession.
Craig - 00:08:28: Our Yelp elite squad is just awesome. And they champion us in the community, but they also provide guidance for everyone else. The reviews that they write are so in-depth and so helpful. It does form the backbone for a lot of what folks are looking for. But I think the nice part about Yelp is that every review is equal. Every review is important. Every review is valuable. And our elite squad is always a source of feedback. We have a lot of feedback sessions with them. But for me, personally, I end every week with looking at customer feedback. So folks who post on Twitter, folks who post on our page, folks who engage with Customer Service. So we have so many different channels, and that really forms the basis for what we prioritize, how we think about our roadmap, and how we think about evolving our article overall.
Melissa - 00:09:19: When you're looking at the product strategy as a chief product officer, coming into a new organization has got to be slightly overwhelming. And I think you had an interesting time, too, because you were fully remote onboarded. Yelp is not a small company by any means. When you came in, how did you get up to speed with what was going on and what problems you need to tackle? What's your way to get set up there?
Craig - 00:09:39: Gosh, that's such a great question. You know, prior to Yelp, I spent a decade at the same company. And so I knew everyone and I used to walk the floor every day and I kind of knew my team and I was really worried about this. And it actually turned out to be a super cool and super interesting onboarding experience. So there were a couple of things that were very helpful, some that I was forced to do and then others were just a byproduct of being remote. So the first was I got a chance to shadow our CEO and founder, Jeremy. And so I basically shadowed him for a couple of weeks, attended all of the meetings he was in and even beyond product, just got a sense for the whole company and how it worked and how and what actually made it tick. And I don't think that that would have been an option had we been in the office.
This was really helpful just to be a fly on the wall. And I thought that was super, super interesting. But what was even cooler for me was because we were a fully remote organization. I knew that there would be folks coming in after me who would need the same kind of context that I was getting as I was wrapping up. And so I decided that I had to codify what the product culture was and what the product ethos was. And so I ended up writing five tenets that I think are just very core to how we think about your product. But what was interesting was I remember taking them to Jeremy the first time and I said, from all of our conversations, I've written down these five. Tenets and he kind of read them. He said, you know, maybe one and two make a lot of sense. Three, I would modify and four and five, I actually don't agree with. And so that set up three months of debates that me and Jeremy and other members of the exec team had. And there were these really in-depth debates that we had just based on these five tenets that I had written. And after three months, we rolled them out to the whole organization. And that's the basis of what we use for our decision making. So it ended up being a really positive experience. And I got a chance to really tap into what made the organization tick and then put it down in actual words and writing.
Melissa - 00:11:45: What are your five tenets?
Craig - 00:11:47: So tenet number one is delighting a two-sided marketplace, which essentially means we acknowledge that we have two different constituencies that have different goals and needs. And while we always look for a win-win, there's some instances where there's going to be conflict and we have to use extremely high judgment to resolve those. Two is a decision making. We talk about one-way doors and two-way doors. And we want to move very quickly to two-way doors and blow things out and experiment quickly. But when there's a one-way door decision, like a pricing change, which would be hard to roll back, we want to be very slow and thoughtful and deliberate. Tenet three is we want to be consistent, scalable, and self-service. And that just speaks to building an MVP is great, but we want to build it scale. And we want to be thinking about that from the time we write down the requirements. Four is we want to be diverse and inclusive in our product thinking. So we have a very diverse and inclusive set of consumers and a very diverse and inclusive team. And we want to be very thoughtful and deliberate about making sure that every feature that we roll out has a broad appeal. And then the last one is think big and learn fast, which essentially means we want to take big swings, but we want to be honest and upfront about the risks and hypotheses. And we don't want to be scared of failing. When we fail, we want to learn fast and sunshine it so that other people don't make the same mistakes. So they sound somewhat general, but if you spend time at Yelp, you'll see that they're at the very core of everything that we do.
Melissa - 00:13:26: I love that you baked the diversity one in there too. And I feel like more consumer-based products can totally learn from that because there's not a lot of people who inherently call that out, especially when they have a diverse user base. How have you seen that kind of shape and help your product manager's decision-making there?
Craig - 00:13:41: So first, I think that explicitly, a lot of product managers want to build diverse and inclusive products. And so what we wanted to do was by making this explicit, we thought that we were empowering them to make the right trade-offs. So very often, I found that people would come up with really good ideas that had broad appeal, but they might favor narrower ideas that your revenue would grow some metric that they were being gold on. And I found this especially with more junior PMs. And so by codifying this, they can now point to a tenet. And even if someone like me or someone else says, hey, I think this should fall lower on the roadmap, they can remind us that it's core to what we're doing. We've said this explicitly, and it's a good checkpoint for us to say, let's go back and move it to the top of the stack.
Melissa - 00:14:33: That's such a good point. I feel like there's almost like a war sometimes between our OKRs and our goals that we want to hit on our product strategy. And then coming back to our tenants or the principals that we actually believe in. And so often in companies, the tenants will get sacrificed for the OKRs because people will see them as higher priority. How as a leader, do you make sure or do you work with the executives or your product managers to kind of balance what you want to hit from the OKR perspective, but then drive it back to that mission and goals? And where's the leeway? I think a lot of people have a question about like, how do I know these things need to get done? I've heard this from product managers, right? Like, I know these things are going to be good for us, but I'm trying to justify it back to our OKRs. And it might not align absolutely perfectly, but it's one of those fundamental things where I think it's going to make a difference. Like, how do you balance those things?
Craig - 00:15:21: I want to tell you a quick story from my past life that kind of speaks to your question. So I used to work at Amazon and I had the pleasure of working with Jeff Bezos a few times. And one time we had launched this new sports feature where you could instantly bring up highlights on your TV. You know, like normally if you're watching a broadcast, you wait for the broadcast show highlights. But we had built this incredible AI-aided model that was able to tell whether something was worthy of being a highlight, clip it, and then put it away. And we basically went into the meeting with Mr. Bezos and everyone was excited to share how many people watched the broadcast and how well it was doing. And he looked at the room and he said, you know how I know that you guys are doing a good job? He says, because of this feature. And the reason I know that you're doing a good job is because this feature. This feature is purely to delight the consumer. No one is going to watch more. We're not going to make any more revenue. But he built this because he knew that consumers would absolutely love it. And it'll take us a long time to figure out whether it moves any metric at all. But that's important. If you build things that people love, you're definitely going to be okay in the long term. And that's, you know, shaped a lot of my product thinking. And so to go back to your question, being a fully remote organization, we spend a lot of time upfront in every product meeting. Every time there's a new product or a feature, we spend time upfront with the tenants and other mental models to make sure that we're not losing sight of the bigger picture. If we're aligned on that, then the OKRs and the short-term goals are almost secondary. So we feel really good about our roadmap overall. And then the minutiae of does this make a difference if we pull this forward to Q1? It may drive an OKR versus if it shifts to the end of the year. That kind of stuff is where we let our OKRs guide us.
Melissa - 00:17:22: Do you do anything to kind of balance that where you're looking at the portfolio roadmap or you're looking at your percentage of time people are spending places? Like, how do you also ensure that it doesn't swing in a super different direction, right? Where you're not hitting any goals, right? And you're just delighting a bunch of customers, but not bringing in revenue. I feel like this is one of those things where product people just fundamentally have such a hard time balancing this. And they're like, how do I make the, you know, make the case for this? But then how do I oversee it and allow people the space to go do that, but not get carried away, right? In a way where we sacrifice the whole company for that.
Craig - 00:17:57: Good question. I think it boils down to balance. So we think about our OKRs or goals as what we want to achieve and our tenets as how we want to achieve them. And so the way our planning process works is there's a top-down set of goals that we start with, and then there's a bottom-up set of goals that the teams come up with. The top-down set of goals are generally from me, and they're based on what I want to see delivered in the next three to five years and working back, what does that mean for next year? And the bottom-up goals are really what our consumer is asking for, what our local business is asking for, what our focus group is telling us. And then over the course of a couple of months, we essentially marry both of those and come up with the right balance. And a lot of it is the ROI, both in terms of the impact it has at our top line and our bottom line, but also the impact that it has on consumer delay. And so that's how we balance the short-term roadmap versus the long-term. This is what our product ethos is.
Melissa - 00:19:04: So when you set the goals, are you going back to the teams in those planning sessions and then telling them, hey, tell me what we can do to meet these goals?
Craig - 00:19:11: Yeah, so we give the team an initial set of goals as a guide, and that helps order how they're thinking about their roadmap. But independently, they have an opportunity to come up with, these are the things that we think consumers really want. So independent of your goals, independent of where we think the company needs to be, this is what consumers have been asking for. And they're asking for it either directly through the various forums that I talked about, or the data is telling us that engaging this grade up until this point in the flow, and then there's a drop-off, we have to fix what's happening. So they have a backlog of items that they know that they want to get to, and then we kind of marry that with, as we get to all of these, how much of our goal do we achieve already? Do we achieve all of it? Are there new initiatives that we have to focus on? And then where do we prioritize those on the road?
Melissa - 00:20:02: One other part that I see with organizations when they're doing this top-down, bottom-up type of planning is they have a really hard time aligning all these things also across products or across teams. What do you do to make sure that the people below you, directors of product on different types of teams or VPs of product that might be reporting to you, how do you make sure they're all collaborating too towards the same goal?
Craig - 00:20:26: I'll say that in some of the podcasts that you've done along with product ops are very interesting to me for this exact reason, this working across teams can sometimes be very hard. You know, the short answer is every team does their best to work out their own dependencies. And then we recognize that they're probably only going to get 60% of them. And so we do have a small product ops team that then owns the overall level roadmap. And then they really get us from that 60% to 100%. So they'll say, well, these three teams say that they have a dependency, but, you know, last time teams D and E also had a dependency and how do we reconcile that? And they find areas that are unfunded. So we typically do an initial roadmap planning that we give product ops an opportunity to kind of go in and see where some of the gaps are, reconcile all of that, and then do a final roadmap for the year.
Melissa - 00:21:22: Do they have a product background, the product ops people you have on your team? Are they looking at data around what you're covering, what you're not covering, and they're putting it into the tools? What do they do to kind of build that final roadmap?
Craig - 00:21:33: So they work across product, engineering, sales and marketing, data science, and business operations. And they're the unifying force. They're a very small but mighty team. And they're like the unifying force of making sure that we're data-driven and cross the T's. I'll just say that I think every year the team gets better with understanding where their dependencies are. But we do have this safety net, which is very powerful.
Ads - 00:22:04: Are you eager to dive into the world of angel investing? I was too, but I wasn't sure how to get started. I knew I could evaluate the early stage companies from a product standpoint, but I didn't know much about the financial side. This is why I joined Hustle Fund's Angel Squad. They don't just bring you opportunities to invest in early stage companies, they provide an entire education on how professional investors think about which companies to fund. Product leaders make fantastic angel investors. And if you're interested in joining me at Angel Squad, you can learn more at hustlefund.vc/mp. Find the link in our show notes.
Melissa - 00:22:38: When they're looking into dependencies too, what type of work do they do to surface that up and reconcile it?
Craig - 00:22:44: Once they find a dependency, I'd say the vast majority of time by simply flagging it to a PM or the engineering leader, teams can essentially work amongst themselves. We definitely follow a distributed decision-making process where we focus on context and judgment and really let people autonomously operate in a way that they see fit. In the rare occasions where accommodating a dependency would significantly push out a team's roadmap. As an example, our commerce team does payments, they do accounting, they do tax, they do a whole bunch of things. And so they are very often a bottleneck and rearranging their roadmap is a high-judgment endeavor. And so then that bubbles up to me.
Melissa - 00:23:32: That sounds very useful for something at scale. And then how big is the product ops team? That's another thing that keeps coming up the more I talk about product ops. They're like, how many product ops people do we kind of need? How do you think about the ratios there?
Craig - 00:23:45: A car ops team is very small. It's like two to five people. So very small. We're just getting started, but they are very visible and they're much more senior than other folks. And so, you know, we train them and trust their judgment. So we have product levels that start at IC2, which is a new college grad, all the way up to IC6, which is a career PM with maybe 15 or 20 years of experience. And I'd say our product ops teams are all in the IC4 to IC6 level.
Melissa - 00:24:19: Wow, that's nice. It's a really good testament to because I've seen again, a lot of people be like, oh, I'll just hire a bunch of junior folks for product ops, right? Like we'll just we'll just give them some grunt work there. But it sounds like your product ops people are making critical decisions like that impact the business and they have to know enough to to be dangerous there.
Craig - 00:24:39: Yeah, and I think so beyond planning, we also have them be a liaison between teams on our most critical projects. Because that's where things can fall apart when you have product and engineering and marketing and sales all trying to march towards the same goal, but at very different paces and at very different stages of product maturity. We had these folks be the unified force and essentially report out on how projects are going. And so they also function as like mini GMs for some of these smaller projects.
Melissa - 00:25:15: Are those projects that don't quite align perfectly to a specific area of the product or what kind of projects are we talking about?
Craig - 00:25:22: As an example, let's take our multi-location ads product, just as like a random example. So the teams involved there are, we have the product and edge team that are actually building attribution products and ad products for much bigger companies that need a lot of treatment. You've got marketing teams that are helping market these, and then you've got sales teams that are pitching these. Now, for a new product in this space, you've got to start planning for when sales is able to pitch. You've got to train these reps to be able to go in the job. You've got to develop the marketing materials. And so any product and end slip, just an example of product and end slips by a month, now you've got to reconfigure and you've got to update all these folks. And so the product ops folks will play this role. They'll attend all the stand-ups to understand what's going on in product and end. They develop a really good sense of when folks say things are going to slip by a week as if it's a new week. Might it be two weeks? And then they're able to translate that for the leaders for them marshalling these huge resources. We have a big sales team, right? And so they're basically able to play that role.
Melissa - 00:26:37: Okay, so they're streamlining the communications between the different areas and keeping them up to date on the progress from the product roadmap and making sure that that's transparent to people. I love hearing about how other people are doing product operations. I think it's always really fascinating. So we talked about you came in, you shadowed the CEO when you started at Yelp, and then you built these product tenants. What else did you do to kind of think about where Yelp was going? I know when you came in, we talked a little bit about it, but maybe you can tell us about how far they came and what else needed to be done. But Yelp had moved in 2019 from like a sales-led model to a product-led model. When you walked in and you were trying to assess like how far did we get or what's changing, what did it kind of look like for you? Because in my experience, that can take years to actually manifest and keep continuing. What do you think was critical to make that change stick? And what do you continue to do as a CPO today to make sure that you don't revert back to sales-led models?
Craig - 00:27:39: I'll start by just sharing my mental model for what I thought Yelp's product needed to be. And then we'll talk about some of the challenges or making that happen as we made the shift. My mental model was actually very simple. And looking back, it hasn't changed in two years. There were three things that I wanted the Yelp product to focus on. And I thought we had a very good start. The first was consumer trust. And I feel like there's two ways that we own consumer trust. One is by the breadth and depth of our content that we talked about. And so it's protect our content and make it easier for folks to contribute. And the second was helping people effectively define their intent. So books come to Yelp for all kinds of reasons, from finding a restaurant to finding like a plumber. And so I wanted us to focus on our search and discovery experience. And really helping people from the simple things that they told us they wanted to do, get them very quickly. They're actually making a connection. So that was the first piece that do our trust. The second was making Yelp enjoyable and engaging. And that just came from being a fan for so long. And so we've done a lot in that space. We wrote out reactions. We just wrote out a home feed yesterday, which is very visual and has video now. And just make the whole experience enjoyable. Every part. We have password lists. We'll lobby and just taking out friction and allowing folks to enjoy. Okay. So we've done a lot of that. And then the third was to focus on elevating services. I've got an older house and I do a lot of work around the house and hire a lot of pros. And there's so much friction in the hiring process. It's not seamless. And so, you know, one of my focuses was how do we come in and remove friction from the process, but also give people peace of mind. And, you know, we've launched Yelp Guaranteed. That actually gives you money if you used our service and then thought that, really want us to focus on.
Melissa - 00:29:45: I didn't know there was a guaranteed for Yelp thing. I also have to hire a bunch of contractors because I have crazy house things, renovations and stuff that I'm doing too. So I didn't even know that existed, but that's awesome.
Craig - 00:29:57: Yeah, if you use our request support feature that a lot of businesses offer, then we guarantee you'll work up to like $2,500.
Melissa - 00:30:06: That's awesome. And I think that's such a differentiator from Google as well, because they don't do something like that.
Craig - 00:30:11: I focus very little on our competition and more on our consumer. So I don't know whether they started offering it recently, but I would say I actually use this feature a lot to give me confidence to hire new and under-brewed businesses. I recently bought an electric car and I had to install the charger. And I looked up Yelp and found a five-star or four-and-a-half-star rated business with 400 reviews that service my area. And I use our request support. And our software recommended I get a quote from this new business. And it turned out to be a phenomenal experience where I basically met this bro. He was just starting out on his own. His business was only nine for like a month. He had such a great story. We really hit it off. And he basically ended up having a relative from the city where I was born. Just like this really cool story. And in the past, I would have been apprehensive to hire someone. That had no reviews. I was just starting out. But knowing this was like a $1,500 install, knowing that I was backed by a guarantee that actually pushed me to give this guy a shot. And he was telling me that basically we got a lot of business in that way, which is cool. Like, this is like a win-win for everyone.
Melissa - 00:31:28: Yeah, that's what I was gonna say. I don't think it only helps consumers. It's like definitely what consumers want. But on the other side, it sounds like it's really helping small businesses are just getting started out there to let people take a chance on them. So this feels like one of those problems where you solved both sides of that marketplace, like very, very nicely.
Craig - 00:31:44: It's still early, but you know, it looks to be that way so far.
Melissa - 00:31:47: So when you're thinking about building two-sided marketplaces like this as well, you know, we have the businesses on one side and the consumers here. How do you think about that balance? And we talked a lot about like making the consumer happy. Like, how do you also think about what the businesses need? Since they're usually the ones who are paying, right? Whether they're advertised or the revenue. And I know you do ads and stuff through that. How do you balance the two?
Craig - 00:32:09: It's hard. And I think the best advice I can give PMs is when you are building a two-sided marketplace, have a clear mental model for how you're going to resolve the conflict. So I've worked in three two-sided marketplaces. Amazon, I launched an ebook subscription service where the two sides were authors and viewers. I worked in Prime Video where the two sides were consumers who were watching and then movie houses were making movies. And then now we have consumers and local businesses. And I think as long as you have a clear framework, you'll be okay. And at Yelp, we always look for the win-win. So the metric that we optimize on is connections between local businesses and consumers. At Amazon, we're very focused on consumers. And so we focus on giving them the best experience while being fair to the other side of the marketplace. And so I think you can build a marketplace as long as you have a clear framework. And stay true to how you're going to resolve that conflict and goals.
Melissa - 00:33:14: How did Yelp land on the connections between the consumers and marketplaces as that? Do you consider that like your one metric that rules them all type thing or the one that you really try to go after the primary metric or is there a system? And how do you get to that?
Craig - 00:33:28: That's been Yelp's mission, to make quality connections seamlessly between consumers and very local businesses. But the way it plays into this two-sided marketplace is local businesses come to Yelp because there's a lot of high intent consumers looking to hire local businesses. And consumers come to Yelp because there's a lot of local businesses that are reviewed and the reviews are deep. And so we landed on connections because that's core to making our flywheel go. And I think that's the other big insight that I got as a PM was you need to understand your flywheel before you design your product. Because like you were pointing out, if you focus on the wrong thing, then you basically design a product which might be successful in moving the metric that you think. Like we could have easily focused on like revenue or something else. But we focus on connections, whether it's advertisers or non-advertisers. And whether it's a phone call or an email. Or a request a quote or going to their website. We want to know that we made a meaningful connection between a consumer and a local business because we know that that's what gets that flywheel going.
Melissa - 00:34:41: The flywheel concept, I feel like it's incredibly important in product management, but not super well understood. Can you walk us through how you would, if you were coming into an organization like you came into Yelp, right? How do you figure out or how should a product managers, right, figure out what their flywheel is? How do they model it so that they can wrap their head around it?
Craig - 00:35:01: Product managers are always looking for product market fit. That's like the core thing that we're taught to look for. And in a two-sided marketplace, a lot of time, the product market fit comes from growth. You need to be big enough for one side of the marketplace to attract the other side of the marketplace. And so I think of Flywheel as what is the mechanism that's going to deliver sustainable growth over time. And that's how we think about product initiatives. So we think of a Flywheel as sustaining growth, and we think of the product levers that will help guide that growth. And that's how we come up with a two-sided marketplace.
Melissa - 00:35:43: So for Yelp, you've got your growth metrics is trying to make those connections, like more businesses, more consumers on the platform, the more you can do that. Those connections is what people keep coming back to. How do you think about the product levers connecting that?
Craig - 00:35:57: We think about both sides of the marketplace. And so for a consumer, we think about the needs that they have to satisfy to make a connection. And it should boil it down to the very basics. I think most consumers, when they're thinking about connecting with a local business, they have three basic requirements. The first is quality. Whether it's the quality of food, the quality of the service, they think about quality first. Two is price. Everybody wants to know how much they're paying. And three is timing. Timing. If it's a restaurant, am I going to get a reservation? Am I going to have to wait? Can I eat when I want to eat? If it's a pro, is this an emergency job? Do I have to get it done now? So we think about how do we give them all of the information with the least amount of friction to address their questions on quality, price, and time. And on the pro side, our local businesses, what they've told us what they're looking for in a quality lead is high intent. So is this person really interested in connecting with our local business and service area? Are they in my service area? Am I in the same city? Do I provide the type of service that they want? And so our product is really focused on taking both sides of that marketplace, taking the consumer's needs, taking the business needs and essentially matching them. And I think that's what we do. Please agree.
Melissa - 00:37:18: The part that I love about you explaining this is that it really all goes back down to deeply understanding your customers on both sides of this marketplace, right? Like what they want, what their problems are, which takes a lot of, you know, user research and empathy and really modeling that out. How do you instill in your organization as a chief product officer the concept of like user research and customer research? Like what kind of research do you do? Like who's going out and talking to people and how do you make sure that you're keeping in touch with your users?
Craig - 00:37:48: So the customer obsession piece is something that we look at from the interview stage. I think most PMs are customer-obsessed, but the real hunger to understand what customers want at their core and then find innovative ways to deliver that. I feel that that's very important to our organization. We talk about it in every meeting. We talk about it when we're making trade-off decisions. It's in our tenets. We just really focus on that. But then broadly was your question around how do we enforce it or what are the channels? We have a really good research team. And so for products where we're not quite sure how consumers will react, we do a lot of testing. We have sessions set up where they play with prototypes and they give us feedback. But beyond that, I think we're very blessed that we have a very large, engaged, and vocal consumer base. You've talked about Yelp elites. They're always giving us feedback about what they like and what they don't like. And we take that very seriously. We have listening sessions where our PMs will listen to calls of local businesses calling in the Customer Service to understand what their pain points are. And sometimes, you know, the pain points are we just build a product that's too complicated for them to understand. And then they're basically simplified. Like a lot of the folks that we're building for, some of them are sophisticated and they've been in business. For like 20 years. And some are just really good at being plumbers and they don't know how to navigate our product. And so we have PMs listening in to those calls all the time. And then even me, like I will end every single week with the voice of the customer. I've got teams that go through all the feedback that comes in. And then like the top 20 or the top 50, they'll send to me verbatim, like with screenshots or Twitter or a detail, a transcript. And then I spend time. Looking at what is going up to the top. I think we have a pretty robust mechanism in place, but you can always do better at like a consumer's never come to a product and say, this is the best possible product I have. You know, there's always ways.
Melissa - 00:39:59: There is this big debate going on right now about, I'm hearing about companies letting go of user researchers and seeing that user research is not important from just a user researcher perspective because they're trying to have all the product managers replace it. This is what I'm hearing for user researchers. I don't agree with this. Just to put that out there. You mentioned you do have a research team. How do you see user researchers and product managers working together on these things?
Craig - 00:40:24: So my personal opinion is that user research is very powerful. And regardless of who's doing it, we should never let go of that as a A-B testing and stuff is great. And that gives you the quantitative answers. But sometimes the qualitative answers are what you're looking for. As a PM, you have to feel the pain of a customer in order to understand how to simplify a product. You know, for me, the best way to use our researchers, and I'd say user research at Yelp, you know, as a formal function is probably about five years old. And we find that it's most powerful when it's directed to solve a specific problem. So when we're looking at why does drop-off happen at a certain point, and we give our researchers the tools to go and come back with answers and not really bias where they'll go, you know, we find that very powerful. We find it less effective when it's a broad, do we think users are going to respond well to LLMs? That's not a useful question. So the research that comes back then is not helpful to help product managers refine their thinking. So I always think of it as a refining step as opposed to an ideation step.
Melissa - 00:41:35: That's really insightful. I like the way that you think about that. Thank you so much, Craig, for being with us here and for sharing your story at Yelp and how you think about product management. I think a lot of people are going to listen to this and have so much to take away from it. If people want to learn more about you or product management at Yelp, where can they go?
Craig - 00:41:54: First, I'd say, find me at Yelp. I always have a bunch of recommendations and then LinkedIn as well.
Melissa - 00:41:59: Great. And we will put all of those links so that you can find Craig in our show notes at productthinkingpodcast.com. Thank you so much for listening to this episode of the Product Thinking Podcast. We'll be back next Wednesday with another wonderful guest, and I'll be answering all your questions if you send them to dearmelissa.com. So make sure that you go do that, and we'll see you next Wednesday.
Craig - 00:42:19: Thank you, Melissa.