Episode 94: Dear Melissa- Answering Questions About Platform Changes, Personalized Prototypes, and Prioritizing
In this Dear Melissa segment, Melissa addresses a recent Twitter topic worth further discussing: how important it is for PMs to carve out the time for creative thinking and processing outside of scheduled meetings and tasks. Then she dives into subscribers’ questions about measuring the value of a platform change, how to create prototypes when the product requires personalization and balancing outcome-focused and functional-focused work.
Have a product question for Melissa? Submit one here and Melissa may answer it in a future episode.
Subscribe via your favorite platform:
Q: How do you measurably prove the value of a platform change?
A: A lot of people struggle to measure the value of a platform change, but at the end of the day, platforms are incredibly valuable to the business, not just from a reduced-cost perspective.
Q: How do you rapidly prototype features that rely on personalization as a value driver without creating the actual product? Do we just build personalized prototypes per test user?
A: It’s important to remember what you’re actually testing with the prototype. This is why I am a big advocate for a series of experiments before you actually land on what you're going to build. Tune in to find out what type of experiment you can use to test your features.
Q: How do we balance outcome-focused work with functional work?
A: Foundational/functional work is usually outcome-focused. You just have to think of it as a level-up; when I do this, what does that enable? It's not just about building the nuts and bolts of a platform, it's about what that platform will actually do at the end of the day.
Resources
Melissa Perri on LinkedIn | Twitter
Transcript:
Melissa:
Hello and welcome to another episode of Dear Melissa. Before we jump into the questions today, and we've got three good ones for you, I wanted to take a moment and talk about something that I just wrote about on Twitter that I felt might be helpful to talk about here too. So I've gotten a lot of questions from product managers over the years about how do I make time for other things besides just answering questions from developers and getting into the head space of the day to day. And I think recently I answered a question about this as well, but I wanna remind you right now that your job as a product manager is a creative one. It's not just execution oriented, but there's a lot of creativity that goes into being a product manager. And when I was first starting out, one of my first bosses, his name is Chris Keen he reminded me of this once I was sitting there struggling to figure out what we were going to build digesting all this data I was doing the UX design as well at the time, but I was just racking my head about how to solve this problem and I didn't know how to.
And I thought sitting in front of my screen for the next eight hours was going to magically solve my problems. And as I was punishing myself about this and really racking my brain and I was getting super, super frustrated and he saw that and I was talking to him about it and I said, I just can't figure this out and it's really bothering me. And it was two o'clock in the afternoon in New York City and he said, You know what? Go home. <laugh>. Just go home. You're not gonna solve that today. Your brain needs a break. You're just totally overstimulated. You're thinking about this way too much. Go home, take a walk, do something else. And I bet you tomorrow you're gonna come in here fresh and actually have some space to think about it and mull it over and you're not gonna be killing yourself and you'll solve it.
And I did. And I felt really weird leaving the office at two o'clock to go home and not be working, especially when I was pulling really long startup hours those days. But I did. And I went home and I just did other things that had grocery shopping and walked around and I just started kind of turning it over in my head casually instead of so angrily like I was in front of my computer. And the next day I was able to come in and solve that problem. And that's what we need as product managers. You are in q4, Hey, yay, q4, which is usually crunch mode for a lot of companies. And we're figuring out strategy and we're figuring out what to do. And that is also creative. You're getting pummeled with tons of information and that's good. That's what we need. We need a lot of information.
And you probably have crazy deadlines to try to pack this all in before the holidays, but you might be stressing yourself out too much and you may be in 8,000 meetings that don't give you the brain space to actually consider a lot of these things. So what can you do? I block my calendar. I have been through periods even consulting and not being full time where my calendar has had at least 50 hours of sales calls or other types of calls or meetings with clients throughout the week. And I had no time to concentrate on the other things that I needed to do that were strategic for my business. I'm sure your calendar looks pretty similar. So I started to block my calendar. I block all my Fridays out. I only occasionally take meetings or calls and Fridays especially when they don't fit in the rest of the week or at something come up.
But I block all my Fridays and I usually block my Mondays so that I can start the week and end the week with some clarity. So maybe block your chunks of time on your calendar too. And it doesn't have to be whole days. I prefer to do that because it takes me a while to get into the groove, especially when I'm writing with this new book coming up. That's also a creative thing. I have to have the brain space to think about it. But for you, you might not need that. That might not be necessary to block a whole day. Why don't you try to block three or four hour chunks? Why can't you do a half a day? Maybe block out times when people aren't in the office or make an agreement with your team where you're like, this is just work time. I've worked with a lot of companies as well that are Tuesdays or no meeting days and then it creeps in.
Be ruthless about that. Don't let that creep in. It only starts to change when you say, okay. And I think if you make an agreement with the rest of your team and with your stakeholders and with your product team that says, You know what, if we put this on our calendar, remember, this is not that we're not doing work. This is important for our work. We need the space to actually think through these problems and come up with it. But I would encourage you to try that, especially as you're going through strategy related things right now, trying to figure out where to go, what directions to go in. That takes a lot of effort. So I don't wanna dismiss the time it takes to actually think through things. And don't get into the mode of where you feel like you can't take a break to actually do work.
Like thinking is still work in a creative pursuit like product management, it's still work. It's the same for designers and developers. We talk about how developers can't do their work well if they're in a million meetings, like product managers need meetings because they have to be collaborative. But you can't be in meetings all day and still get your work done. So block your calendars, stick to it. Be really ruthless about what meetings you have and push back on what could be a meeting, what should not be a meeting. And then also think if you're having so many meetings, what are ways that I can actually streamline some of this information? Is it requests for information? Did I create reports for them? Did I create a weekly email? Did I create a cadence? Think through why you have so many meetings and think about other ways you can get people that information.
If it's collaborative and it's a working meeting, great, keep those, right? But if it's just a status update are you lacking a roadmap tool where everybody can just log in and check it out? Are you lacking other sorts of information or cadences across your organization? Think through those types of ways of working first because it will free you up and this is what we need to do. We need to make sure that we're free to actually think. So I just wanted to put that out there cuz it was gaining a lot of traction on Twitter yesterday and I thought maybe that would be helpful for you guys to listen to too. But without further adom, we're gonna get to the main event here on the Dear Melissa episode, which is the questions. So here's our questions. And I wanna remind you too, before I get in here, if you do have questions for me, please submit them to dear melissa.com, Lead me a voicemail and tell me what you're thinking about.
I love hearing what's on your mind. Also, I also wanna remind you, I've been doing this for the better part of coming up on a year, two years. I've been doing this for a while. So there is a huge backlog of already answered questions that might answer your questions too. So if you had to dear melissa.com, you will also see all the old episodes in the questions I've already answered and maybe they can help you out. So without further ado, here we go.
Dear Melissa, I'm a technical product manager that has transitioned to the role from being a business analyst. My biggest challenge is being able to quantify the benefit that our platform transformation will deliver. in a consumer product world. We can measure more easily. For example, measure conversion right now, make a change, maybe run an experiment and measure it after. Pretty straightforward. Part of our transformation is to consolidate separate APIs, which essentially do the same thing for each of our products into a single capability, which is product agnostic. Reduced operating costs from deprecating the legacy redundant APIs is easy to measure, but we also wanna be able to reduce the speed to market by being able to affect all our products with a change to one service rather than changing one service for each product. The essence of a platform, we are sure this will increase speed to market, but how do you measurably prove it? You can't really have a speed to make a change as a kpi, although this is essentially the thing we wanna measure because different changes will take different amounts of time unless you make the same change on the legacy architecture that has the replicated services and the new platform, that wouldn't make much sense to do. This is just an example, but hopefully you get the idea of the challenge. I'd love to hear your thoughts.
All right, this is a good question and I think a lot of people struggle to measure the value of a platform change, but at the end of the day, platforms are incredibly valuable to the business, not just from a reduced cost perspective, which is key.
I think costs are pretty straightforward for you to measure, which is great, but let's talk about what the value is. It is exactly what you said, reduce the speed to market, but also the scalability and how many people you can actually serve. So it's not all about consumer products, we can just measure revenue tied to conversion rate, all those things. But platform products, you are tying that thing back to platforms. Do speed to market means unlocking revenue faster. So think about what speed to market actually means. It means that you can serve customers faster, you can realize revenue recognition faster.
That's really what you want. You could put that on there. Now of course you're gonna have to take into account the sales team leads and all that stuff, but let's say that you're trying to serve a new group in the market. You wanna expand geographically, You can't do that without the new platform or it's just gonna take 80 times longer to be able to serve different geographies on your old legacy systems than the new platform. Can you show how much faster you would be able to open up that revenue recognition that's speed to market? I don't think that's bad. You're concentrating on the, I think we're getting tripped up, is you're concentrating on the individual KPIs to make a change, right? Speed, to make a change. That's not necessarily the business outcome of what you're doing. If I can make a change, if I can develop a feature 10 times faster on this platform, I can make money 10 times faster in this market and I can also not get blocked out of the market by competition coming in.
That's the thing I think you need to actually look at. So let's take an example. Maybe your strategic intent is to go upmarket or your business business challenge. I talk about them in strategic intents. If you haven't been listening to this for a while, but your business challenge is to go up market. You are going to go into the enterprise market. In order to go into the enterprise market, you have to be able to release this whole suite of reporting tools for the managers of the enterprise and for the C level because they wanna know what they're getting for their product, they're measuring roi, let's say something like that. Easy, really common use case here in your legacy system, if you were to build all those features is approximately going to take, I don't know, two years to do that. But if you build this new platform, building out those features on top of it would actually take three months.
All right, so what do we have there? We're saying that in order to enter the enterprise market, it would take two years on the old platform, it would take two months on the new platform. That's a pretty big swing in the right direction. That's showing us that we can unlock value that way. But of course it's gonna take time to build that platform. All right, So it's not gonna take just two months to enter the enterprise business. It's gonna take cost length of building the platform time plus two months to enter the legacy business. This is also why we do platform transformations and why it's important to actually slice them vertically and think about how can you start developing on top of the platform without building the entire thing and launching it. Is there a way for you to do the groundwork that you need to do but slice it in a way where you can start to open up the platform for the most strategic things right now?
So you also wanna add that in there. And I think what you wanna show is how this is a hockey stick approach. It's not just a steadily growing approach, it is a hockey stick approach. So what you're showing is that yes, it's gonna take time to actually build out this platform and we could go a little bit faster as we're building it out while we're unlocking those vertical slices to start developing on top of it. So for example, let's say maybe you look at the company strategy and you're like, if it's going into enterprise, the most important thing that we could do right now is reporting tools. I'm gonna concentrate on the part of the platform that really can embrace reporting first and then we will migrate that into the other. Things like that will help us with that piece. If I develop that first, now I can start to put reporting on the new platform as we start to build out the other things.
But that allows me to start selling that into the enterprise quicker. That's the type of thing you wanna do. And I know that's not a 100% beautiful scenario and there's a lot of implications in that, but that's the type of mindset that you wanna take for a platform approach. Now you're showing that you can unlock value while you're building it out, and that's really important to do. So you wanna make sure that when you're doing these platforms, you wanna unlock value along the way as much as you possibly can. And I know there's probably gonna be a period where you can't actually release anything to the users for a while. You gotta do the nuts and bolts of building out the platform, that's fine. But you wanna show people how this unlocks rapid growth later on. Because remember, this is the connection here. You can't sell anything until you build it.
So show how you will be able to sell things faster. And that is speed to market, but that's okay. You can measure that right to do that. So how can you show things faster? Now, another part where we get a little tripped up it's not about measuring this down to an absolute T, you said, Oh well I can make a change on the old platform and then make a change on this one and compare it. Generally measure right now, and you can do this by pulling information outta Jira about ticket times. If people are good at actually updating their Jira tickets to starting work, complete work, all that stuff. Take a couple sample examples out of Jira on the epics you've worked on or the big ticket items and then pull that out and measure when did it start and when did it complete? And then compare that to how much faster you think it will be, right?
Say it took us about two months to develop this big feature. We believe with the new platform it will take us one month. Okay, that's a 50% chance of 50% faster speed to market time, Okay? Do that, right? You don't have to get down to a T, just generally measure things and say, we believe it's going in this direction, Pull the information out, Jira time, some of the stuff that you've done before, and then think through realistically what's your hypothesis on how this will make it faster. We still need to think about good architecture cuz it helps us scale too, so maybe show as well. Here's another one I I've done a lot with platforms show when things break, if we keep going with our legacy, we can only handle X amount of clients per year onboarding them or servicing them. If we wanna move into new markets, if you wanna move into new markets faster, we go to the platform and we'll be able to service X, Y, and Z number of customers more.
It shows that your platform's gonna break and it's not scalable, but I don't think we have to get into super specific conversion metrics when we're talking about platform transformations. You wanna generally show that the current state is this and you can measure that right now. Go back historically into your tickets and figure out how long things took in general. Take a size out, just be like it's a medium size ticket, took X, Y, and Z to build a medium size feature, a large size feature, blah, blah, blah. Look at our strategy. There's a huge numbers of features on here that would take us years to be able to complete. This will take us much longer, shorter time, that's okay. That's a business case that you're gonna make here. So I really won't get hung up on these super specifics on here as speed to make a change. Talk about how this enables the business and you'll get the buy-in that you need. What leaders care about how can I enter markets faster? How can I do this faster? How can I get these things out there? So don't worry about that. Don't go go too nitty gritty into the details.
Right. Next question. Dear Melissa, My question is about prototypes. How do you rapidly prototype features that rely on Personization as the value driver without creating the actual product? Do we just build personalized prototypes per test user? So with this one, it's important to remember what are we actually testing with the prototype. This is why I am a big advocate for a series of experiments before you actually land on what you're going to build. Now, when something is personalized, and I think this is a pretty frequent question for a lot of people who are doing personalized products or things that rely on algorithms or showing up exactly what these people need a great test for, that could be a concierge experiment, right? Where you are asking them what are the outputs that they need to see, what taking all the criteria in that the algorithm normally would do, and then helping to surface up the data that is relevant for them.
Typically, in a concierge experiment, what you're doing is delivering them what they want to see, but it doesn't look like the real thing. That's where I think you test if the criteria for the algorithm is correct if you are getting the right inputs for it, if you're surfacing up things that they're actually want to see, that's where you're looking at really the contents that would go into the prototype. So big fan of that type of experimentation. Then I think what you do is you typically move on to a prototype like you're talking about, where you mock it up and then show it back to people and let them click through it. I think you are right on the right track here. Do we just build a personalized prototype user? If you're trying to test if it looks good and the flow's good, yes. That's kind of what I would do.
But yeah, you would just take the prototype, you would use the inputs that you got from the concierge experiment and then start to display them in the way that they would look like on the prototype. At the end of the day, that's the best way to do it. Get the feedback there, let them click through it, show them what would happen. The idea here is that you wanna get to a good confidence level that what you're building is the right thing to build, but it doesn't have to be a hundred percent confident where you're like, I showed them absolutely everything on the prototype, and that's gonna, how that's manifest exactly on the screen. You wanna just get into directional evidence that what you're building is the right thing. So what I would do is probably run that test with a group of people, maybe like 10 to 20 of a good subset of your users who represent. Different types of personas or user patterns that people might have that use your product that are different, and then get that feedback, digest it, and then go for building the whole thing, and then keep iterating as you build it along the way. But yeah, that's how we would do personalized prototypes.
All right, last question. Dear Melissa, I work at a small startup as a UX designer. Our product team consists of me, one of our co-founders as PM and two engineers, we're trying to structure our work using the product kata and drive towards an outcome. My question is this, how do we balance outcome focused work with functional work like ensuring our registration process is functional or experiences that we think are strategically crucial? As an example of something that seems strategically crucial, we are building a reading app that uses chat fiction formatting as a differentiator. This format requires extra editing work to be done over a standard book format. We currently don't have a standard ebook format, but are considering adding one because it would allow us to publish more books faster and easier. Increasing our content library is important to our growth, but it doesn't align exactly with our current focus of increasing our conversion rate of visitors to registered users.
How do we balance or prioritize these things? So here's one thing I don't think you need. You have to remember, it's not just you don't have one goal as a company. Usually you have two. So you've got two goals. It sounds like the second one is just lower on your priority, but I think you just wanna weigh how fast it it is to get that ebook formatting out there and will it help. I would also say this is outcome focus work, what you just described to me. So you just talked about adding a standard ebook format in there to publish more books faster and easier. That's totally an outcome focused thing. You just talked about all the outcomes to me. Publish more books faster and easier, Unlock all that for your customers. Great, cool, right? Outcome focused. Now that foundational work too is important. And foundational work is usually outcome focused.
The thing is that we're not an experiment mode when we realize things are foundational. You have to think about the big picture. So foundational work is outcome focused. You have to just think about it as a level up. When I do this foundational work, what does that enable? And I think you have to associate outcomes at a higher level with that foundational work. So wrap it up, right? It's not just about building the nuts and bolts of a platform, it's about what that platform will actually do at the end of the day. So if you've got, I need to put an ebook format in here and you're thinking that's foundational to be able to do the editing and do all the stuff on top of it, bring it up a level. It sounds like to me, adding this standard ebook format is going to unlock you to publish more books faster and easier.
And I feel like you kind of solved your question in <laugh> in writing this question to me, I feel like you answered it. You know what to do here. So let's balance, Let's talk about balancing and prioritizing, cuz I think that's more important. I think your mind is at the right spot about outcome focused versus not. Foundational work is usually outcome focused because if you don't do that foundational work, you won't reach your outcome. So tie it back to what it unlocks for you. Totally. Okay. Okay, problem solve there. Second part, how do we balance or prioritize these things? You still need to think about what are the overall strategic business challenges that you're going after? So if number one is increasing your conversion rate of visitors to registered users, that means that the majority of people or in your organization should be working on that.
But that doesn't mean that people don't work on number two, which is increasing your content library. So you need to think about distributing this as a portfolio. So as an organization, what we would really be looking at is maybe 60% of our time is spent on the first one, and I don't know 30% on the second one and 10% on something else, which could be bugs or other things. But I'm throwing out numbers here. It has to be right for your organization. But what I'm saying is that it's not a one size fits all for just the first goal. You can be working on multiple goals.
So you're a small startup and having one laser focused goal is really important. But also, think about it this way, if you're focusing on increasing your conversion rate from visitors to registered users, those people want content, which I think feeds into your conversion rate. So at this stage when you're small, usually conversion rate is almost like a lagging indicator. What are all the things that make people convert? Think about that. Is it your content library? Is it having more stuff on the content? Is it the ebook? I think you need to pull back a little bit and think about what are all the things that get people to convert? And sometimes it's not just optimizing the front end of your site, it's not about just funnels on that piece, it's about the offering of your site as well.
And this will become really, really important as you start scaling. You need to think about your whole offering, not just the individual pieces of your components. And I think that might be where this is getting a little tricky. So definitely take a couple steps back and think about the entire experience. You're the principal UX designer, fantastic. So you're the right person to think about this. Think about the entire journey. Think about what's valuable to the user and what's going to continue to be valuable to users as you scale. When you're small, usually you're optimizing for the first batch of people, but you also wanna be thinking about further ahead. When do things break, right? If I get to 10,000 users, does my content library break for them after six months, then it would be like, Yeah, you definitely have to increase your content library, but this sounds like something that you should be working on now to load it up so that when you get more and more users, they have things to go actually read.
So I wouldn't think about these in isolation. I think they actually feed into each other. And I would say too, it's not you have to finish things when you're prioritizing stuff. As a small startup, you wanna be laser focused on the thing you're working on right now to finish it, release it, and look at it. But you also wanna be thinking about what other factors contribute to the success of conversion rate. What other factors contribute to the success of getting more users? So it's not just about optimizing the beginning of your funnel, it's about actually thinking through your whole product and making sure that's solid. So this to me sounds like a critical thing that you should be working on as well. I don't know what the other things you have are that are maybe higher priority, but think about prioritizing them with what's the first thing we need to do that contributes to this?
How do other things feed into conversion rate? And not just how do we win right now, but how do we set ourselves up to win for the next five years? When does this break? That's the stuff that you need to be prioritizing as well. All right. That is it for dear Melissa this week. Thank you for listening in. And if you have any questions for me, I just wanna remind you, please go to dear melissa.com. And if you enjoyed this podcast, please subscribe to the Product Thinking Podcast wherever you listen to podcasts. Thanks and have a great day.