[00:00:00] Brian Balfour: For a lot of these AI tool systems to take place is one is I think there's the business context data. And that's, where do we store information? What is our strategy? What is the vision? What is the goals? What are the initiatives? What is the org chart? All that kind of stuff. Where are we storing that? And how are we making that accessible to this new tooling? Because the more information that has about that, the more things that I think these AI tools will be able to do with it.

[00:00:35] Andrew Michael: This is Churn.FM, the podcast for subscription economy pros. Each week we hear how the world's fastest growing companies are tackling churn and using retention to fuel their growth.

[00:00:48] VO: How do you build a habit forming product? We crossed over that magic threshold to negative churn. You need to invest in customer success. It always comes down to retention and engagement. Completely bootstrapped, profitable and growing.

[00:01:01] Andrew Michael: Strategies, tactics and ideas brought together to help your business thrive in the subscription economy. I'm your host, Andrew Michael, and here's today's episode.

[00:01:12] Andrew Michael: Hey, Brian, welcome to the show.

[00:01:14] Brian Balfour: Thanks for having me again.

[00:01:15] Andrew Michael: It's great to have you for the listeners. Brian is the founder and CEO of Reforge, providing professional education programs to experienced practitioners and more now, which I think we'll touch on a bit in the show. Brian is also a previous guest of the show and was actually the very first episode of Churn.FM more than five years ago now. So it's been a while and prior to Reforge, Brian was the VP of growth at HubSpot and his essays during his time there have become the foundations for what most startups still structure their growth teams initiatives today. And Brian is also a venture partner and investment advisor to many startups.

[00:01:47] Andrew Michael: So my first question for you today, Brian is actually relevant as well, because normally this would be the last question I ask at the end of the show, but since it's been such a while since we spoke, I'm interested to ask, what's one thing that you know today about churn and retention that you wish you knew before you started Reforge? Because obviously you knew a ton about churn and attention before Reforge, but what's one thing that you've learned today since then that's changed your perspectives?

[00:02:12] Brian Balfour: I mean, this is a weird way to answer your question, but it's just like, I just don't think you can break what I call like the foundational laws of retention. And I kind of learned that a little bit the hard way on some of our Reforge initiatives. And by the foundational laws, I mean, you know, a lot of the things that we've talked a lot about for years and reforges that retention is really grounded in whatever your use case is.

[00:02:38] Brian Balfour: And by use case, I mean, you know, what is the problem you're solving? What's the alternative? Why are they choosing you over the alternative? And the most importantly, like what is the natural frequency in which your customer segment experiences that problem? And that really dictates your retention metrics, like how easy it is to build a habit, how you wrap a business model around those elements and all those pieces.

[00:03:07] Brian Balfour: And so I think that, you know, we tried some things with Reforge that tried to break those foundational laws, meaning, you know, a lot of the use cases in Reforge are on the more episodic lower frequency side of natural frequency. That's just kind of fundamental to the nature of professional education, education as a whole. And we really tried some experiences that would essentially help, you know, increase that frequency.

[00:03:36] Brian Balfour: And at the end of the day, we just weren't able to like really move the needle on it. And we invested a lot of time and energy into it. And which I wish I could totally take back. But I think this is an important lesson because I know we're gonna talk a lot about AI. Maybe I'm jumping the gun here, which is I'm sure we're gonna talk a lot about how AI is changing things. And that's what everybody's talking about. That's what I'm talking about mostly when I'm, as I'm writing now.

[00:04:05] Brian Balfour: But I think it's really important to stay grounded also in the things that the principles and the things that we've learned around product and growth the past couple of decades that I don't think will change. And I think this is one of the places that doesn't change. Like retention's still going to dictate, you know, the winners in every category, that retention is grounded in your use case, that use case is grounded in your natural frequency. Like you gotta align all of the things to that and sort of build from there.

[00:04:36] Brian Balfour: And I think a lot of the first wave of AI products, like, you know, we went through a relearning period around this where I saw a lot of AI products attack use cases that like with, that just like weren't high frequency enough. And as a result, they have, and still have like quite a bit of churn. That's probably one of the biggest issues in, you know, these new AI products is that they are experiencing incredibly high churn and I think this is part of the result is that a lot of them are ignoring these fundamental laws.

[00:05:11] Andrew Michael: Yeah. I think it's a very interesting topic. I think like, I can't remember where the analogy came from, but it's sort of with the natural frequency is like trying to change user behavior is incredibly hard and trying to change the natural frequency of user behavior is like even harder. It's like, brushing your teeth. Like you, most people do it twice a day. Like you're not going to get them to do it 10 times a day. And if that's like a requirement for your business to work, like that business is never going to work sort of thing.

[00:05:35] Andrew Michael: So it's like, I do think though, like there are certain behaviors that AI enables to change now and people can get a little bit more creative from that perspective, but it's only to some degree and they're like always needs to come back to like the first principle. And like, as you say, the use case and then the natural frequency of that use case. But yeah, it's very interesting that obviously like, I think reforges like retention courses, probably hands down, one of the best courses I've ever taken. And on the internet, like if anybody's looking to do something, I think like that's definitely where I always point people towards.

[00:06:05] Brian Balfour: That's funny. That course we, well, one has probably been way more evergreen than I would have expected when we first created it. But two, like we're constantly updating everything, but that is also one of the ones that we've needed to update the least, you know, compared to some of the other topics, like with all of things emerging and so it's just interesting. I don't think I would have predicted that when we first created it.

[00:06:28] Andrew Michael: Yeah. No, but I think, cause as you say, it's like, it goes back just to the basic fundamentals of user behavior and to like, just thinking through what does success look like, what is the use case and like, that's really all churn and retention is at the end of the day. Like people ask me like, what's the biggest thing is like, if you have a problem, they come to you with a problem. If your product solves that problem in the solution and time, I mean, like they're going to stick around. And if it doesn't like, and you just go through all the different steps and how you get to that point, so highly recommended.

[00:06:53] Andrew Michael: But you mentioned talking about AI as well. And before we get there though. Obviously you just announced a new acquisition of Monterey AI. I'm keen to understand first of all, like the thinking behind that, like, does this have anything to do as well with what you've learned as well when it comes to Reforge and the side on retention and getting people to repeat usage? We'll start there.

[00:07:13] Brian Balfour: Yeah. I mean, look, I originally created Reforge because I was really interested in the frontier of practices around, you know, our different functions, particularly product and growth. And we did a lot of that through like education, but one of the biggest things we always heard from our customers over time was they really wanted from us something that bridged the gap between the learning and the implementation in the organizations.

[00:07:42] Brian Balfour: And we never went there in kind of the first phase of Reforge. But now we're kind of going through in even more, like a much more monstrous shift than where Reforge started, which was around the emergence of the growth discipline, and which is that we fundamentally believe that AI is going to change how, is changing how product teams kind of think, work and build.

[00:08:07] Brian Balfour: And a huge part of making that transition to what we call AI native product teams and what those new things look like is two parts. It's going to be both the tools, but just as important is the knowledge and the skill and the behavior change that needs to come along with it. And so we started going down both of those paths and now we're not only helping on the education side, but really investing in all of the tooling that we think will support how these new like AI native product teams will work.

[00:08:44] Brian Balfour: And so our first product is what we call insight analytics, which is, was through an acquisition of a tool called Monterey AI. And really the problem that it solves is that over the like years, basically we've started collecting way more qualitative information about our customers, whether that's from like sales calls, customer support tickets, success calls, things people are saying and talking about us on Reddit to their user research, like all these things. And this is really valuable information, but it's fragmented across not just a bunch of tools, but all of those tools all have different owners, like they're typically owned by different functions.

[00:09:19] Brian Balfour: And this is, this creates what we call the feedback fragmentation tax which is there's all these taxes that emerge from this. The most one that most people are familiar with and can see right away is that product teams just spends all this time hunting through all of these different tools, trying to piece together information about their customers. And typically it takes so much work and so much friction that people just abandon the process.

[00:09:43] Brian Balfour: But there's these other problems too, which is like this game of telephone happens within the organization, where all of the organ, like every single slice of the organization that owns one of these slices of the data set kind of summarizes their slice and like starts to pass it through the org and it just kind of goes through all these filters. But the most important one for the product teams is that because of that, every part of the org kind of has their own belief on the customer because of their slice of data.

[00:10:11] Brian Balfour: And as a result, you know, these product decisions and product roadmaps end up being all of these like defensive negotiations within the organization. And that's a big problem because the product team is the one responsible for making the product decisions off of like a 360 view of the customer. And so anyways, the product solves this. We ingest all of these sources of data. We aggregate it all in one place. We use AI to do a bunch of analysis, enable a bunch of querying through natural language. But most importantly is we also help you act on the data. We are able to get that data into tools like Linear Jira alongside Notion wherever you're writing your PRDs and using AI to help incorporate it into that workflow. And so that's the first piece of it.

[00:10:58] Brian Balfour: But the last thing I'll mention about this is that I think that the reason we're so... that we've decided to really focus on this is that I think we're actually underestimating how much AI is probably going to change how we develop products. And the reason is if you look back at the shift between from on-premise to cloud, you can actually go through all of these categories of things that happened over time.

[00:11:25] Brian Balfour: It changed the development methodology. It used to be waterfall, it went to agile. It changed the tools that we used to support that process. And we built this extensive ecosystem around version control, with the GitHub, the cloud IDs, like all this stuff. It changed the monetization models from like transactional to subscription. It changed the growth models. It enabled PLG. Changed how the metrics that we use to measure our business. I could keep going on and on, right?

[00:11:50] Brian Balfour: And that happened over the course of a couple of decades, but I think we're gonna see the same thing with AI, but I think we're gonna see it on a condensed timeline. And as a result is like, AI is not just about new features and new products. It is going to change the way we work, the way we measure success, the way that we work together as a team. And that's just gonna require a lot of reinvention, a lot of evolution.

[00:12:16] Andrew Michael: I see from my own experience, like how crazy things have changed. And I think like in the past, what would typically like require three to four engineers to build a product, like I've literally built products by myself in months, like in a month, rather than like six to 12 months where you would need a full team. And in that as well, I've also been thinking about all these different processes that change.

[00:12:37] Andrew Michael: And like the other day I was chatting to some friends and I was like, where they were talking about Figma. And I was like, I think almost like Figma becomes redundant in this new world where it's like people are just preferring to design in code now because it becomes a lot more easier. And like, this is one example and then [inaudible], you're an idiot, it's bullshit. And then like Paul Graham put out a tweet, I think something like this, something on those lines. And then they were like, okay, oh, maybe there's something what you were saying. But I think definitely like things are changing incredibly fast.

[00:13:04] Brian Balfour: I'll take the counter take on that by the way. So my guess is there's space for both, right? In the sense that I think we're still in the phase where the largest early adopters are certainly people that are more technically savvy. And as a result, the adoption by developers or those with coding knowledge has been quicker. But if you look at like humans and professionals as a whole, there are way more people that are visually oriented in the way that they think. Like that's their starting point.

[00:13:39] Brian Balfour: And to the extent that Figma can enable those people to go from, you know, getting their idea out to a visual and going to code versus starting with the code and the prompt. I think there's huge room for that as a result, just because of human nature. So my guess is we have both, but the lines are blurring. The lines are blurring for sure.

[00:14:01] Andrew Michael: Yeah, I'd push back as well on that. I'd like if you've used tools like v0 and things like this where you can literally like give a prompt and you can view why it's like literally that. I think like, and maybe there's a space for like this generative AI image generation. But like, I think when we think about like user experience in UX, it's probably just becomes easier to say like, hey, click this button and make this go here, rather than trying to like wire up prototypes in Figma and then trying to share a link and like.

[00:14:27] Brian Balfour: Oh, yeah, I don't think it's gonna be like wiring up prototypes and stuff. But I just think about, I think there's a whole segment of the audience that has, that finds it easier to sketch out their idea than it is to describe it in words. And so to the extent that they can enable that interface- I think, but the thing is you're pointing out is like that interface can come from multiple directions. Like it's not like Figma owns that, right? It's just, so it's a...

[00:14:55] Brian Balfour: Dylan Field, the CEO, did just do an interesting podcast on Invest Like The Best, where he talked a little bit more about it from the design perspective. He has some, it was some pretty thoughtful stuff around like design orientation in this world. And so it's a, it's a good listen.

[00:15:11] Andrew Michael: Nice. We'll definitely leave that in the show notes. I'll pick it up and add it. Yeah. I think at this stage, like nobody really knows like what's going to happen and how things are going to evolve, but you're taking obviously this bit now from a product perspective. What are some of the other things like you feel are going to be changing now in this new paradigm and this new like AI native teams, product teams?

[00:15:32] Brian Balfour: Yeah. Specifically on the teams front, I think I find it helpful to first like start with like a previous shift and think through all the categories of change. And then that gives us a starting point on the categories of things that we think might be changing. So I kind of mentioned a few of them previously, but like, once again, if we go back to the shift from on-premise to cloud, the categories and places things change was the development methodology, the tools that we use, the monetization models, the growth models, the measures of success, what defensibility basically, what types of defensibility mattered, skill sets and roles in team and organization design.

[00:16:11] Brian Balfour: And so my guess is, you know, I don't have all the answers to how all of these things change specifically, but I think there's some early signs around some of these. So like, what are some of those early signs? One is that I think a lot of people are talking about from a development methodology perspective about the productivity like you mentioned. And I do think that's true. Like it will help us move development cycles faster. But one of the things that Scott Belsky, the Chief Product Officer of Adobe pointed out that I thought was really, really good was that, that AI can not only enable faster cycles, but it can actually enable us to explore a lot more paths, like within that cycle.

[00:16:59} Brian Balfour: And the reason that's powerful is, I think kind of historically, we've been really limited on the paths that we can explore around a problem to a solution. Just because we always inevitably run into time constraints, resource constraints, something like that. And a lot of the times, we just kind of don't even take the time to explore. You know, we're just like, I don't have the time. I'm just going to go with this solution. But what AI could potentially do is enable us to explore many paths and parallel around these solutions.

[00:17:28] Brian Balfour: And I think in terms of getting to a better product, getting to a better solution, that exploration process is just as important as the number of iterations, like how many cycles you can get in. And so I think that's like a really, and I find that even in myself, is like for developing a course or just even potentially like writing a blog post or a new landing page.

[00:17:57] Brian Balfour: Yes, I'm doing it more efficiently, but I'm actually the bigger change that I'm seeing is that I actually explore many more ideas within that timeframe than I do get gains on like the total time that I'm saving like from start to finish. So I think that's like one.

[00:18:16] Brian Balfour: The second one which I'll just mention because I think it's probably a lot of people have talked about it is like this idea of rather than going from idea to doc, idea to prototype. And I do think that's powerful. I kind of [inaudible] a LinkedIn post about this, which is, I hate annual planning. I hate it because it just ends up in this document death spiral of like, oh, like we're all, all these docs are written. We're reviewing the docs. We're debating the docs. It's just like, we're debating all of these things in words.

[00:18:46] Brian Balfour: And I was like, oh man, what if instead of, we were debating 20 different docs, we were debating 20 different prototypes? And I'm like, ooh, now that would be interesting because I love my feeling, seeing, touching, playing with it. I think it just like turns into much more productive conversations. And as a result, I think could like turn annual planning from something I would love to avoid, you know, like the plague to something that I would actually like look forward to as, as like an exercise. So I think, I think we're kind of seeing changes around that.

[00:19:20] Brian Balfour: We're seeing changes around the redefinition of roles on teams. Those lines are blurring. The engineers doing product work, product people are writing code, product people are writing some of the marketing copy. All these things are blurring together. And I don't know where this will eventually end up in new team formations. And I don't know exactly what that looks like, but principally, I think what it could do is help us maintain what I call the magic of startups which is in a startup, you tend to produce a lot more product with a lot less people and it feels like magic.

[00:19:57] Brian Balfour: But it's actually like not magic at all. You get these... basically startups create the constraints for super tight feedback loops. And those tight feedback loops create like, quote unquote, founder intuition. Everybody on the team does a little bit of everything because they have to. The constraints are what make that possible.

[00:20:16] Brian Balfour: But then over time, we add all of these specialized roles which creates all these layers of communication. It slows down feedback loop between the builders and the customers. A lot of this intuition over time is replaced by documentation and process. And so I think a lot of what AI can do is help maintain that magic, that quote unquote magic for longer, which are these super tight feedback loops.

[00:20:43] Brian Balfour: And then the last one I'll mention, we can kind of keep going on this to the team one, is that I do think we are going to see major retooling around product teams. And that's because in the past 15 years, we've essentially gone through this like chaotic accumulation of very specific purpose built tools. And so if you look at any mature product organization, it's like feature flags over here, analytics over here, customer feedback over here, you know, tickets over here. Oh, more analytics over here, because everybody always has three analytics tools, right? Like at least three, like that's the rule.

[00:21:22] Brian Balfour: And it's just, that will not work in an AI era because the biggest problem with AI and AI agents right now is what Dharmesh Shah, the founder of HubSpot, calls the compounding error problem, which is that these systems need to be sitting on top of really high quality data. And that's because an agent that's using an LLM that's based on probabilities and predictions, none of those probability predictions are a hundred percent.

[00:21:51] Brian Balfour: So even if like every single task they are generating an output for is 95% quote unquote correct every time, but they're doing that across a dozen different tasks that error rate compounds to the point that the end output there's only like less than a 50% chance of it being like high quality or high correct. And the thing that feeds that compounding error problem is fragmentation of data systems and tools. So I think we're going to see a major retooling there. So that's just a quick list, but I think there's more.

[00:22:25] Andrew Michael: I think the Scott Belsky point is very interesting and it's actually something like previously on the show, we had Mohannad Ali, who was the previous CEO at Hotjar and we're talking a little bit about like AI and obviously like competition increases exponentially now, anybody can build software. And like, we sort of like, I think both agreed at that point with that, it's like, there still is like, it's not easy to build really good software. Like there's this craft and perfection and that you need to keep going back. And that was my initial impression back then, is that like, anybody can still build software, but not anyone that will be able to build great software. And it requires like this dedication to the craft.

[00:23:02] Andrew Michael: I'm slowly changing my mind on that as well, a little bit, just how good the models are getting now and letting them know, I want you to have a really good UX and then I think at some point, like the UX will be on point from that. But this point that you make as well around like exploring multiple different paths and channels comes into that side of like creating really great experiences. And I think that is definitely one thing that at the surface level sounds very interesting. I wonder in practice, how many companies are actually doing this today?

[00:23:33] Andrew Michael: Because it's, yeah, it still doesn't feel this stage where like, okay, we can explore it still feels like the pressure is on, you need to build, you need to get started, you need to keep moving. And I wonder if like, because of this rate of change that the notion is nice, but just people will not be able to do it because they won't be able to keep up with what everybody else around them is doing.

[00:23:53] Brian Balfour: Yeah. So there's this whole other post that I need to write because I've been collecting anecdotes on this from a bunch of product leaders and product teams. And so there's the tooling problem, but the bigger problems right now of trying all this stuff, there's all these hurdles.

[00:24:11] Brian Balfour: And just some of the hurdles are, one is it's essentially a time hurdle. It is like, nobody actually feels like they have the time to take and explore the different ways that they can potentially integrate this into their product development process. And because it does require exploration, it does require tinkering and playing and starting to get in. And so that's one problem.

[00:24:39] Brian Balfour: A problem number two is honestly just straight up behavior change. We are freaking humans. And a lot of people in product now have been doing this for over a decade or more. We have gotten used to like ways of doing things. And I don't underestimate how powerful those ingrained habits are in the resistance that people will put up to making that adaptation. So that's problem number two.

[00:25:14] Brian Balfour: Problem number three is like, the larger you go, the more risk averse people are on the security side of things. And so that also [inaudible]. And so I think, you know, I'm starting to collect these anecdotes of teams that are really starting to think on the front lines. And I think a couple things are starting to emerge.

[00:25:36] Brian Balfour: The first and foremost is that you can't boil the ocean if you want to get started on this. I think the smartest teams are picking very specific use cases in parts of their product development cycle to essentially use AI and change how they're doing things.

[00:25:53] Brian Balfour: The most common one that I'm seeing right now is certainly the adoption of coding tools to, like GitHub Copilots or Cursor or something like that to help developers be more efficient. And just focusing on that, just getting your developers used to that, just getting the adoption across the board, like all that kind of stuff.

[00:26:13] Brian Balfour: Or of course, the second one is what we're doing with insight analytics, which is get rid of this chaotic mess of all your customer information and qualitative information across the place and really start to get your team using all this feedback at the different points of the product development process. AI can help aggregate, clean, synthesize, generate a novel, all that kind of stuff in that.

[00:26:39] Brian Balfour: So that's, I think, one is like you have to pick a very specific starting point versus if you sit on Twitter and LinkedIn all day, reading these things. All these ideas. You're like, Oh my God, like I got to redo everything like right out of the gate. So I think that's number one.

[00:27:00] Brian Balfour: And then I think number two is probably, and I actually go back and forth on this is that people start in smaller parts of the organization and kind of build some success use cases and then spread out. But the risk of that is it kind of creates this dynamic of like, this is the AI team. And since we have a team focused on AI, like the rest of the team doesn't need to worry about it. It was kind of the same dynamic with growth, right? It was like, oh man, I now have a growth team, so I don't have to think about it. Which wasn't true.

[00:27:33] Brian Balfour: So anyways, like I think that yes, like there are a number of hurdles that I'm starting to like synthesize around that go way beyond the tooling that will make it tough, that will be friction to doing this. And I think this is just where leaders need to stand up and be like, yep, this is going to be high friction, but it's important and we got to get started.

[00:27:56] Andrew Michael: Yeah. And I think like on that growth team sort of notion, actually like at Hotjar back in the day, like the way I convinced David to start a growth team was to call it the experience team because like he was adamant that like growth is everybody's responsibility and it's not just an individual teams. And I see the same thing, like as you say, happening now with AI teams and like this really should be something adopted across the org and figuring that out.

[00:28:21] Andrew Michael: The other mentioned thing you mentioned as well then was this idea around like creating this context layer within your organization, because this is all something like, I think LLMs are incredibly powerful, but the real value that will be created now for companies is really having this good context layer about your business or around your tooling and across your stack.

[00:28:39] Andrew Michael: I'm keen to double click a little bit more into this tool stack then that you see evolving now. And obviously like there's the candidates we see today, like Cursor is being adopted in engineering. And what are some of the other sort of ways that you're seeing companies now consolidate with their tool stack to enable LLMs to take advantage of their underlying data?

[00:29:00] Brian Balfour: I think we're very early in this. And part of it is because, is that the data that we need, like the data that these tools could use to really thrive, there's a couple things about this. One is that most orgs don't have their data in a great place. The second thing is there's actually new data that needs to be collected and stored in new ways. So when we say data, most of the time, what people are thinking of is like, oh, our product usage and customer data living somewhere in like Snowflake or a data warehouse or something like that. But there's actually...

[00:29:39] Brian Balfour: For a lot of these AI tool systems to take place is one is I think there's the business context data. And that's, where do we store information? What is our strategy? What is the vision? What is the goals? What are the initiatives? What is the org chart? All that kind of stuff. Where are we storing that? And how are we making that accessible to this new tooling? Because the more information it has about that, the more things that I think these AI tools will be able to do with it. So there's that segment of information.

[00:30:07] Brian Balfour: There's also just like the set of information about like communication, you know, between our employees and having that kind of context on what's going on. So, and then, sorry. And then another category of information is essentially the external information, like what's going on in our market, what's going with like our competitors, what's going on with like, I don't know, like industry trends, different things like that. That information can also feed these tools to help informed decisions.

[00:30:37] Brian Balfour: But most of that stuff is typically lying around in some random Google Doc that hasn't been updated in like two years. Right? So anyways, there's that whole layer there that I think is very interesting to think about. And I think what I see a lot of these AI tools doing is that they enable you to upload some of this information. But I look at that and I'm like, ah, man, our company is really going to want to upload and maintain this information across 10 different... Like all their different tools? No, probably not. So I don't know what's going to happen there. But it is a problem that I'm seeing emerge.

[00:31:16] Brian Balfour: I think the other things that I find interesting is that I won't name names, but I've talked to a few people who are pausing on the adoption of some product analytics tools. And I actually think it makes sense. Because what they see right now is like the existing product analytics tools, but then all of the new things. And they're like, well, the new things aren't quite there. But if they work, then I have... adopting these existing things makes zero sense.

[00:31:50] Brian Balfour: And so as a result, they're like, I'm just going to wait a little bit. I'm doing fine with whatever I have. And I think that's fascinating. I think that's a fascinating point too. So... And that makes me think like what other categories is that happening in as well, from the customer perspective? But I don't know, these are some of the dynamics that I'm seeing.

[00:32:14] Brian Balfour: And sorry, the last thing is, this kind of gets to a different point, which is, I was talking about this like toward, like middle of last year. And I was like, okay, well, a lot of these tools are, for example, like developers, in developer, like Cursor and stuff are writing a lot of the code for us. Then what happens when you need to go adopt something. I don't know, something like feature flags as an example. Who's making that decision? Is it the human or is it the AI code writer that is suggesting you implement here?

[00:32:51] Brian Balfour: And then the Perplexity founder came out and did a podcast and was like, oh yeah, I think agents will... he made a good point. He was like, no human loves doing a vendor analysis and adopting it. And so he's like, maybe we'll have agents being the one doing the adoption process. And that got me, and that makes me think, okay, well, all of a sudden, like if I'm no longer marketing to humans, how do I market to these agents that are informing the decision? And that's like a whole new, that's a whole new world.

[00:33:19] Andrew Michael: I think that's a whole new debate going on. Like I've been chatting about this. And I think one of the things that SEO used to be very important, like for your on-page sites. And now I think like, it probably feels, and like this would be the bet is that, the more places you can get your brand exposed, the more likelihood you have of showing up in these other LLMs.

[00:33:38] Andrew Michael: Because, at least today, the way they're working in from the observations is that you have a query. It goes, searches the first 10 blue links. It creates a summary and then it makes suggestions. And like, you don't only need to be like number one and two anymore. Now you need to be everywhere. And there's many places possible to increase the likelihood of showing up. But I think that will also evolve now over the next few months. It's very hard to predict.

[00:34:00] Andrew Michael: I see we're running up on time. So we talked about something before the show. I'm keen to hear your thoughts on it as well though, is the notion of product market fit now disappearing? And I think you touched on it a little bit as well, like with these product analytics tools now where people are holding off because of these new wave of tools and then existing services just become redundant almost overnight. Can you share a little bit more about your thoughts and like how you see product market fits evolving now?

[00:34:23] Brian Balfour: So I don't think product market goes away. What we've seen a few examples of, and I think we're going to see more of is what I'm calling product market fit collapse. And so a framework that Fareed Mosavat and Casey Winters came up with a few years ago in our product strategy program is this concept of the product market fit treadmill, which is it's not like product market fit is this static point. It's this thing that you reach, but you need to like, that it evolves. It keeps increasing over time because customer expectations evolve and increase over time. So you have to keep up with it. And if you don't keep up with it, then you fall out of product market fit. And that's why it feels like this treadmill.

[00:35:09] Brian Balfour: And I think in previous technology shifts, customer expectations have accelerated. They do change. But the increase in the slope of that curve was more gradual. And the reason is because a new technology is released, products need to be built on top of that technology, those products that need to be distributed and adopted. And as a result, customer expectations change. And the timing, the year, like it was years in between those steps.

[00:35:40] Brian Balfour: Like if you just even think about the shift to mobile, the biggest friction item is it took many years for the adoption of smart devices with high speed enough connections to enable like the use cases, right? And as a result, it gave products that had really strong product market fit time to adapt. And that informed how you would deal with that situation, the strategies you would take, the portfolio of bets that you might place to adapt to that over a period of time. But what we've started to see is actually the timeframe of those steps actually really condensed, like really fast into the matter of months.

[00:36:18] Brian Balfour: And the most recent examples of this have been like, Chegg, for example, it took only nine months from the launch of ChatGPT for them to lose 90% of their market value. 90%. And sitting today, they're like valued at something like 150 million in market cap, even though the last 12 months, they produced 600 million in revenue and they've got cash on the- it just, you add it up and you're like, oh, the market thinks this is going to zero.

[00:36:44] Brian Balfour: And the reason is that is that their core value prop, their core product market fit of high quality answers from curated humans around homework just completely broke overnight in the sense that through ChatGPT, I could actually for many topics get personalized instant answers that were really good enough and I could interact with it like on a timeframe. And as a result, they lost a half a million subscribers in a very short period of time.

[00:37:11] Brian Balfour: And now they're in this tailspin. Because their core growth model was designed around, okay, I get more subscribers, I fund more human curated answers, which helps me bring in more subscribers and those types of things, but that's now in a tailspin. And so now that I think that also happened to Stack Overflow, where their growth model and the incentives to contribute really started to break. And I think we're seeing it happen with Shutterstock and Getty who just did a defensive merger around this stuff as well.

[00:37:42] Brian Balfour: Yeah. And so the reason I think we're going to see more of this is because, you know, if you look at the plot of- people have been plotting the progress of these models across benchmarks. And with the latest release of O3 from OpenAI, it's very clear that the capabilities on the benchmarks, like O3 is now past the inflection part of the exponential curve and is starting to move up it.

[00:38:09] Brian Balfour: Whereas most of the products we've seen built today are still built on the technology that's on the flat part of that exponential curve. But once you see new products developed on top of this new technology, which enables fundamentally better results, and we don't really have the distribution adoption friction of the mobile wave where mobile devices had to be rolled out, or even the shift to the internet where people had to get hooked up to the internet.

[00:38:38] Brian Balfour: I think we're just going to see that timeline condense. And as a result, customer expectations spike really fast. People fall out of product market fit really fast. It breaks their core growth model. And it becomes incredibly tough to respond and react and get out of that tailspin in enough time.

[00:38:58] Brian Balfour: And so as a result, I think as we think about how to avoid this, I'm not convinced that the strategies that we learned from the patterns of previous technology shifts are the ones that help you avoid PMF collapse in the shift. And so that's the conundrum. That's the really tricky and to me, it's both exciting and also very scary because as a founder, I'm living this on a day-to-day basis.

[00:39:28] Andrew Michael: I can imagine. And I think the same thing as well, like the old adage of like these big, giant corporations, like the question of like, what if Google just did it tomorrow? Like that question is now more relevant than ever because like in the past, you would have had a very good excuse to say, oh, they're a massive company and like, it'll take them months to even organize and get around to things.

[00:39:49] Andrew Michael: And like, now I think it feels like everybody's like all in on this and like everybody's like moving at light speed, even like these large corporations, Microsoft and stuff. And so like, I think like the question of like, what if they do it is like way more relevant now than ever, because it's way easier for them to do it now. And they're way more invested because like it is also for them like this life or death situation. So I think you're stacking that on top as well just makes it incredibly more difficult to navigate and figure out where to go from. Yeah.

[00:40:16] Brian Balfour: Well, I think that gets to the second you mentioned Google, which kind of gets me to another thing I'm formulating in my head, which a lot of people are like to claim the death of SEO. I haven't seen definitive evidence of that yet, but that doesn't mean it's not occurring. It just means that we might be on an exponential curve, just on the flat part of it. And it will inflect meaning there will be an inflection of people that shift over to these AI experiences like Perplexity or ChatGPT search. And as a result disrupts SEO in a very short period of time.

[00:40:51] Brian Balfour: If that happens, then what we'll see is not product market fit collapse, we'll see product channel fit collapse. There are tons of businesses that are built on the back of channels like SEO. In SEO's case, things like TripAdvisor, which has the user-generated content loop or even things like Pinterest and stuff like that. And so if we see that in a major channel, you could also argue maybe we see that on email, too, with all these AISDRs creating all the noise and all that kind of stuff.

[00:41:23] Brian Balfour: Anyways, if we see that, then that has a pretty big second order effect on all of the companies that are built on the back of those channels. So I don't know, like, I don't know. I'm like trying to find data to make me believe one way or the other. And I haven't necessarily found that yet, but it's a non-zero probability in my mind.

[00:41:48] Andrew Michael: Yeah, for sure. I think it was [inaudible] who had recently mentioned that like everybody's saying that it's going to be an [inaudible] and in fact, they've seen like increase in searches and it almost becomes like people now have more trust to search for these things and the AI snippets have enabled that.

[00:42:02] Andrew Michael: But yeah, for those as well listening, like I think a great piece that Brian wrote is called The Four Fits of How to Build $100M Company. And in this, he talks a lot about the product channel fit. I think for me, this is probably one of my favorite essays of yours. Like we actually ran the exercise at Hotjar. I've helped other startups as well, like modeling that out and like, that part. So I definitely highly recommend reading that. I'll add that in the show notes as well to get some better context.

[00:42:26] Andrew Michael: Brian, it's been an absolute pleasure chatting today. Is there any sort of final thought you want to leave the listeners with before we wrap up today?

[00:42:31] Brian Balfour: Just, I'm doing a lot of writing on this right now, a lot of thinking on this. I encourage, I love discussion and comments on this. So I'll be publishing on my personal site, brianbalfour.com, the Reforge blog, which is this reforge.com/blog. And then I'm posting a lot on LinkedIn these days. So you can just follow me there. I've kind of abandoned Twitter at the moment. But those are the three places and I would love discussion and commentary.

[00:43:02] Andrew Michael: Amazing. We'll make sure to leave everything that we discussed today in the show notes. And yeah, so thanks a lot for joining and wish you the best of luck now navigating this new path.

[00:43:11] Brian Balfour: Yeah, thanks for having me again.

[00:43:12] Andrew Michael: Cheers.

[00:43:14] Andrew Michael: And that's a wrap for the show today with me, Andrew Michael. I really hope you enjoyed it and you're able to pull out something valuable for your business. To keep up to date with Churn.FM and be notified about new episodes, blog posts and more, subscribe to our mailing list by visiting churn.fm.

[00:43:34] Andrew Michael: Also don't forget to subscribe to our show on iTunes, Google Play or wherever you listen to your podcasts. If you have any feedback, good or bad, I would love to hear from you. And you can provide your blunt, direct feedback by sending it to Andrew@churn.fm. Lastly, but most importantly, if you enjoyed this episode, please share it and leave a review as it really helps get the word out and grow the community. Thanks again for listening. See you again next week.

‍