Eric Ries - The Lean Startup - Talks at Google 2011 - Full Transcript

Note: here's a link for the the related book "The Lean Startup", which he refers to in the video and came out not long after.

FULL TRANSCRIPT

Who saw 'The Social Network'? OK. Right. I feel like that's probably the best modern example

of the entrepreneurship story we're all used to.

  What do we do as entrepreneurs that actually makes a difference? We spend our time trying to figure out which customers to listen to and who to ignore, how to product-prioritize product features.

I mean you guys, how many product prioritization meetings do you go to? It's not exactly the stuff of movies. It's unbelievably boring.

And how do we hold people accountable? How do we measure to figure out if we're actually making progress or building something that nobody wants? See, watching somebody pretend like they don't have anxiety that their vision is wrong, is not very good for movies. But that's what most entrepreneurs do.

And so, we're gonna have to talk about stuff like management and accounting.

My goal, my mission in doing this whole Lean Startup thing has been to try to put the practice of entrepreneurship on a more rigorous footing. And so, I started out with a definition.

Here's mine: What is a startup? "A human institution designed to create something new under conditions of extreme uncertainty."

So I think the most important part of this definition, and for our purposes today, a very important part of our discussion is what it excludes. It doesn't say anything about what the size of your company is. It could be five people, 5,000, or 50,000. It really doesn't matter.

It doesn't matter what sector of the economy you work in. It really doesn't even matterwhat industry you're in.

If you fundamentally are operating with extreme uncertainty about who is your customer, what product do they actually want, and how do we build a sustainable business, then you are an entrepreneur. And when I work with large companies, one of the things I have been trying to do, is to get them to adopt entrepreneurship as a job title.

Entrepreneurship is a career. When you become an entrepreneur, you are no longer an engineer. You are no longer a marketer. You are no longer a UX designer. Whatever it is you used to do, all of a sudden now, you have a different job title and you've entered a new career path.

But unfortunately, we don't get the memo that tells you that. So, it can be a little bit confusing.

That's all a fancy way of saying a startup is an experiment. What I mean by "experiment" is not just like let's ship it and see what happens, OK? That's not science.

If you just put some compounds in a beaker and heat it up, you might look like you're doing science. But unless you have a hypothesis that you're trying to test, you have theory, it suggests which experiments are gonna help you and then you make specific predictions, then, fundamentally, you're not conducting an experiment.

And we mean, in a Lean Startup model, "experiment" in the scientific sense. We're trying to create a science of entrepreneurship that will help us to stop waste people's time, because that's what we're doing on an industrial scale.

Anyone's who's worked on new products knows that most of them are doomed to failure. And when you get at the end of that product -- I mean as an engineer, I kept having over and over again the experience of working on amazing technology that is today sitting on a shelf or worse, that fundamentally nobody is using.

And I kept looking for more and more technical solutions to that problem. I thought, "If we could just get the right development methodology, if we just had the right amount of unit tests or the right this or the right that, then we could stop that happening."

But the biggest waste that product development faces today is not building things inefficiently, but building things very efficiently that nobody wants. And I brought a demonstration.

We all know that most startups fail. Who remembers Web 2.0? Remember Web 2.0 when that was really cool? At the height of Web 2.0, 2006, a graphic designer put together this graphic. Have you seen this before? This was the like the logos of all the incredible Web 2.0 companies that were gonna change the world.

And in just three years, in 2009, a different graphic designer was feeling a slightly different set of emotions when they put together this graphic. Our three year report card in Web 2.0. I mean, the blood red Xs, these are all companies that are just dead.

I think for our purposes today though, a much more important part of that chart to look at are the green circles. I won't point at any of them in particular, but some of those green circles are supposed to be the success stories of Web 2.0. But for this chart, what that means is there are companies that were acquired by a larger company, including this one.

And listen, I'm all for people making money. So when a company gets acquired by another company, usually investors and the founders make some money and that's all good. But my question is which of these companies are actually success stories?

Success stories by the higher definition, not of "did anybody make money?" But rather, which of these companies succeeded in living up to the aspirations, dreams, time, talent, and energy of the founders and their investors? And more importantly, their employees.

See, look, we all know that when big companies buy startups, at least half the time, they

die afterwards. So, we buy something for hundreds of millions of dollars. And then we wind up selling it three years later for tens of millions of dollars.

That's not supposed to happen. In general management, that doesn't happen. When you buy an asset, it depreciates in a predictable way. But when big companies buy startups, it doesn't happen exactly like it’s supposed to.

And I think the problems that corp-dev departments have when deciding how to buy startups and which startups to buy and then how to integrate them into the parent company, are the exact same problems that internal innovators who are trying to create brand new startups inside big companies have.

And they're the exact same problems that venture-backed entrepreneurs have with their investors.

All of us lack a theory of entrepreneurship to guide our behavior and so we're falling back on tools and methods that are not appropriate to entrepreneurship. That's my belief.

So, I don't think it has to be this way. See, it's one thing if startups were failing because they were taking too much risk. If one of these companies was working on teleportation and then it turned out to be too hard – we couldn't quite get the technology for quantum entanglement like we thought -- I accept that kind of failure; that happens.

But I chose Web 2.0 for my demonstration, especially for you guys. You know. There's not a single company on this chart where you would say, "Boy, I wonder if that can be built."

Right? "Geez, I wonder if that new social network -- is it possible to build it?" We all know. Software companies, we can build anything we can imagine. Think about that for a second.

The dominant question of our time is not can it be built, but should it be built. And the issue is: can we build a sustainable business around a particular product? So, the future of our society, our economic growth in the future, the GDP growth of industrialized countries in the future is going to be dependent on the quality and character of our collective imaginations, which I think is a very strange place to be.

That is really different than the kinds of economic problems that general management was designed to solve in the 20th Century.

Now, most of my startups have failed. So I know that's not how you're supposed start one of these talks, like, "Hi. I'm a professional expert and I have had more failures than successes. So you can be just like me if you'll follow my advice."

 So, I'm sorry about that. But those of you who spend any time around entrepreneurship know the truth that where there's startups, there's a lot of failure. And it has to be somebody's fault in a talk like this and obviously it shouldn't be my fault 'cause I'm the expert.

And preferably, it should be the fault of someone who's dead so that they can't argue. So I chose this guy.

[laughter]

This is Frederick Winslow Taylor. He died in 1915, which is very handy for the purpose of this talk because it means he can't talk back. So, sorry, Fred.

 Fred Taylor invented something called "scientific management" in the early 20th Century, which today, we call "management". See, people don't really remember Fred Taylor.

And those who've studied scientific management in school probably remember him for some of his outdated and really ridiculous ideas, like time and motion studies or the idea that a worker is basically just an automaton and should be told what to do.

The reason we don't give Fred Taylor the credit he deserves is because he invented things that to us are so obvious, we can't imagine them ever having been invented. It doesn't make sense.

Like, everybody knows that work should be done as efficiently as possible, right? And that we should treat work like a system and that we should have managers organize that system. That's just plain common sense.

And my favorite, Fred Taylor, invented something called "The Task and Bonus System", which we just call "tasks".

The idea was if you want to do a large project, the best thing to do is decompose that project into a series of individual tasks, assign those tasks to functional specialists. And everyone just does their part knowing that if everybody does their part well and everybody else does their part, the whole will actually work out like the manager said.

And here's the best part. If you do your task particularly well, better than expected, you should be paid a bonus rather than being penalized. What could be more obvious?

Except in the 19th Century, the way work was organized is that if you did your task better than expected, you were penalized.

Why? Because that showed a lack of integrity. You obviously could have done it better before. But you didn't. So that proves that you're a liar.

It gets worse. Not only are you a liar, but what about all your compatriots, your coworkers, who do the same task the old, slow way? They're liars, too. All of you have been lying this whole time and you should all be penalized.

Imagine working in a factory where if you can come up with a better way to do your repetitive job, not only would you be penalized, so would all your coworkers. Can you imagine the culture that would grow up around such a thing? That everybody is working really hard to make sure that nobody ever does anything in any way better because then we'll all be in trouble.

That phenomenon was so widespread in the 19th Century, they had a name for it. They called it "soldiering". That all the workers were intentionally soldiering on, trying to do work as slow as possible so that nobody would get in trouble. Now, we laugh when we think about that kind of thing happening in a factory. Because to us, the way that we manage physical products, and just all regular general management, is light years beyond what was possible in Fred Taylor's day.

But they also believed something else that I think maybe you'll find a little bit familiar. They had this idea that what basically was the great man theory of work. That fundamentally, the job of managers was to select the best possible person. Of course, this is the early 20th Century.

So a great man of upstanding character with good integrity, the right attributes, put them in the job and fundamentally leave them alone. Because if you just trust a great man to figure out what needs to get done, they would figure it out.

Does that sound familiar? We still manage knowledge work and innovation work that exact same way. We still believe in the great man theory of entrepreneurship and we believe it especially about the great man software developers.

And yet, when we look back on this time, decades from now, I think we're gonna laugh at the kinds of things that we do when we need to develop new products. It will seem as antique to our future selves as Fred Taylor feels to us.

Because I think that entrepreneurship is management. It's just a different kind of management than the general management that has been practiced since Fred Taylor's day.

So, we need to create a different paradigm for management that's not better than general management. It's not worse than. It's simply a parallel discipline specifically for entrepreneurship.

And so, here's my attempt.

The first concept in the entrepreneurial management toolbox is this thing we call the "pivot". Who's tired of hearing about pivots already? Anybody? OK, I apologize. In Silicon Valley, everybody's hand is up, by the way. This word has become ridiculously overused. Believe it or not, I saw this just the other day in the New Yorker magazine.

Can you read this caption? "I'm not leaving you. I’m pivoting to another man."

[laughter]

So this word started in Lean Startup and then it became this monster of an overused, overhyped concept. Typical for Silicon Valley. We went from not knowing what the concept was to being tired of hearing it and claiming that it's overhyped, without having passed through the intervening stage of ever learning how to use it correctly. That's just – that's how we operate with the hype-cycle in Silicon Valley. So, I'm sorry about that. I really didn't intend.

But you can't understand anything about entrepreneurship unless you have a word for this concept, because it is the universal constant of all successful startups. If you can get the real story of what actually happened in the early stages of a company, you will find out that successful startup founders do not have better ideas than the failed ones.

Contrary to what you see in the movies, most startup founders of successful companies had ludicrously bad ideas at the beginning.

And what's amazing about the successful startup founders is not that they just persevered indefinitely, but that they had this funny combination that when they run into difficulty, it's not just that they gave up ship and went home.

Neither did they persevere the plane straight into the ground.

They do this thing we call the "pivot". By analogy to basketball, one foot firmly rootedin what we've learned, changing one other thing about the business at a time.

And the premise of the Lean Startup is as follows: If we can reduce the time between pivots, we can increase our odds of success before we run out of money.

See, the way you think about startup runway, is not how many months of burn do I have left? It's fundamentally how many opportunities to pivot do I have left?

And sure, we can extend the runway by raising more money. But we can also extend the runway by figuring out how to pivot sooner.

And every day we shave off that time is a magical extension of our runway by a proportional amount.

So, to increase the odds of success, we need to figure out how can we pivot sooner. We need to bring our focus to a validated learning.

Anyone know this? This is the waterfall methodology of software development. It's traditional in one of these talks when you're gonna beat up on methodologies to call one the "waterfall methodology". So this is mine.

This is just Fred Taylor's idea as applied to software development. When I was trained

as an engineer in Silicon Valley, I was taught this as the manufacturing metaphor of software development. You can imagine, incidentally, how pissed off I was when I found out that they don't use this in manufacturing anymore. This is way obsolete.

So it's not clear to me what our excuse is in software development for copying an obsolete model of manufacturing. But I understand why it’s so appealing.

The idea is that since software is so intangible, we like to imagine our work traveling on an assembly line, a virtual assembly line, from department to department. And if everybody does their part and trusts everybody else to do their part, everything works out fine.

It's entertaining to beat up on the waterfall methodology because it has such a bad track record. But it’s important to understand that waterfall works as long as the solution and the problem are both relatively well understood. So if we were building something that is fundamentally similar as things we have built in the past, this works great. If you're building a video game sequel, amen. If you're building the next iteration of a product that zillions of people already use, that's fine.

But entrepreneurship doesn't deal with those circumstances. So it's basically a waste of time.

Now, when you do waterfall, you have this problem I call "achieving failure" where you successfully build the wrong thing and you boast about how good you are at doing it.

My question is, if you go to a startup board meeting, or a milestone review meeting for most new products, what do we talk about? Milestones, right. We are on track. We're building features we said we were gonna build. We talk about our gross numbers like – hey, we have this many customers just like we said.

I remember being in a company once that was looking, had a plan. They said we were gonna have this nice hockey stick-shaped growth. And we're on the nice, long flat part, and everything's going according to plan. Sounds a little familiar, maybe.

And you're going to court like, the plan is genius. Like, nobody is using our product as expected, as expected. But next week, it should turn up.

And how do you know if you were on the long, flat part and you're gonna just, or you're on the hockey stick and you're gonna just coast indefinitely? I believe we can actually answer those questions quantitatively. We'll get to that.

So, to me, we are bragging about how we're building a product that nobody wants. But we're doing it on time and on budget. As if, I have this image of a general manager driving a car off a cliff and while they're driving their like, "But I'm getting great gas mileage!" right off the cliff. That's what startup failures mostly look like.

And I think that's true in big companies, too. Not that I would presume to talk about you. But in other companies I've seen, for sure.

So, in manufacturing, they abandoned that linear way of working. That's why we call it Lean Startup by analogy to lean manufacturing.

These are two other unfortunately deceased men who made this possible. Deming is famous for having said, "The customer is the most important part of the production line."

By which, he meant everything that we do should be gauged to be decided by whether the customer cares that we do it.

So if the thing is of high quality in the customer's eyes, that matters. But if we're doing a lot of internal transport, with the meetings that we have, the specifications, the customer doesn't care about any of that internal stuff. So let's try to eliminate it.

When these ideas were handed down to me as a Silicon Valley engineer, they looked like this. There's our very own guru of extreme programming, Kent Beck. You guys know Agile very well, so I won't bother getting into it.

Suffice to say that all Agile methodologies have their origins inside the IT departments of big companies. Every single one. And there's a reason for that. They are designed for situations where it is the problem that's known, but the solution is unknown. And so, by building something that is well-understood iteratively, we can increase the odds that the project will be successful.

So, the classic -- Chrysler Corporation needs a new payroll system. Agile to the rescue. But this isn't the world that we live in as startups either. If the customer is the most important part of the assembly line, what do we do if we don't know who the customer is? That in whose eyes should we judge our work? In extreme programming, which customer should we sit down next to the engineers to tell them what to do?

The assumption of Agile and all previous management approaches is that there is somebody who can give us an authoritative, definitive answer to design questions.

And in entrepreneurship, that assumption breaks down.

We are working on products where nobody knows what the customer wants. At best, we have a theory, a hypothesis, a plan, a hope. And so, this is what Lean Startup looks like.

Now, at Lean Startup we have our own guru, Steve Blank. He's still alive, but I put him in sepia tones just to be consistent.

[laughter]

Steve invented something called "customer development", which is an iterative process of trying to figure out who your customer is, which we can merge in parallel with Agile development to this company-wide feedback loop of learning and discovery. This changes the unit of progress from making stuff to validated learning.

Let me try to illustrate what I mean. I created a company called IMVU in 2004. We made a 3-D avatar instant messaging technology. And at that time, we wanted to be the next AOL back when that was still cool.

And we wanted to take over the hot, new social technology of IM. We really thought that was the wave of the future. Whoops.

And here was our plan. See, everybody knows that instant messaging is a network effects business, right? So, therefore, if you wanna get someone to switch from their IM network to yours, it's kind of a pain 'cause they'd have to bring all their friends with them. So there's high switching costs.

And therefore, IM isn't an industry characterized by high barriers to entry. That's the MBA analysis of the instant messaging market. And we spent a lot of time figuring that out at the whiteboard.

And we said, "Ah, we need a strategy. A strategy for avoiding that problem and here it is. We'll create an instant messaging add-on that interoperates with all of the existing networks and can bring 3-D avatar technology to your IM client. So, we take your boring 2-D IM and make it 3-D. Wow."

This is before 'Avatar' and the current 3-D craze. So, we thought we were on to a real trend. And so, here's the reason we got so excited about that strategy. It would be inherently viral. Because when you would decide you wanna go 3D, you would have to be IM’ing with somebody and they would automatically get a text link inserted into the chat stream that they could just one-click, pick-up boom, IMVU installs. Now you're both in 3D. Doesn't this seem like a good idea?

Well, we met with investors at that time. The strategy part of it, they're like, "That does sound very promising." And when I tell the stories to MBAs now, I get a lot of nods, like "That is good stuff. I don't know what the hell this guy's been talking about until now, but this I understand. This is strategy."

And the strategy actually is very good, except for the tiniest, tiniest problem, which is that every single thing I just said is false.

Customers actually don't have high switching costs for IM. Their network effects are way overblown and our customers refused to invite their friends. It was a total deal breaker.

We'd have customers come in an in-person usability test. We're paying them to be there.

And we'd be like, "OK, download our instant messaging add-on."

The customer would be like, “What is that?"

"An instant messaging add-on. It interoperates with all your IM."

And you gotta picture of a 16- year old like, "What? Is it an IM client?"

And we're like, "No, no. You won't have to run a whole other IM client."

They're like, "Why not?"

Like, "Oh, it would be so complicated to download."

They're like, "Dude, I have like seven IM clients. What's the big deal?"

And we're like, "There are seven IM clients?"

[laughter]

So that was problem number one.

We're like, "Listen, we are paying you to be here. So how about you download this thing? OK?"

"All right. Fine."

"Download the thing."

OK. "Customize your avatar." They love this part.

"Like, ooh, that's really cool, interactive like that." Great.

"OK, now you customize your avatar. Invite one of your friends."

"No way."

"Why not?"

"I don't know if this thing is cool, yet and I'm not gonna invite my friends to something that turns out to be lame."

See, I know people who are selling like business software are used to the concept of "mission critical". We didn't understand that in our business, mission critical, like the law of commandments of mission critical software, one of them needs to be like, "Do not make teenager look lame in front of their friends." Total deal breaker.

They wouldn't do it. We were like, "We're paying you to be in this usability test."

And it's just like, "I'm not doing it. You can keep your money. I will not. It's not worth it to me."

And they kept saying things like, "Let me use it with -- let me try it out first. And if it's cool, then I'll invite one of my friends."

And we were like, "Oh, OK." We're from the video game industry.

So we knew what that meant. That meant single-player mode. So we built another version of the product, single-player mode. Allowed you to -- we had another teenager come in for a usability test.

"Hey, download this instant messaging add-on."

"I don't know. What the hell is that?"

"Just do it."

"OK."

"Customize your avatar."

"Oh, that's cool."

"OK, try it by yourself."

And they could check out the avatar, dress it up, do the moves and the whole thing. Learn how to use the chat bubbles. And then we're like, "OK, now invite one of your friends."

Teenager: "No way."

"Why not?"

"This thing is lame."

And we're like, "But we told you it was gonna be so lame."

I mean, we're supposed to be listening to customers, but they don't know what the hell they want. And they told us to build this thing and we're like, "It'll be so cool once you invite some of your friends."

And they're like, "Listen, old man. I'm not doing it."

[laughter]

And we were really devastated. Okay. So anyway, long story short, this was a total deal breaker. This strategy is completely flawed in every respect because it’s based on empirically incorrect facts.

Now, we wound up having to pivot the company and we created our own instant messaging network and it all turned out fine.

But I'd like you, if you would, just for a minute to sympathize with me personally. OK? Because I was the engineer. I was the CTO of this company.

It was my job to write the software to do instant messaging interoperability. So I wrote, I don't know, 25 thousand lines of code or something. I did it all Agile, refactored, really elegantly structured if I do say so myself. Good unit test coverage, the whole shebang. And all of my code got thrown out.

The good code got thrown out and the bad code got thrown out. The well-factored code got thrown out. The stuff I was proud to show my mom and the stuff that I wouldn't want anyone to see at all was equally thrown out. If you don't know who the customer is then you don't know what quality means. So failure is a great equalizer of quality. It all had to be thrown out.

And I was really depressed. Because you gotta understand, we had spent six months killing ourselves to build this product. And we had spent I don't know how many hours of my life that I can never get back arguing with each other about the following. Which bugs did we absolutely, positively have to fix. And which ones could we live without? Sound familiar? Which features just had to be in version one or which ones could we just maybe could maybe postpone to a different release? That's what we spent all of our time doing.

And yet, we had this problem, which was that customers would not download our product. Like, this product sucked. It was really buggy. It would crash your computer. I was really embarrassed to have shipped it. And we almost didn't ship it, I was so embarrassed.

But then, I was actually relieved cause nobody found out how bad it was because nobody would use it. And I was like, "Wait, something is not right here. Why am I relieved that nobody's using the product? That doesn't seem right."

And long story short, my cofounders dragged me, kicking and screaming, to the realization it was time to pivot. We had to throw that code away. And we created a standalone IM network and we were much more successful, la di da.

But here's the thing. I had to make myself feel better somehow because I was like, "Gosh. Would the company have been just as well off if I had spent the last six months on a beach somewhere, having nice drinks and doing nothing?"

And I was, "Did I even need to be here given that all the work that I did was thrown away?" Anyone feel like that's true? Anyone know what the excuse I used was to make myself feel better?

What did they need me for? Why was it worth having done this exercise in the first place?

[audience]: You learned something.

Because I learned something. Thank you. The last excuse in the book. If you've utterly failed to execute, you can always claim to have had a good learning experience.

At least you learned something. I mean, I don't tell you guys. In general management, you claim to have learned something, you're likely to be fired. A general manager who learned something -- one of two things is true. Either they didn't make a very good plan, in which case, definitely should be fired. Or even worse, they made a really good plan and failed to execute it. I'd definitely fire that guy.

So, I think it’s natural that we have a little bit of an aversion to wanna just say that we learned anything because that is very dangerous. But in entrepreneurship, failure is, not only is failure an option, it's practically the only option. It's what happens when reality intervenes with our plans.

So here's what we learned specifically. We learned, the hard way, that customers did not wanna use our product to connect with their existing friends. They wanted to use it to make new friends. That doesn't seem like a very big deal. I mean, it's all a very modest change in semantics.

But from a code and product point of view, that is a radically different product. It required a very different experience. And we didn't throw out every line of code. But we had to throw out a lot. The pivot was quite dramatic.

And I made myself feel better with this whole learning story until I asked myself the following question. I mean, literally, I was up nights once I had this question asked to me.

Which was, "Wait a minute. If my goal of the last six months was to learn this important thing about customers, why did it take six months? How come the word 'learning' is only coming up now after we failed and we need an excuse? We never used the word 'learning', not one time during those six months. All we ever did was argue about features and bugs."

And then I was like, "But would we have had the same learning if we'd built a slightly different first product?" Like, for example, did we have to support all seven IM networks? What if we'd supported only three? Would the learning value have been the same? Sure. Customers won't download, so who cares? What if we'd supported only one network? Learning values the same. Now, that's a lot of code between seven networks and one, that's a lot less code that needed to be written.

But this is the thought that literally made me sick to my stomach. I'd say, "Wait a minute. What if we had just created a single web page and in three hours created a photo mockup of what the product was going to look like and said, 'Hey, download this amazing 3D avatar instant messaging add-on.' And had a big download button. Would we even have had to create the second page where we admit that we didn't build the product, or would a 404 have been adequate?"

Come on, it's the 404, obviously.

Because nobody would download the product. It was a deal breaker. Nobody wanted it. That meant that we didn't even need page two.

And that was really upsetting to me, personally. Why? Because I look at my business card and what did it say? It said a lot of things, but all I saw was "guy who writes code." My job is to make features.

So if I went home at the end of the day and I write good code, I had a good day. And now, but if my goal is to learn this thing about customers, and I can do it without code, is that my job? Is it possible that something I could do in three hours is just as meritorious as something that requires 25,000 lines of code? It didn't seem right.

But I think that's actually true. Fundamentally, startups exist to learn how to build a sustainable business. We call it "validated learning" 'cause we have to back up that learning quantitatively. Any old idiot can tell a good story.

But we need a system for rigorously assessing, "Are we actually learning how to build a sustainable business?" And everything else is a complete and total waste of time, including our precious code.

Now, in the lean manufacturing revolution, the first question they had to teach people to ask was "what is the difference between value and waste"? And in a factory, this is actually relatively straightforward. Value is the stuff that we make. The customers want. And waste is everything else.

But if our unit of progress is gonna be learning, then our unit of value has become intangible and now we have an issue. Which is -- OK, we can eliminate all the stuff that we do that doesn't contribute to learning.

 So, we have this concept in Lean Startup called "minimum viable product", which is, what really needs to be in that first version? And now we have a good answer.

Only what is necessary to learn whether our plan is correct or not. Everything else is extraneous.

But that's still a little bit vague. And so the next step in lean manufacturing was to focus on cycle time. And so what that looks like is this. Very simple heuristic. This is the flux capacitor of Lean Startup.

All we are as a software startup is a catalyst that turns ideas into code. When customers interact with that code, they create data which we can choose to measure quantitatively and qualitatively. And then if we want, we can learn impacting our next set of ideas.

This, we can use to put the concept of the pivot on a more rigorous foundation. A pivot is one major turn through this feedback loop. And the heuristic for any kind of startup advice that anyone wants to give you is really simple. Does it minimize total time through this loop?

So I don't know about you, I go to a lot of startup talks, I read a lot of startup blogs.

All the advice is like this. "You know, it's really important to have great design. Design always wins. Except craigslist didn't have very good design and neither did EBay. So sometimes it's fine to have no design. But make sure its very scalable, cause you don't want to be the next Friendster. Except that Facebook wasn't very scalable and it was fine. So make sure you have good design and design doesn't matter. It's scalable, but not too scalable."

It's not very helpful advice and if you go down the list, it's like make sure you raise plenty of money, but not too much money. Make sure you have the right kind of people, but not those other kind of people, but actually, sometimes those other kind of people are fine.

And we focus on all this contradictory stuff 'cause for any particular piece of advice, I can find you somebody who followed that advice and then made a lot of money. I can also find you somebody who didn't follow that advice and made a lot of money. I can find you people who followed that advice and made no money and people who didn't follow that advice. I can find you all four quadrants of a logical possibilities chart.

So, how do we know what advice to take and what not? I think this is the heuristic you wanna use. If it gets us through this feedback loop on a sustainable basis faster, it's a good idea. And if not, not.

There's a lot more, of course, to Lean Startup. There's a zillion things on this graph. You can read them all on my blog, Startup Lessons Learned. Of course, you can buy a certain book. I've heard it's coming out in the fall. It's really good.

All of these techniques, like continuous deployment, where we put software into production, like 50 times a day on average. So, 20 minutes from the main trunk to production, no branches.

Things like net promoter score where we can evaluate in real time using a tracking survey, what customers really think about our product. Everything you know about usability tests, five whys, which is drawn from the Toyota production system.

Each of these techniques has -- they operate at one stage of the feedback loop, but they have the net effect of minimizing total time. That's what the Lean Startup is about.

But I wanna mention one more really boring topic called "innovation accounting". See, we've forgotten what accounting was designed for. I mean, we think of accounting as that thing that the really boring people do to keep track of where the money goes, right?

That's pretty much what it is. It's just a ledger that says, "Where did all the money go?"

But accounting was invented for a very different reason. It was invented to drive accountability across departments.

Because if you wanna have a large company with many different divisions, you have to be able to hold the managers of those divisions accountable to some things so that you know that they did a good job.

General Motors, which invented most of our modern management paraphernalia, had this concept. When I first read this concept, I literally laughed out loud; I couldn't believe it. It was called the Standard Volume. It was the ideal number of cars that General Motors should sell, division by division, in an ideal year. And they actually had the math, and staff, the macroeconomic staff to figure out, given all the macroeconomic data available, how to translate the standard year into our coming actual year.

So they could go division by division and tell each manager, "Given that we're in a recession, or the economy is booming, you should sell this number of Oldsmobiles. And therefore, if you sell more than the standard number, you get a bonus. And if not, you failed."

And it's not fair if you didn't have that concept, then if it's a good year, all the managers seem like they're doing well. And in a recession, everyone seems like they're doing badly. You can't tell which manager actually made a difference.

Now when I read that concept, I laughed out loud because I was like, "Wait a minute. Are you telling me there was a time when people could make forecasts about what was going to happen in the future, and then it actually happened?"

I don't know about you. I have never in my whole life seen a forecast of anything that turned out to be remotely accurate. No startup I have ever worked for has had a roadmap that turned out to be remotely true. I have never seen a company say how many customers they would have in the future and then deliver. Never seen it.

So to me, the idea of the standard volume is ridiculous. But I understand when you have a sufficiently large company and you have a sufficiently long operating history, you can do this. Maybe this sounds a little bit familiar.

So, if we're using accounting to drive accountability, but all of our accounting depends on having a long operating history and a lot of customers, how do we drive accountability if there's no customers yet?

If the CFO of a company, hypothetically speaking, gives a certain team a bunch of money and sends them off to some remote location to do their work, like to Australia or something. And then they hang out in Australia or whatever. And then a year later, they say, "What are your results?" And the team says, "They're not very good, but we're on the brink of success."

How is the manager who gave them all that money supposed to know if A: They are in fact on the brink of a success or they're just on the flat part of the hockey stick, or if they've just been goofing off for a year? Or more likely, if they're just executing a bad plan.

And at what kind of milestone should we hold them accountable to if we can't hold them accountable to the gross numbers of customers? 'Cause that's fundamentally not fair. If we're focusing on the gross numbers, incidentally, we might decide we're gonna do a lot of publicity and PR and be like, "This thing is gonna be amazing," to drive customer awareness.

But we all know if you happen to find yourself on such a team, that that early awareness is fundamentally lethal.

But it's not fair to just say, "Well, just let them do whatever and hope for the best." You guys know exactly how that turns out. So, what is the solution?

I think we can answer that question now with something I call "innovation accounting". Here it is.

Instead of focusing on product milestones and gross numbers, we have three learning milestones we can focus on. We have to take our attention away from the vanity metrics.

Vanity metrics are the numbers you put in a press release to make your competitors feel bad. Like the total number of pages on the internet that you've indexed. I happen to like that one a lot.

There was a time when we had a big index, number of in pages indexed battle. And it's like, "We have four billion and you only have two billion." But like, what does that actually tell you about the quality of somebody's business? Absolutely nothing.

They could be four billion really dumb pages. It could be one guy's website who's just really excited about the number four billion. Or, it could be four billion people who each have one crappy website.

If you read "TechCrunch", you're gonna see a zillion stories about "This company has sent 400 million messages through their platform." But is that 400 million people who are all about to turn out, or one very excited customer? We don't know.

We used to have a competitor in IMVU that would report on the gross GDP of the value of their whole user to user economy. And my CEO would sometimes come to me and be like, "These guys have a four hundred million dollar GDP. What's our GDP?" I was like, "What does that even mean in our context? If two users exchange some virtual currency, is that part of the GDP? I don't know." It's a completely meaningless number, but it sure made us feel bad.

I'm all for vanity metrics and press releases. Go to town. That's fine for making your competitors feel bad. But what happens when we use those numbers to guide our own business?

Is that "when the numbers go up, it's always because of what I was working on"? Everyone thinks I made this feature last week and now the numbers went up. So obviously it's due to me. Of course, the people in marketing feel like it's 'cause their new marketing campaign, etc. What happens when the numbers go down?

Anyone ever been in that meeting? Oh, it's seasonal effects. Did anyone ever hear seasonal effects used to describe numbers going up? Never in my career. It's always like, if it's going up, it's features. If it's down, it's seasonal effects, or worse, those idiots in marketing.

And over time, each us lives in our own private reality where the stuff I do makes numbers go up and the stuff that those guys do make numbers go down. So is it any wonder that we think each other are idiots?

Now, expand that organization larger and larger and larger as people are in ever more permanent silos, speaking their own language, living in their own private reality. Is there any wonder that they have trouble working together?

Maybe that sounds a little bit familiar. Okay. Just checking. So instead of that, we're gonna use actionable metrics, which are about per customer behaviors, things that can be measured at microscale.

And the first thing we're gonna do is establish the baseline. So now we can put the purpose of the minimum viable product on a much more rigorous setting. Somewhere in our business plan, there is a model that says, "Hey, if customers behave in this way, then we’ll have zillions of them over time." And we can't get into all the details on how to build those models.

In the meantime, what we wanna do is just figure out what are the real numbers for each of those inputs at microscale? That's what the minimum viable product is for. So, if there's some number, some spreadsheet somewhere, that says, "Hey, ten percent of customers who come to our website should register for our product." Then we should have a big banner in our office somewhere that's like, "We must have ten percent conversion or we die."

And then, we each have a minimum viable product as soon as possible to find out what that number is today.

And most likely, when you do that experiment, the baseline will be horrible. Like, it'll only be one percent and it's supposed to be ten percent. And like, oh my God. In general management, that provokes a crisis cause now we failed and uh-oh.

There's this thing called the "audacity of zero", which is how much easier it is to raise money and get people excited when you have no results. Or having zero dollars of revenue in a startup is a great time to raise money. Having one dollar of revenue is a disaster. 'Cause with zero, it's like, "Well, why is it zero?" "'Cause we haven't launched." So, obviously it should be zero. Everyone's like, "Oh, that makes sense."

God forbid you have one dollar of revenue, 'cause then they'd say, "Why is it only one dollar? I thought this thing was gonna be an overnight success and now you're proving to me that it isn't."

So with zero you can always be an overnight success. With any other number you're screwed.

But we need to change that. We need to say, "Finding out the truth of where we are right now is progress. It's a milestone that we should celebrate."

And then we do step two, which is we tune the engine. We make product development changes that are not designed to drive huge gross numbers, but to make those conversion numbers go from the horrible baseline to the ideal in our business plan.

And whenever I've done this with teams, I've only ever seen two cases. Case one, it's supposed to be one percent. It's one percent but it's gotta be ten percent. So, a few iterations in, it's one percent, three percent, six percent, six and a half percent, seven and a half percent.

Now, it's not ten percent yet. So the model isn't exactly working, but you can say, "Are we gonna get there?" Yeah, probably. Each thing that we do seems to drive the number up a little bit. We seem to be heading in the right direction. We're split testing to make sure that the changes we're making are in fact driving the change. It's all good.

Here's situation number two. It's one percent, three percent, three and a half percent, 3.75 percent, 3.8 percent, 3.81 percent. Now, the numbers are going up every time. So it's not like the numbers are going down. It's not like it's zero. But you might ask yourself a question four, five, six months into hitting that asymptote. Are we ever gonna beat ten percent? I think it's safe to conclude the answer is no.

Of course, theoretically, it is possible. The next iteration will be that magic one more feature that gets you to ten percent. But in reality, that's not the case. When the team gets to the point where hitting that diminishing returns, everybody knows you're not gonna make it and you enter the land of death march.

So instead, I recommend we do three. We schedule the meeting in advance. That three months from now, six, whatever it is, we're gonna have a meeting to decide whether to pivot or persevere.

And by that meeting, we will have the data about whether our efforts to tune the engine are working or hitting diminishing returns. And so, we have all these concepts in entrepreneurship, like product market fit, that are very vague. This system allows us to put those concepts on a much more quantitative basis. We can't turn whether to pivot into a formula.

I can't tell you what to do. I still rely on human judgment, just like science does. But if we make specific predictions, if we use innovation accounting as our accountability model, then we can be training our judgment to get better over time, just like in science.

So, don't do product development astrology. Do product development science.

I left a bunch of questions unanswered 'cause we only have a short time together. Like, how do you know specifically when to pivot? What's the relationship between our vision, our strategy and our product? What exactly should we measure in each of the engines of growth? How is it that products grow? How do we know if we're on that hockey stick, or on the long, flat part forever? How do we test if we're creating value? What specific features should be in the MVP? Can we go faster? The answers to these questions and so many more are in the new book coming out in the fall, called "The Lean Startup". You can, of course, preorder it at lean.st.

So we have time for a few questions? OK, let her rip. If I stumped all of Google, I'm gonna be pretty proud of myself. That's going right on Twitter.

>>Female Audience Member #1: So, I just wanted to touch on something that you mentioned is reviewed too many times; the pivot. I have trouble understanding exactly how much work to put in for the first pivot. But just starting off, like how much work should you really put in for that first pivot?

>>Eric Ries: There's no way to answer that question in general, honestly. You have to put yourself into a position where the team will know if it's working or not working. And the problem is that most teams have a plan, which is basically to ship it and see what happens.

And the problem with ship it and see what happens, you can feel like you're being very agile, but you're guaranteed to succeed at seeing what happens. And so, you'll therefore always feel like it was worth doing and you'll feel like you're on the right track no matter what.

The only way to get yourself into a position where you have to pivot is to make specific, concrete predictions ahead of time that if they turn out to be wrong, will actually call your theory into doubt.

 And the issue is that we all know that most projections for new products are complete BS. You have to tell the CFO, or whoever, that you're gonna have a zillion trillion customers in year five. Otherwise you won't get the money to do your project.

But we know that we just made those projections up. So when they don't prove to be correct, we're like, "Well, that doesn't prove that our vision is wrong. It just proves that it took longer than we expected." So, yeah, the hockey stick is still gonna happen, but it's taking longer.

If we do innovation accounting and we make very specific per customer behavior predictions -- like one thing we'll often have people do is sell the magic version of their product on a landing page somewhere.

And it's like, don't even say what the product, like how it works. Just say the benefit that it gives you and see if you can get people to sign up. If people won't sign up for the magic, they're certainly not going to sign up for your product, 'cause magic is always better.

And if magic isn't even good enough, if the conversion rate on magic is too low, then you already know that you have a problem. Not that that means that therefore give up, go home. It just means there isn't already enough latent demand for what you're doing. And so you're gonna be in a different kind of market then maybe you expected. Does that help? So the minimum viable product truly is the minimum, the least amount required, to get that first information.

It's not, "Oh, if it doesn't turn out into an overnight success, we give up." And if you release that pressure to get it right the first time, like, I feel like a lot of us feel like we have to do this circus act to make it seem like we could predict the future. Like, one of those brilliant visionary, the next Steve Jobs. Not even Steve Jobs is as good as Steve Jobs. That's a story that we've all been told.

And every company I've worked with internally, there are these genius heroes who always seem to be able to get it right on the first try and everyone else is trying to emulate. But when you meet the hero, you're like, "How do you do it?" If they're being honest with you, they're like, "I actually don't know."

Or, "Actually that's not how it happened and it wasn't as easy as everybody thinks. Now I feel incredible pressure to replicate that success again." And so, you can imagine the negotiation that happens with the superstar over their next project. They don't ever want to be in a position where they do something and it doesn't work, or they'll have to quit and go to convince some other company they're a superstar.

>>Male Audience Member #4: So most of the rapid feedback you're talking about seems to be very applicable to consumer applications where the cost of trying something in the amount of time it takes to try something is pretty low. I will or will not sign up for Twitter or Facebook or whatever the next thing is.

How does it apply if it's a bigger thing? If it's an enterprise sale or if it's a bigger commitment, do you just basically take the same process, but the time scale is longer because the commitment process is longer for each step?

>>Eric Ries: Yes. I mean, that's the short answer. I mean, the long answer is my background's in consumer internet. So I talk that way just naturally about large sample sizes and split tests. I can't help it. That's my whole career.

What's funny is that Steve Blank, who I mentioned earlier, has a great book called "The Four Steps to the Epiphany", that I hope you all read. When he talks, his background is in enterprise software. He gets this question too, all the time.

Except when he gets the question, people say, "Well, of course, this will work in enterprise software. But how will it work on consumer internet?"

Because we just assume that the tactics that have worked in one industry don't work in another. So the answer is truly, it's the principles that matter, not the tactics.

And so, the principle of the build-measure-learn feedback loop is cross-industry. So for example, in consumer internet, because we're used to having large numbers of customers, we do all this analytic stuff as a crutch. It's actually -- it's because we have it worse than the enterprise people.

When you have a lot of customers, you can't get to know any of them particularly well. In fact, you just have, you have to rely on archetypes and summaries and assumptions.

When you have a small number of customers, it's a huge asset because you can know each of them extremely well and the experiments that you do can have much higher fidelity.

So like, physicists don't get to ask electrons what they're thinking about doing. They're not available for comment.

But when you're studying something that can actually interact with you, it's a whole different, whole different ballgame.

>>Male Audience Member #5: What product should Google be pivoting on right now?

>>Eric Ries: What? What product should Google be pivoting on right now? Listen, I would never presume to tell Google anything about that. I mean, look.

>>Male Audience Member #5: You were invited.

>>Eric Ries: What's that?

>>Male Audience Member #5: You were invited.

>>Eric Ries: I was invited. But look, but seriously. Like, the outside expert doesn't know anything. You guys know your business way better than I do. And you know what products need to be pivoted. You know. You don't need me to tell you.

The better question is, how on Earth are we gonna get to pivot them? Because we're stuck in a management and accountability framework where pivoting is failure is a problem.

So for example, a certain Google product that I know especially well because it was competing with something that I was working on at the time, I won't get into too much detail.

One of our cofounders at IMVU went to work on this product for Google. So, you're Google people. You can talk about this. So they had a lot of inside information about what we were doing available to them.

And at first, we were really nervous because it meant that we were gonna face competition from Google.

Here's what happened. Google spent two years working on that product. Spent, I can't imagine how much money developing it. And when it finally came out, they launched it with a big bang, put the Google brand right on it. And then when some things didn't go as expected and it turned out to be an embarrassment, like, it got pulled and killed.

And the whole time, we were like, "What is going on over there?"

'Cause we were iterating and changing constantly over that whole two year period. By the time they launched the product, we felt like they'd launched a product that did what our product did two years ago. And by giving it the big launch hype, putting the Google stamp on it, the inevitable problems, the same problems we had had when we launched, but we were able to pivot because we had a pathetically small number of customers and no one had ever heard of us.

That's a huge asset. It's actually a relief. 'Cause it means you can screw up. Nobody knows. Nobody cares. Obscurity is a benefit. By putting the Google name on it, by claiming it was the next big thing, Google put that team, I assume, in a position where there was no way for them to pivot. It became an embarrassment to corporate, or somebody, and the thing got pulled.

Now, that product could have pivoted. It was a really good product. It had a lot of really good things about it. There was no physical reason why it couldn't pivot.

But two million dollars, two years and millions of dollars in, with all these expectations and the Google name, I can't imagine the pressure they were under. I felt bad.

So, the right question is, what could Google have done to build that same product and put it in a position where it could have pivoted? I actually think that leadership in today's economy means creating platforms for experimentation for your teams.

I borrow that phrase from Scott Cooks, founder of Intuit, who's been a big adopter of Lean Startup. So if you want to be a general manager in a big company like Google and you want Google to be more entrepreneurial, my point of view is it's on you, not on your team, on you to give them a safe sandbox for experimentation.

So for a risky new product, I would never put the Google brand name on it. For shame. And I would never talk to the press about it at all, if you can avoid it. And I would try to have it have as small a number of customers as humanly possible while still doing that innovation accounting.

Then, teams will pivot just fine. Nothing wrong with Googlers. It's a management problem. In my humble opinion. Yes.

>>Male Audience Member #6: So let me follow up with that directly. Let's say you write a new mobile application and you post it to the app store or the market. If you do that at Google, with the Google name in a blog post, you'll get a hundred thousand downloads no matter what it is.

It can be almost anything. But it's true. And by virtue of that, you are now, when people browse to categories, shopping, personal finance? You will have that visibility and those will lead to clicks and you will be featured. And you will jump start in a way that you can't do if you're nobody and you put up an app out there. You may very well just be in the noise and you'll never break out of that no matter how great the product is.

So, I mean having done this twice -- So I can tell you there's real value. I mean, I know the successes that I've had are in large part because of having that name attached to it.

And that initial leapfrog made up for, was more than I could’ve bought.

>>Eric Ries: Yeah, look, you get free marketing, but so what? Free marketing is worth so much less than we think. Like, yes, that accelerated your process of success, but only because you had the right product and it worked for your customers.

So putting rocket fuel into a rocket that has problems with it is not a good idea, right? You accelerate too fast. The thing blows up. Now, sometimes you achieve liftoff because you actually do have the right product.

But marketing, marketing dollars are not as hard to come by as they used to be. The really great products today have an engine of growth that allows them to grow organically anyway.

So, yes. Google has huge advantages of putting their brand name on it. It can get you a lot of downloads. But also, there's a huge liability about that because—

>>Male Audience Member #6: Well, that's a good question then. So why is it not OK for

Google to fail? Why not take risks and just fall on your face?

>>Eric Ries: Listen.

>>Male Audience Member #6: and admit it quickly and move on?

>>Eric Ries: If it was me, I would be celebrating failures and I would make those people heroes.

If it was up to me, but that's the culture that I live in now. But see, failure is not what we want to celebrate. We wanna celebrate successful pivots away from failure. And that's challenging because it's easy.

If you're charismatic, you can get resources for anything and you can ship stuff and get people to use it and then you can engage in a ton of what I call "Success Theater", to try to make it seem like you're a big success. And so, those people tend to get the celebrations, whether they actually add any value or not. They're like a cancer on most organizations, in my opinion.

But what do we do instead? We don't want to celebrate those people. We don't wanna just celebrate people who fail because they accomplished nothing.

So we have to have a system for evaluating which failures were actually instructive and then we have to celebrate the learning and what the learning turned into in the next try.

So yeah, if Google corporate was fine with people failing and having the Google name be associated with nasty stuff, that'd be fine. But I just think that's unrealistic. I mean, I wouldn't be that comfortable.

If I was Google, I would be launching those things under a different brand name altogether. I wouldn't tell anybody that they're made by Google until they're actually proven to be viable.

How do you think Apple does it. The stuff that we all consume and wait in line for, we're the first people -- the first person who buys an iPad at the Apple store is the first person to have used the iPad? Come on. That's my humble opinion. Helpful?

>>Male Audience Member #6: I didn't follow the last line.

>>Eric Ries: Then maybe I shouldn’t have said it. Cancel that. I do think giving teams a platform for experimentation, having clear analytics for testing whether they're making progress or not, and celebrating pivots is the answer. Whatever Apple does, who knows?