What if every decision you made had a bet attached to it? You could lose something from it. And of course you can gain something from it. In today's episode we talk about exactly this proposition and the idea that actually every decision you make is a bet. We just don't think about it that way. And the value of changing our thinking so that we do that more often. Today's guest is Annie Duke. Annie is a former professional poker player but now she is a consultant and she is interested in helping people make better decisions. We talk about where this interest comes from and this idea that if you can shift your thinking away from black and white decision making and more towards probabilistic thinking you probably have better outcomes. In the second part of this interview we talk about practical ways to approach thinking in bets and there are some ways that you can kind of provide heuristics to your brain, short cuts or you can think about them as mental hacks to help you think more probabilistically. Now let's get straight into the interview with Annie Duke. Annie welcome to the show. Well thank you for having me. I am ecstatic to have you on the show. Shared your talks and your book with quite a few people now and I'm excited to chat with you about a whole list of topics but first I would love to ask you a question that maybe you aren't asked very often. I don't want to introduce you. I want you to introduce yourself. How do you want people to remember you? Oh my gosh. It's like on my epitaph. Or just tomorrow. How do you know if you are not going to somebody about this episode? How do you want to do that? I specifically about this episode because I thought it was going to be like I hope they think I was a good mother. That's great. Actually that's great. Yeah, mainly just good mom. I would say that what I hope that people remember me as is someone who took concepts that are pretty mathematical and complex and managed to communicate them in a useful way to people in a simplified them and while allowing people to execute on the methods of the base wasn't overwhelming them with those mathematical concepts. So in other words, let me show you again, I would say like take, I want people to think of me as someone who's taking very deeply mathematical concepts and putting them into words so that people can understand and execute on them. That's a really cool vision to have because so much of your career and experience has kind of required that of you to be able to understand these concepts and then actually use them in practice. It's one thing to have this theoretical background but it's another to be able to apply that and I think developers actually face this problem but I'd love for you to talk about ways that you see and you can probably hear my dog in the background. What are some ways that you see that playing out and more specifically in your career, what are some times that you have had to take this theory and really see it through? Well, I can kind of think about this. I can kind of think about this on both sides from when I had to be executing the ideas myself and also when I had to be communicating the ideas to other people. Let me start with the second thing first. I think a lot in terms of the way that I communicate about decision making in general and how to think probabilistically and feedback elicitation and the kinds of things that I write about in the book which is just how do you deal with the fact that when you get an outcome that it's hard to know what you're supposed to learn from it. This is kind of the base of what I write about in the book that any particular outcome doesn't necessarily tell you very much about what the quality of your decision was but what we act like it does and how do we kind of deal with that problem. As I think about how did I sort of tackle that issue for myself? On the teaching side, I think back a lot to when I first started teaching poker in particular. I started doing these seminars and obviously there's a lot of math to poker. There's a deeply complex game in terms of the way that you're thinking through the probabilities and what the right choice is and how you're figuring out what other players have and if you figure out what other players have, how you're sort of trying to figure out what the probability of success for different actions are, what the payoffs are. You can hear from just the way that I'm talking about it's complicated. When I first started teaching, I think that I was trying to give the people that I was talking to all of that. I need to tell you what all the math is. I need to tell you how to do these equations. I need to tell you all the things that you should be thinking about so and so forth. I kind of realized that while that might have been really good for me for showing off what I knew in a sense, it wasn't good for them because we learn stepwise. When you take math at school, they start you off with some simple stuff and they start to build and eventually you're doing calculus. Obviously, that's true with anything that you do for developers. It's not like you start off rating incredibly complex code. You start simple and the thing that I sort of realized was that some of the biggest jumps that people can make are in these, what I sort of think about is framework jumps. How are you thinking about the world? Then if somebody's curious beyond that to start really digging down into the math, there's lots and lots of ways for people to go find that stuff out. Getting to that first jump of how do you think about the world in a way that's useful for you solving this problem was much more important than some of this other stuff. Particularly, much more important than me showing them what I happen to know. In poker, one of the big jumps that you need to make in terms of framework is don't think so much about what your hand is, but think about the way that the other person might be viewing you. Think about the way that the other person might be reacting to their own hand. To get outside of your own view and what you saw about your own hand and start thinking about yourself from the perspective of somebody else, that's such a big change in the way that you think about the world that that in itself was such a huge jump in the way that they played poker. Then, as I think about how do you apply that to decision making, that's this really big jump that I'm trying to get people to make, which is when you make a decision, there is not a single outcome that can occur. There's many, many possible ways that the world can turn out. Once you have an outcome, that's only one of many possibilities. There's all sorts of ways in which is human beings. We look at that one possibility and we think it's the only thing that could have happened. We think it's a very, very strong signal for what the quality of the decision was. In this way, it really leads us astray because we lose sight of the fact that there's so much uncertainty. There's such a big intervention of luck between when you make a decision and whatever outcome that you've got. Once we lose sight of that, a lot of things can really start to go astray. Obviously, there's other frameworks that I'm offering within thinking and bets, but as you know, that's the opening framework. Let's think about this problem. There's quite a bit of math to thinking about signal and noise and how large a sample size do you need given the variance in the system in order to start to learn something about the outcome. I don't really talk about that because people can go read Nate Silver for that, for example. I'm trying to get that big shift in the way that people are viewing the world. That's what I'm hoping to accomplish. Hopefully, I've made a little dent. That's what I'm hoping to hope is. I think it's very interesting that you make this distinction between this wide variety of things that you could know or that you could learn all of the intricate details of the math, but that not all of those things will add the same increment of progress or the same increment of value for you. Shifting the way that you think could change your outcome, whatever the outcome is, whether you're playing poker or developing software or any other decision system that you're in, it could shift it by 100%. But then another small thing that may take a similar amount of effort, that might be a very small refinement that changes it by 1%. That's interesting to me about what you're presenting here is that when you look at world class athletes, for example, you're seeing the differences in their, let's say, their 40 yard sprint time. You're seeing those differences in very, very small numbers, right? And so you know that those athletes are focusing on those tiny, tiny details. But when you're in the earlier part of kind of your journey of expertise, when you're just starting out playing poker, you're just starting out sprinting, you're really looking for these bigger, almost heuristic style gains, right? It's think this way, not that way. Rather than move your ankle slightly to the right, it's think about running in a totally different light. These are bigger boulders rather than, you know, tiny pebbles or even grains of sand that you're moving. Right. And I think that that's the thing. It's like once, you know, you're moving, you're making these really big changes in kind of the way that you think. And then obviously, if you want to move toward expertise, those little tiny changes start to make really big differences because now you've closed the gap, right? So now you've got people who are all at a particular level where now small changes make a really big difference, right? That those are the deciders. So I'm focused on the first part, right? How do you teach somebody just basics of form, right? Like, let's think about what the mechanics of running are in general, right? And if you think about the fact that the way that you swing your arms really makes a difference to the way that you run, that's going to make a really big difference in how fast you run. Now, am I going to make you an Olympic sprinter? Well, no, like Nate Silver will make you an Olympic sprinter, but I'm going to get you ready to confront what he what he has to say. Yeah. Right. So so that so that I can sort of pass you on having given you these really good building blocks of how do we think about the world? How do we think about outcomes? How do we think about the way that we're communicating with each other in such a way that either depending on how we do it, we might be a listening feedback that's really, really distorted or a listening feedback that has really high fidelity, right? And that depends on how are we thinking about how we communicate with the world? What are the systems that we're setting in place for ourselves to make sure that that we're getting the best the best feedback? How are we thinking about in general the way that we're scenario planning, the way that we're forecasting, the way that we're thinking about how things might turn out or how things might not turn out? How are we thinking about our ability to really embrace and accept uncertainty and that understanding that not swatting uncertainty away, but allowing it into your life is what allows you to be a better decision maker. Like these are kind of the broad questions that I'm trying that I'm hopefully trying to create like a shift in the way that people are viewing the world and viewing information in that way. And it sounds like I feel a little bit like, oh, I'm biting off this really big thing and somehow I'm doing this. But it's my goal and I know that what I'm doing is what I'm trying to do is make a dent in it, right? If I can get a little bit of a shift in the way that somebody thinks, I mean, I don't think that I can necessarily accomplish all of these big goals that I have, but you know, dream big, right? Right. And then if someone thinks about, you know, an outcome slightly differently, if someone says, oh, you know, just one time, like maybe I'm resulting or well, there are lots of different ways that things can happen. But let me try to think about the likelihood of those things or how am I allowing myself to fall into an echo chamber or any of those things? If I can get little changes in those, I just, I do feel like it has a really big impact on people's lives. I mean, I hope so. That's what my goal is. Absolutely. I think, you know, for me, probably the biggest shift for me was, and I'm going to kind of walk this out slowly. Belief, for me, has always been kind of this mystical thing. That belief is something that you hold that no one should have access to, right? We don't try to influence each other's beliefs because they're almost sacred. And we almost have like this cultural appreciation for avoiding discussion about belief. And I think this, this is a confusing thing because beliefs are kind of conflated with values. And so the thing that really changed for me when I was reading through thinking of beds and watching your talks and, you know, what's this shift, this concept of changing your, your definition of belief to something that is a little bit more about, you know, would you, would you put a bet on it or how much would you bet on it, right? And so demystify this concept of belief. And now I see belief as what I see to be true. Today's episode is sponsored by Century. Century is a perfect sponsor for this episode because when you're thinking probabilistically about the way your application code runs, you can think differently about how to solve problems. For example, if I told you to make a bet about how good your test coverage is, then hopefully you know that you can't make a perfect bet. You can't get 100% test coverage. Why is that? Well, humans are not very good at writing tests. We do okay at it, but we're actually not going to be able to cover all of the use cases because it's really hard to predict, for example, how people are going to interact with your application. So what can we do about it? How can we make sure that we are catching errors when they occur? And hopefully we can proactively fix them before they impact our customer base. Well, Century provides you an avenue to do exactly that. The very first moment any error shows up in your code, Century will alert you in pretty much any channel you can imagine choosing, for example, Slack. And it'll let you know. There is the error occurring, it'll give you a full stack trace, it'll even give you a link back to the commit to the code that actually caused this error to happen. And this will help you track down how you can fix it. Go to Century.io to get started today. Thank you again to Century for sponsoring today's episode, that's Century.io. I really love for you to talk about that for a moment. The idea that a belief is something that you have developed some prediction, right? You've developed some kind of statement about the world. So could you walk that out? I know it's kind of like the main thesis of this book and I'd love for you to explain that. So if we think about, if we think about a decision, we can think about sort of what is the kind of anatomy of a decision or the life cycle of a decision we've got. We have the things that we believe. And that's really what is our model of the world, right? What is our model of the objective truth? So the world doesn't map perfectly, what is actually objectively out in the world or true of the world. We know it doesn't map perfectly onto what our beliefs about the world are. We don't have a perfectly accurate model of the world. The way that we can find that out is pretty quickly. I can just ask you, is there something that you believed when you were 20? You believe very strongly that you no longer believed to that. Right. And of course, when I was 20, I was a totally different person, right? In my mind. And so I can say, oh, that guy was not smart at all. Right. So we know that 20-year-old Jonathan's beliefs did not map perfectly onto what is objectively true. And there's no reason to think that you today, that that would be, that that would all, that your beliefs are also not imperfect. Right. The goal is that we're trying to sort of create this accurate model of the world because those beliefs then inform the decisions that we make. So it informs the options that we think that we have. It informs our thoughts about what our resources are, but what the cost of those options might be. So we can think about, for example, in the development world, there's a cost of time. Right. So one option might take a week. Another option might take two weeks. Another option might take a month. So there's a time, you know, there's a time cost, right? We can think about costs in terms of, there can be reputational costs. There could be costs of money in terms of investment. So we can think about each option costs something. We know that our beliefs are going to drive which options we think we can choose in the sense that we can't choose all options at once. So we have to think about that. And then our beliefs are also going to determine what we think the possible outcomes are of the choice that we take. And also what the probability of each of those outcomes are. So our beliefs are sitting at the base. They're the foundation of every single decision that we make. And no matter what the type of belief is, you are essentially betting on the quality of that belief because it's driving those decisions. So if you think about it like, for example, the fact that you get into an airplane, right? You have beliefs about airplanes and pilots and safety and in this particular case physics, right? Because we assume that the plane will stay in the air and things like that. And you're willing to bet on your beliefs about airplanes by getting into an airplane. It's what causes you to not do certain things like your beliefs make it so that you do not jump off buildings. At least without without like a parachute or a parasympath. Something you don't jump off a building make it. Because we have beliefs about the world and about physics and gravity and things like that that we feel that we would go splat. So those things are really obvious because those have to do with the physical world. But we also have beliefs, for example, about things that maybe we don't end up splat on the ground. But when we think about the decisions we make in politics, for example, we have beliefs about what particular policies are going to cause outcomes that whatever our values are and my values might be different than yours are going to produce the best set of possible outcomes, balancing out what I want for myself versus what I want for society. And I want it to align with what my values are. Right? So that's going to be whether like I beliefs about trade or about climate or about other economic issues outside of trade, about immigration, about whatever it might be. And so I hold these belief systems and when I go to cast my vote, I'm now betting on my beliefs. And we're doing this, for example, like in your world and software development. You have beliefs about how long things will take, what the payoff for those things might be, how often you're going to have success or failure, with whatever it is that you're putting, you have beliefs about what the syntax is, because obviously you can have choices about that and what's going to be best for you. I hear, I don't know, I hear there's like lots of debate about like semi-colons or whatever I'm not a developer. So people have very strong beliefs about those things though, right? Those are almost values to be honest. Well, there's like, so people have beliefs about the littlest tiny detail to the biggest detail of like what are the possible outcomes of this particular, if I code in this particular way. What features do I think that people will like or not like? And literally your experience is informing all of these beliefs that you have and your beliefs are driving the decisions that you make. And so given that at the moment that you go to decide to, like I'm going to develop a particular feature, you're choosing all sorts of things like what language are you developing and like how are you writing that code, how long is it going to take? How does that compare in terms of the resources that you're going to have to put into that versus some other feature that you might develop? My beliefs are driving how much I think that people will like that feature. For example. And so every single day in every single way, whether it's what are you ordering in a restaurant, are you getting on a plane, what project, what feature are you choosing to develop over other features? How are you choosing in particular to code that? Every single decision you make is a bat that's driven by the beliefs that you have. And what that means is that we have to be much more vigilant about our beliefs than we generally are. And we can think about our beliefs kind of, we can broadly divide it and our beliefs as categories. There's stuff we know and then there's stuff we don't know. And we want to think about two things when we're thinking about beliefs because we really want that foundation to be strong. The first thing is how do we get more of the stuff we don't know into our own head? Yeah, yeah, obviously that's incredibly helpful, right? Like I want the stuff I don't know to get into my head. So that has to do with how do you create an attitude toward the world of curiosity and open mind in this because kind of to your point of about saying like you felt like your beliefs were sacred and siloed, that kind of attitude is actually going to reduce your ability to get stuff you don't know into your head. Because you're sort of unwilling to put things up for discussion or to hear what other people say, particularly if it's in conflict with your own beliefs, right? So we want to think about how do I get stuff I don't know into my head? That's really important because that helps to fill in your knowledge gaps. But then there's another thing that's really important to do that we do actually much less of than the first thing, which is how do I make sure that I'm really doing good internal audits of my own beliefs? Because we hold all sorts of beliefs kind of going back to that like, would you believe when you were 20 that you don't believe anymore? We hold all sorts of beliefs that are not completely correct. I mean, they also very often aren't completely incorrect. I mean, sometimes they are. And that's sitting in the stuff I know category, right? Because I think I know it. But you know, it's usually somewhere in between, totally correct and totally incorrect. And very often we are overconfident in the beliefs that we have. And it would be good if we pulled the confidence back. Very often there's calibration that we can do around the beliefs or the belief is biased. And we want to be sort of like having a lot of vigilance around the things that we know or at least think we know that are in that stuff I know box so that we can start to clean that up and get that to be better as well. And by doing those two things like being a really good information extractor, like how am I getting things that Jonathan knows that I don't know into my head, right? And then also how am I doing these really good sort of clean ups around my own knowledge? Now because that's all informing the decisions I make that at its base is going to make me a better decision maker. Yeah. Yeah. And this is such an important discussion for developers. I think, you know, we go through these kind of bike-shag conversations as developers where we're trying to say is this the right technique, is that the right technique? You know, is this framework the right one? This language is 20 times better and I'm going to write 17 blog posts to explain why. And the truth is that very often these things are not in absolutes. And I think this is another takeaway that really helped me kind of see things as well as a less competing and more kind of circumstantial. So you have a decision in a given context, maybe right or maybe effective is probably a better word. Whereas the decision in another given context may be effective. And so to write those in kind of vacuum decision making pods is really difficult to do. You have this human element when you're working with software, you have a human element of this looks good to me and it may not look good to you. So it's not that it's purely subjective but rather that, you know, there's not a 100% right and a 100% wrong most often. Every once in a while, like you're saying, there's that things we know, right, and we can have a 100% confidence or close to 100% confidence, but so much of what we deal with is circumstantial. So much of what we deal with is in that middle ground, even when we're talking about things that are measurable. And so, you know, I think that we end up doing this really bad thing where we judge the behaviors of either ourselves or other people, let's say you're a manager and you judge the behavior of your teammates or the people that you manage. And I think this can be really damaging. I love to talk with you about this idea of being wrong, resulting and judging people based off of those results. And how can we draw that line when, hey, you know what, it's my job as a manager to kind of judge the performance of the people that are managing. So how can I do that while also recognizing that sometimes these results, you know, they aren't a factor of the quality of the decision at all. They have nothing to do with it. Yeah, so first of all, let me just say, like what you just said, I think, is so important on kind of two levels. Actually, I would say three levels. Thing number one is that a decision that's good in circumstance, say, may not be a decision that's great in circumstance B, right, like just just because a particular thing you did was really good in one situation. It doesn't mean it's the right thing in another situation, but also across people that you use that's true, right? So what works for me may not work for you. And just because it works for me, it doesn't mean that I should say that you're doing it wrong because that doesn't mean that it would necessarily work for you or that it would be the right decision for you. So I learned, by the way, I learned that in poker all the time, like I would watch other people play and I would see that they were executing particular tactics or strategies that were working really, really well for them. But I recognize, well, it's important for me and I understand that they're doing this and just sort of see what the value of what they're doing is because I think it's actually a good tactic or strategy, but it's not one that would work for me particularly because I couldn't fit it into sort of the sum total of the way that people viewed me at the table or I was lacking a particular skill that you would need in order to make that tactic work or whatever it might be. So you can recognize that it's not like a one size fits all situation. The framework is, right, but not the actual way that you actually execute. And then the other thing that you touched on, which I think is really important, is that my values might be different than yours. So the conclusion that the outcome that I'm trying to get to might be different than yours. And that's fine, right? So I think that we're very quick to judge when people have different values than we do. Almost judge them as beliefs, right? Right, we do. And here's the thing, like if we both go into a restaurant, you know, you might be trying to get something that's the tastiest and I might be trying to get something that's the healthiest. Yeah. So we could order two totally separate things and both be completely right for us, right? And why am I supposed to look at you and judge you for your, that doesn't make me sense, right? Sure. So I just, I just want to say that. But yeah, so I think here's the problem is that when we start to judge people based on outcomes, there's some bad things that can come from that. So let me just define resulting for everybody. So resulting is saying, if I know the quality of the outcome, that tells me everything I need to know about the quality of the decision. Now that's true for some things. You can do that for some things. So for example, if I'm playing chess and I lose to you and all you know is that I lost to you, we do actually know something about the quality of my decision making comparison to yours. It was worse. So in that particular case, resulting happens to get you to the right conclusion, which is just saying, tell me what the quality of the outcome was and that then, then I can get to the quality of the decision. But for most things that we do in life, that's actually not true. It actually kind of leads us to a very bad conclusion. So I mean, obviously you can think about poker. If I lose a hand to poker to you, that doesn't mean I played it poorly because I could have had the best hand and just gotten unlucky because of the turn of a card. And you know, just something as simple as if I go through a green light, I can get in a car accident. So obviously it would be absurd to say, well, if I know, if I know that you got in a car accident that I know you drove poorly, because we understand that that doesn't have a strong enough relationship between the decision and the outcome in order to be able to get there. And even if you're thinking about something like code breaking, it doesn't necessarily mean that the decision making that led to that was poor because there are unknowns, right? And one of the things that we need to remember is that the thing that we can't know when we're in the process of making a decision, there's one thing that we can never know, which is how it's going to turn out. That's information that only reveals itself after the fact. So if you're in a situation where you're coding and there's different choices that you can make about how you might want to code that, sometimes you can't find out that the code will break, that it will break until it actually breaks. So which sounds simple, but people don't act that way. Right, absolutely. Right. So now when it breaks, they're like, ah, you were stupid, you made a mistake, you know, whatever. No, it's that. Whatever you are. Right. Right, exactly when it's like, no, there were a variety of different choices that I could make. And this seems like the high, seem like the highest probability for things to work out well. But then there was something that I couldn't be for, you know, that I could only figure out was a problem until after the fact. So what I'm trying to do, so this isn't something as simple as that that seems so straight forward. Right. And at the time that you're, at the time that you're actually writing the code, there's different choices that you can make, right? There's different branches that you can sort of branch off on a take. There's different ways that you can write it. And what you're doing is trying to choose what the highest probability of successes for things to actually come out well. And sometimes it breaks. And you can't know that until after that's happened. And then that will sometimes reveal where, where the stress point was, right? You know, what are you doing to people when you're, when you're resulting on them in these cases? You're not giving them freedom to sort of feel like they can try or they can take risks or they could do new things. So if we want to think about like, how could we write the most, the most elegant or efficient code, right? Well, you're going to have to break stuff along the way in order to figure out like, how can I really stream like this or get this as elegant as possible? You're going to have to break stuff along the way. So you have to give people freedom to be able to sort of experiment along in there because that's the way that you find new paths. And that's the way that you find like efficiencies that you couldn't find before or ways that things are more elegant than you could have otherwise seen or things where you can speed things up, you know, or places where you can slow things down. A huge thank you to Annie for joining me on Developer Tea. This is part one, the end of part one. If you don't want to miss out on part two of this interview, then I encourage you to subscribe and whatever podcasting app you use before the end of the episode. Huge thank you to today's sponsor, Sentry. You can find errors in your code before your users leave your application by setting up Sentry. Head over to Sentry.io to get started today. Pretty much every language is supported by the way. So go and check it out, Sentry.io. If you found this episode or other episodes of Developer Tea, valuable to your career or to your personal life or just in general, if you like what we do, the best way that you can help Developer Teaout and help us continue doing this is to leave a review in iTunes. The reason this helps is number one. It helps other developers just like you find the show and decide they want to listen to it. The second reason is it helps iTunes know that there's people out there who like this show. So go and leave a review in iTunes. The best way to do this is just look in the show notes. We've got a link to do exactly that. Today's episode and every episode of Developer Teacan be found on spec.fm and there's a super cool feature we've talked about it before where you can search for different topics across every episode on the spec network. This is not just Developer Tea. It also shows like design details and react podcast and framework and orthogonal and does not compute. These are all shows that you can find on the spec network with excellent content. It's waiting for you to listen to it. Head over to spec.fm to check that out. Of course, today's episode wouldn't be possible without our awesome producer Sarah Jackson. Thank you so much for listening to today's episode and until next time, enjoy your tea.