Inadequate Equilibria ePUB –

When Should You Think That You May Be Able To Do Something Unusually Well Whether You Re Trying To Outperform In Science, Or In Business, Or Just In Finding Good Deals On EBay, It S Important That You Have A Sober Understanding Of Your Relative CompetenciesThe Story Only Ends There, However, If You Re Fortunate Enough To Live In An Adequate CivilizationEliezer Yudkowsky S Inadequate Equilibria Is A Sharp And Lively Guidebook For Anyone Questioning When And How They Can Know Better, And Do Better, Than The Status Quo Freely Mixing Debates On The Foundations Of Rational Decision Making With Tips For Everyday Life, Yudkowsky Explores The Central Question Of When We Can And Can T Expect To Spot Systemic Inefficiencies, And Exploit Them

10 thoughts on “Inadequate Equilibria

  1. says:

    I ended up liking this book a lot, and than I was expecting at first.The first three chapters didn t really say anything I don t already believe and hadn t already thought about a reasonable amount, but they were a clear and entertaining presentation of the material The last four chapters were basically a thoughtful examination of why people hate Eliezer and why he thinks that those people are wrong I m not sure if I agree with everything in them I think that I think it s harder to choose between disagreeing experts in fields in which you are not expert than he thinks it is I agree that if you can just try, and make bets, and see if you re right, and update, that s good I think there are a lot of cases where it s just hard to do this But I think being skeptical of reasoning motivated by status regulating emotions is an important and good thing for me personally, at least to do.If you like Eliezer, you ve probably read this book already If you hate Eliezer, I think you should read this book, and if you still hate him at the end then you can give up If you don t know who Eliezer is, probably don t read this book and read Slate Star Codex instead.

  2. says:

    This book has noticeably changed my thinking when considering new projects Really great epistemic advice, as well as damn fun and easy to read.Some of the useful things inside Microeconomic tools for judging the epistemic incentives of groups of experts Concise and useful explanations for how major insitutions academia, medicine, politics, venture capitalism break Case studies in how to successful use Aumman s agreement theorem in real lifeThe most valuable part of this book by far is chapter 3, which is an extended dialogue between a cynical conventional economist C.C.E or Cecie and a visitor from a different world, trying to explain why a particular institution is killing babies The example is a real one the FDA hasn t approved a simple set of fats for intravenous baby food, causing severe brain damage in something like dozens of babies per year And the explanation is a whirlwind tour of how our institutions work and how they break, with many concepts that I ve been able to use elsewhere to great benefit.Above all, this book has changed the way I think about learning from experts I ll end my review with an example of how I analyse things like this now.Recently someone came to me proposing a project for building a math education website combining math explanations and machine learning algorithms, connecting you with the best explanation given your background knowledge And of my first 3 thoughts, 2 were about how the site would function and how a user would interact with it it s important to concretely visualise a product to figure out if it feels like it would work However, my 3rd thought was to do an adequacy analysis I looked for data points about whether I should expect that, if this is a good idea that can work, why hasn t someone already done it Think about all the money and time people spend on education each year all the tutors in universities, all the teachers in high schools, all the assistant lecturers and support staff and government subsidies Surely, if this project was a good idea, someone would ve built it and it would be a common website we all use Is the fact that this doesn t exist already enough evidence that it won t work In general I don t see the educational marketplace of resources changing rapidly in response to new tech and research There was a bunch of research about spaced repetition and how memorisation works, that nobody has jumped to incorporate into how universities work Many people can tell you the best physics math textbooks e.g Feynman Lectures but most students will never read them It doesn t look to me like a space where it would require a great deal of effort to out do the best in the field You might counter by pointing out the success of Khan Academy, but personally I m not sure I d consider the fact that a single dad can make lots of videos and do better than everyone else, a sign of success for the educational market I m not actually very confident in this analysis However, the best part is that I can learn about the whole field by working on the one project If the project fails, or I find out that someone else has tried this and failed, then I ll change my assessment of other projects in this space On the other hand, if the project succeeds, I will update about how good the educational market is at incentivising people to create useful things like this.I wouldn t have made these models and hypotheses before, and for that reason I m really glad I read this book.

  3. says:

    A quick, useful read The concept of inadequacy that Eliezer introduces here is new to me, anyway and potentially valuable It made me think about certain economic problems in a way that I hadn t before The second half of the book is mostly a defense of the inside view, but I m a member of the part of his audience that s prone overconfidence than under, so for my own good I will take his advice and ignore most of what I read there.Five Things I Want to Remember About This Book view spoiler 1 The housing market is broken because you cannot short sell real estate Houses can be overpriced and they will simply never sell because no one can capitalize off the seller s poor pricing like they could on e.g a share of stock you will rarely find a home that is underpriced because that part of the market functions as intended This leads to a situation where there is a housing bubble, and everyone who s paying attention knows there s a housing bubble, but no one can do anything to correct it because uninformed buyers are still buying.2 You can safely deviate from a modest epistemology when you perceive an inadequacy. An inadequacy generally results when the incentives make a given situation unexploitable e.g you can t short houses , information asymmetry e.g lemons problem , or very broken systems that lead to dominated Nash equilibria which no single agent can fix by defecting e.g coordination problems Inadequacies are situations where the full resources of civilization are not being brought to bear on a given problem, and therefore you can pick up some free utility by taking a contrarian position Eliezer says you may encounter such civilizational inadequacies which you can personally benefit from as often as once a year That number seems very high to me 3 Don t exclusively rely on the outside view Sometimes the inside view is useful Foxes know many things data , hedgehogs know one thing well theories, models usually you should be a fox, but sometimes you need to be a hedgehog especially when your dataset 0.4 Even if you have a good idea, it isn t always exploitable A good idea alone is not enough if you come up with a website that s better than Craigslist, that doesn t guarantee it will replace Craigslist because there are many other factors e.g marketing, network effects, charisma.5 You cannot beat the stock market I should not try to beat the stock market hide spoiler

  4. says:

    In the conclusion of this book, Yudkowsky mentions that the intended audience of Inadequate Equilibira are people with a so called modest epistemology I happen to be one of these people so this book was especially useful to me What Yudkowsky means by modest epistemology is something close to the view that, You can t expect to be able to do X that isn t usually done, since you could just be deluding yourself into thinking you re better than other people There is a lot that this book delves into to fully expand on what this epistemology is and why Yudkowsky doesn t expect it to be useful, but I ll stick to touching on a few things that stood out to me as important takeaways, in a bit First I want to outline some examples and terminology which will serve mostly as a summary of chapters 1 3.First is that inefficiency does not imply exploitability There are places like the stock market where inefficiency does imply exploitability if you know a stock is underpriced before the market does, then you can exploit that but often this is not the case For example, in a housing boom where it is easy to predict that the price of houses will rise and then eventually fall, it is still difficult to enter the market and make a profit since it is overcrowded and difficult to short sell Inexploitability refers to this concept Second is that adequacy refers to whether the low hanging fruit have been picked from a particular sector of the economy For instance, Yudkowsky s wife suffered from Seasonal Affective Disorder SAD They had tried several standard interventions which usually include sitting in front of a bunch of lights and when those didn t work decided to tryadding lights It s a really obvious solution when you consider, like Yudkowsky did, that the sun is just a lot of light And it worked But he couldn t find a single experiment which had tested this hypothesis Why Well, because of civilizational inadequacy, that s why There is an entire chapter in this book that explains civilizational inadequacy in a very digestible and humorous way I liked how he introduced concepts that sound complex on a first pass in an intuitive way, mostly through giving many examples I think the biggest takeaway, and maybe the most incisive way to describe why our society is woefully inept and sometimes insane, is that many aspects of our civilization are caught in suboptimal Nash equilibria, hence the name of this book What this means in intuitive language is that we are caught in systems where everyone in the system can look at it and say, wow this system seriously sucks and does not output what the ideal system would output , and yet it is still in every individual s best interest to act according to the status quo Typically there is an obviously better state of affairs a better Nash equilibrium that we could move to, but in order to do so we need a coordinated effort and buy in from a lot of players, which is difficult For instance, sticky traditions can develop in a system which really shouldn t be there If startups need to go through many rounds of funding in order to succeed, with different people judging them at each step, then if most startup founders have red hair, even if this is an absurd thing to judge the value of a startup on, everyone at each stage has to choose only the red haired founders, since they know no other funder will fund a non red headed founder This is an example where you need a huge coordinated effort i.e., each funder has to agree ahead of time not to judge on this to get past inadequacy Another place where we see an inadequate equilibrium is in healthcare The beginning of this passage is called total market failure , and rightly so Many of the obvious desiderata for healthcare are routinely not met, such as, you know, saving people or providing statistics or results or anything that could indicate to buyers which seller they should go with There are two heartbreaking examples provided in the book where it s obvious just how horrendously broken our society is such that it allows people to die needlessly when there is ample evidence that extremely cheap interventions could save them Central line infections, in the US alone, killed 60,000 patients per year, and infected an additional 200,000 patients at an average treatment cost of 50,000 patient Central line infections were also known to decrease by 50% or if you enforced a five item checklist which included items like wash your hands before touching the line wider adoption of hand washing and similar precautions are now finally beginning to occur, after many years with an associated 43% nationwide decrease in central line infections After partial adoption Another example is that some babies born with short bowel syndrome need nutrition delivered intravenously, and this has historically used soybean oil as its source of fat Switching from soybean oil to fish oil reduces the death rate of these babies from 37% to 9% This was discovered in 2012, and for a variety of reasons, it wasn t until 2015 that the FDA finally resolved this problem.But how do we fix this No hospital wants to be the only one that starts delivering non FDA approved fish oil No hospital wants to be the first to start sharing statistics of how well they perform, which means that hospitals really don t have a financial incentive to, well, actually save people Which means things like central line infections are allowed to keep happening for years It s sobering to see the world this way, to look at society, a system that we re told is supposed to support us and to see all the ways in which it fails us But it s also extremely important not to delude ourselves, and to understand the mechanics of what is happening here Sometimes there will be a way to help, and we have to know how the system works in order to identify those places.Important takeaways things for me to remember mostly from chapters 4 7 People with a modest epistemology rely heavily on outside view Taking an outside view means that you first look for a reference class to which your situation is analogous to e.g., if you re building a new theater, you would compare your project to other theater constructions and then you base your predictions about your project on that reference class Taking the outside view can correct for biases on things like predicting how long your Christmas shopping will take, and how many days it will take to finish a term paper, but, as Yudkowsky points out, it is not useful when the reference class is not obvious and or when there are few instances of the reference class People with a modest epistemology think much like foxes than hedgehogs Fox thinking means one relies on data and observation than theory The latter is referred to as hedgehog thinking It is considered presumptuous arrogant to use the inside view from a modest perspective To this end, Yudkowsky considers how taking the inside view can look like a status grab To take the inside view is to say that you have knowledge that other people don t, and that carries status In particular, I found his point that having ambitious plans as being analogous to saying I m going to be high status in the future very insightful I think this has been a huge problem in my life I don t want to violate status , and from his writing, it seems like the effective altruism community also suffers from this From an outside view having ambitious plans without a background of greatness is like saying, I have no reason to believe that I ll be great, but I m arrogant enough to think I do anyways At this point, Yudkowsky asks us to consider that an inside view with well calibrated models and theories could actually carry an ambitious plan and that we should look at that, and less at status.Anxious underconfidence This is another area where I have struggled immensely with, and Yudkowsky sums it up really well He calls the situation wherein people won t try ideas out of an extreme fear of failing as anxious underconfidence I want to quote this entire section but I ll lead with one particular quote, If you only try the things that are allowed for your reference class, you re supposed to be safe in a certain social sense You may fail, but you can justify the attempt to others by noting that many others have succeeded on similar tasks On the other hand, if you try something ambitious, you could fail and have everyone think you were stupid to try Obviously, not all ideas are good, but like Yudkowsky says, the anxious underconfidence goes to an extreme which isn t healthy people abandoning entire careers because the prospect of failing at one interview was too scary The conclusion of this book wraps up with some tips on how one can gradually come to have a view closer to Yudkowsky s, and farther from modest epistemology These include Try things Fail quickly Do cheap tests Bet on everything Have some common sense Okay so I quite liked this book There were a few things I didn t like, which caused me to give this book a 4 instead of a 5 I ll describe those here.I thought this book varied from being a very straightforward, digestible read to being a bit convoluted and wander y I think this was a balance that Yudkowsky was trying to strike between offering, in his words, explicit principles and implicit mental habits However, I found the wander y parts hard to follow and I often didn t feel like I had a good idea of where he was going with something or what I was supposed to be relating the current passage to All of these confusions were eventually resolved, but the experience of reading them was a bit tiresome Since he was often trying to relay implicit mental habits, some things left lingering confusion which I had to spend extra mental effort trying to resolve For instance, compare the two following passages Startup founder 1 I want to get primitive version of product in front of users as fast as possible, to see whether they want to use it or not.Eliezer I predict users will not want to use this product.Founder 1 Well, from the things I ve read about startups, it s important to test as early as possible whether users like your product, and not to over engineer things.Eliezer The concept of minimum viable product isn t the minimum product that compiles It s the least product that is the best tool in the world for some particular task or work flow If you don t have an MVP in that sense, of course the users won t switch So you don t have a testable hypothesis So you re not really learning anything when the users don t want to use your product The second passage concerns whether or not Yudkowsky and Salamon should plan a lesson for a class or try to improvise the first time The first lesson is to not carefully craft anything that it was possible to literally just improvise and test immediately in its improvised version, ever Even if the minimum viable product won t be representative of the real version Even if you already expect the current version to fail You don t know what you ll learn from trying the improvised version What s the difference between these two I can see a few possible distinctions The first is the amount of time it takes to create anything I don t know what the startup was, but it seems plausible that making anything for users to test in that environment is way time intensive than improvising a class is The second is that you have a better sense for what types of things you ll learn when doing user testing for a startup than you will when you run an improvised class Without having much experience in either this distinction is not obvious to me, but it seems like what he is going for given the last sentence of the second passage Regardless, I wish he had spent some time highlighting his implicit models which concluded that these were distinct situations Overall, I really enjoyed this book I don t recommend it to everyone since it is somewhat depressing and a bit technical However, if you consider yourself anxiously under confident and or modest, I think it s well worth reading Now onto truly internalizing this mindset

  5. says:

    Some very important lessons here that I want many people to read, however it s not always easy to parse unless you have a lot of experience with Eliezer s ways of thinking and use of vocabulary already.

  6. says:

    A fantastic book, Inadequate Equilibria, is about how to think about certain things that affect your own life in other words, what model to use and thus how to improve your decision making and your life with it.The author points out, through concrete examples, how certain systems are badly broken, stuck in an inadequate equilibrium, sometimes with dire consequences like in the case of feeding a bad source of protein to babies in the US that still causes several thousand deaths and explains in details how this is possible The way he explains the completely avoidable deaths of thousands of babies through a conversation of a visitor from a Better World, a Conventional Cynical Economist and Simplicio, a major university student with no knowledge in economics is simply brilliant and makes that long chapter not only extremely insightful but also fun to read.Another great thought that Yudkowsky is making is that the modest epistemology , thinking that I m no smarter than the experts in a field and thus cannot know better, is very often the wrong way to approach a problem because of all the brokenness of systems a good example is how he cured her wife s SAD with a simple method that wasn t published.Read this book.

  7. says:

    I m just going to get it out there Yudkowsky, along with Scott Alexander and SSC, LessWrong ers, rationalists, etc , irritates me on a personal level Is my review biased based on this Yeah, probably, so you can consider it with that in mind That being said, there are at least snippets of wisdom in this book Usually, when things suck, it s because they suck in a way that s a Nash equilibrium So far, every time I ve asked you why someone is acting insane, you ve claimed that it s secretly a sane response to someone else acting insane Where does this process bottom out The Gell Mann Amnesia effect You will detect inadequacy every time you go looking for it, whether or not it s there If you see the same vision wherever you look, that s the same as being blind You can say holy shit, everyone in the world is fucking insane However, none of them seem to realize that they re insane By extension, I am probably insane I should take careful steps to minimize the damage I do When you previously just had a lot of prior reasoning, or you were previously trying to generalize from other people s not quite similar experiences, and then you collide directly with reality for the first time, one data point is huge If you and a trusted peer don t converge on identical beliefs once you have a full understanding of one another s positions, at least one of you must be making some kind of mistake If we were fully rational and fully honest , then we would always eventually reach consensus on questions of fact Hey Guys I found out how to take over the world using only the power of my mind and a toothpick You can t do that Nobody s done that before Of course they didn t, they were completely irrational But they thought they were rational too The difference is that I m right They thought that too If just anyone could find some easy sentences to say that let them get higher status than God, then your system for allocating status would be too easy to game Try to make sure you d arrive at different beliefs in different worlds You don t want to think in such a way that you wouldn t believe in a conclusion in a world where it were true, just because a fallacious argument could support it Emotionally appealing mistakes are not invincible cognitive traps that nobody can ever escape from Sometimes they re not even that hard to escape.

  8. says:

    It s no secret that I m a fan of Yudkowsky s writing and singular style of critical thinking This book gave me no reason to change that In the words of the author himself This is a book about two incompatible views on the age old question When should I think that I may be able to do something unusually well Those two views are epistemological modesty and inadequacy analysis, and the book opens by contrasting them through various examples.Yudkowsky examines the topic of civilisational inadequacy using the terminology of contemporary economics In particular, the concepts of efficiency and exploitability are used to provide a framework through which the shortcomings of incentive structures can be understood.Yudkowsky s most poignant example of this is how, in the US, babies with digestive problems are intravenously fed formula with an imbalanced lipid profile, leading to liver damage and death See here Whilst superior formula exists and has been demonstrated to drastically reduce mortality, the only way for babies in the US to receive it is if 1 they already have liver damage, 2 the doctor parents are aware of the problem and the superior formula, 3 the hospital is legally allowed to import the superior formula Yudkowsky then challenges the reader to explain this to a visiting alien race and defend humanity s apparent disregard for the lives of babies The book goes on to examine the different elements of Moloch s Toolbox, how inadequate systems can be identified, and the challenges faced by those attempting to remedy the shortcomings of society.Whilst this work definitely appeals to a Rationalist worldview, I suspect that anyone with an interest in basing their decisions in reality will find it not only fascinating but also truly useful It s also freely available online Some of my favourite extracts follow Efficiency Microsoft s stock price is neither too low nor too high, relative to anything you can possibly know about Microsoft s stock price Inexploitability Some houses and housing markets are overpriced, but you can t make a profit by short selling them, and you re unlikely to find any substantially underpriced houses the market as a whole isn t rational, but it contains participants who have money and understand housing markets as well as you do Adequacy Okay, the medical sector is a wildly crazy place where different interventions have orders of magnitude differences in cost effectiveness, but at least there s no well known but unused way to save ten thousand lives for just ten dollars each, right Somebody would have picked up on it Right If you want to outperform if you want to do anything not usually done then you ll need to conceptually divide our civilization into areas of lower and greater competency My view is that this is best done from a framework of incentives and the equilibria of those incentives which is to say, from the standpoint of microeconomics This is the main topic I ll cover here If I had to name the single epistemic feat at which modern human civilization is most adequate, the peak of all human power of estimation, I would unhesitatingly reply, Short term relative pricing of liquid financial assets, like the price of SP 500 stocks relative to other SP 500 stocks over the next three months This is something into which human civilization puts an actual effort A market that knows everything you know is a market where prices are efficient in the conventional economic sense one where you can t predict the net direction in which the price will change We can see the notion of an inexploitable market as generalizing the notion of an efficient market as follows in both cases, there s no free energy inside the system In both markets, there s a horde of hungry organisms moving around trying to eat up all the free energy In the efficient market, every predictable price change corresponds to free energy easy money and so the equilibrium where hungry organisms have eaten all the free energy corresponds to an equilibrium of no predictable price changes In a merely inexploitable market, there are predictable price changes that don t correspond to free energy, like an overpriced house that will decline later, and so the no free energy equilibrium can still involve predictable price changes I ve seen a number of novice rationalists committing what I shall term the Free Energy Fallacy, which is something along the lines of, This system s purpose is supposed to be to cook omelettes, and yet it produces terrible omelettes So why don t I use my amazing skills to cook some better omelettes and take over And generally the answer is that maybe the system from your perspective is broken, but everyone within the system is intensely competing along other dimensions and you can t keep up with that competition They re all chasing whatever things people in that system actually pursue instead of the lost purposes they wistfully remember, but don t have a chance to pursue because it would be career suicide You won t become competitive along those dimensions just by cooking better omelettes What inadequate systems and efficient markets have in common is the lack of any free energy in the equilibrium We can see the equilibrium in both cases as defined by an absence of free energy In an efficient market, any predictable price change corresponds to free energy, so thousands of hungry organisms trying to eat the free energy produce a lack of predictable price changes In a system like academia, the competition for free energy may not correspond to anything good from your own standpoint, and as a result you may label the outcome inadequate but there is still no free energy Trying to feed within the system, or do anything within the system that uses a resource the other competing organisms want money, publication space, prestige, attention will generally be as hard for you as it is for any other organism.

  9. says:

    I first picked this up many months ago, and at the time I found Eliezer s writing a little difficult to parse, but in the intervening months I ve spent a lot of time in close proximity to rationalist friends reading thematically related books, so when I picked this up again I found that I actually had the vocabulary and mental framework for most of this to be coherent and useful I have longer book notes somewhere else, but the essence of the book is how do you know when you can do better than what exists with some amount of effort, thought, etc It applies to problem solving at various scales eg personal medical issues, starting a startup, solving some global crisis, changing The System if a solution exists, why hasn t it been solved yet Two main reasons 1 misalignment of incentives the chapter on inadequacy analysis moloch s toolbox was interesting, if a little long, and reminded me a lot of the book the elephant in the brain the idea of looking beneath the surface explicit purposes of a system, and understanding what s really motivating the individual actors It brings in concepts like the principle agent problem, common knowledge, etc, and takes the frame that systems aren t suboptimal just because humans are stupid, but rather that we re responding to sometimes non explicit incentives 2 limited resources time some other factor X, opportunity cost this is where we may be able to find opportunities to do things differently, if we are particularly endowed in some relevant area or uniquely determined passionate knowledgeable about some issue, etc This doesn t mean that all problems are solvable, better states exist, etc But there are opportunities to do better sometimes, and it s about understanding the system thoroughly AND coming up with the right ideas AND putting in the work to actually do it for example, so much of Uber s contribution as a company was to make things work with regulations The concept of an inadequate equilibria seems like a useful model, and the ability to identify its sources and figure out if you can transcend that with effort seems like something you can get better at with practice As someone who tends towards modest epistemology, this is really relevant to me If you ve never wasted an effort, you re filtering on far too high a required probability of success Aside I am a fan of thermodynamic analogies To take if further hello high school chemistry a lower enthalpy favorable state exists, but the energy barrier means that it takes a certain energy catalyst pizzazz to get there.

  10. says:

    The author combines the outlook of a physicist, economist, stock trader, psychologist into a theory that involves viewing rationality as a way to extract and exploit economic goods ultimately the way living systems are on the hunt to extract free energy against the second law of thermodynamics A little grim our situation when you think about it However, the book spends a lot of time on the efficient market hypothesis that the market is statistically smarter than you at figuring out the future value of a product Philosophically although I reject neoclassical economics, I get the dilemma of when to follow the crowd or consensus or the experts and when to bet against such things is really tricky All progress went against some consensus but a good deal of folly did as well One of those tough calls I don t buy the author s political instincts but he has a lot of incisive and interesting things to say about this dilemma.