My views on crime and justice are fairly unusual for my class. I tend to believe that:

• crime is a largely solvable problem with massive low hanging policy fruit. Basically, we could be pretty similar to Singapore if we just chose a slightly different policy mix
• the solution to criminality is longish prison sentences for most crimes, especially for repeat offenders or serious crimes
• this solution has massively positive EV for society generally but especially for those who are lower down the social-econ spectrum and thus suffer more from crime in various ways (victimization risk, risk of being seen as a criminal, etc…)

I spent some time a week ago debating criminal justice with a few friends. I think one things I lacked before then was a clear idea of what the cruxes of my beliefs are. After some discussion and thought, I think the core cruxes are as follows

• The distribution of criminality follows a power law
• Criminals are largely irredeemable (at least using current tools) other than through aging-out effects

The reason I believe criminals are hard to reform is that successive governments have tried and yet reoffending rates remain extremely high. The UK’s known reoffending rate within 1 year of release is like 30%. Considering how low the UK’s detection and prosecution rate for most crimes is, that implies a sky-high true reoffending rate let along the rate of crimes per released prisoner.

The reason I believe that crime has a power law distribution is mostly just reading about it. Most of the stats show it. My personal experience with it in school/my estate growing up is similar. Most of the problems seemed to come from a tiny % of people who consistently made life hell for everyone else.

What would I need to learn to change my mind? I guess one of a number of things could change my mind here:

• Evidence of scaleable, cost-effective non-prison ways to stop criminals doing crime. This could be rehabilitation but could also be other things like home-arrest, electronic monitoring etc…
• Evidence of a replacement effect. AKA, would criminals who are locked up just get replaced by the next best criminals from the population? If individual propensity to offend fairly fixed or are there large pools os plausible substitutes available?
• Evidence that the cost of imprisoning criminals is higher than the social cost of the damage their actions cause. (AKA: if the cost of crime to society/people is low, then maybe prison is overkil as a solution)

Let’s say you want to find out roughly how much of a country’s economy is controlled by the state vs the free market. How would you do so? My first instinct would be to look at state spending as a % of GDP, or maybe taxation as a % of GDP. If that state spends 90% of GDP, it’s pretty clear the government controls mostly everything and visa versa. Now, this isn’t a bad approach. State spending is a good indicator. The problem is it only captures a part of reality.

Imagine two countries. In both the state taxes and then subsequently spends 50% of GDP per year (no deficit or surplus). But let’s imagine a few different ways they could have more or less state control

• Both states want to use their % of GDP to buy the same basket of goods. State A uses it’s tax revenue to spin up state owned enterprises. These state owned firms only get inputs from other state owned firms and produce fixed quotas of certain goods. State B runs a competitive bidding process and then has it’s departments (e.g: the army) buy from the market.
• State A gives decides what the nation needs and spends all the revenue on that basket of goods. State B spends 10% of it’s tax revenue on core needs (the military, police, judiciary, etc…) and gives 90% of it back to citizens as direct cash transfers. Citizens then spend this money on whatever they want to buy from the private sector
• Both states spend their tax revenue in an identical way. Alongside this, state A imposes immense restrictions on the private sector. There’s a license raj. What you can build. Where you can build. Which specific business structures and processes are allowed and which aren’t. Everything is regulated down to the T. Opening a corner store or selling a new kind of Tea requires months of form filling and lobbying. State B largely lets the private sector do what it wants.

TLDR:

• % of GDP is a good indicator of state vs market economic control, but it’s a faulty one at best.
• If you want to increase economic freedom, it may be worth focusing more on specific, massive distortions of the private sector (zoning/planning, migration, etc…) than on decreasing the state’s tax take by a percentage point or two

Lisa wrote an article on coercion, paternalism and irreversible decisions. I have some thoughts.

The basic groundwork goes something like this:

• coercing people is bad
• coercing people to prevent them infringing on the rights of others may be justified in some cases
• coercing people to stop them doing something they freely want to do, and which has no impact on others, is on face value unjustified.

When can paternalistic coercion be justified? A few obvious justifications I can think of are

• When a person is mistaken about the consequential outcomes of an act. (e.g: John thinks that taking act X will give him $1’000. Actually it will give him with -$1’000)
• When a persons actions would lower their own utility
• When a persons choice is not a reflection of their “true” preferences. (e.g: an addict who wants drugs but doesn’t want to want drugs. A person in a temporary depression who wants to kill themselves)
• When a persons choice has overly large effects on their future self, and those effects are locked in. e.g: selling yourself into slavery

The “You’re Wrong” justification for paternalism

This justification is basically that someone is mistaken about the practical consequences of a choice they’re going to make. Hence coercing them to do what would best suit their preferences is okay.

On one hand I’m not too convinced by this justification. If someone is mistaken about the consequences of a given action, you can always tell them why you think they’re mistaken. If they still disagree, it’s not clear what justification you have for coercion. Imagine you live in a society where the following is true

• There are shops which sell “ill-advised consumer goods”
• the goods in these shops are unregulated and hence often dangerous or harmful to the user
• the government specifically advises you to not buy things from these shops and entering them requires you to sign a waiver stating you understand that it’s a bad idea for most people who do so
• If you don’t trust yourself to make these kind of decisions in the moment, you can also choose at the beginning of every year to be banned from ill-advised goods for the entire year.

In this kind of society, people are very clearly

• given a choice of whether they want to defer to the state/experts or not (the annual opt-in)
• made aware of why society thinks a certain act is bad

I guess the general argument here is that if you think someone is mistaken, you can tell them why you think they’re mistaken. If they still disagree subsequent to hearing your reasoning, I’m really not clear why it’s okay to assume that you’re right, they’re wrong and the use of coercion is okay.

The utilitarian justification for paternalism

Not much to say here. I think some (many?) people are dumb and make bad decisions. Forcing them to make better decisions can raise their utility, whether measured by their own subjective preferences or by some kind of objective list variant (e.g: hedonic utilitarianism). Whether in practice this forcing is actually effective depends on the society/government/political equilibria but in principle many such paternalistic interventions exist.

If you’re a hardcore utilitarian, this is enough to convince you. For me I value a plurality of goods with freedom from coercion and utility being two of them. I guess in some cases the tradeoff between utility and freedom is sufficiently lopsided that I would think paternalism is okay. That being said

• I think my intuitions generally put a lot more weight on preference consequentialism as opposed to any kind of hedonic utilitarianism. Hence if someone chooses not to wear a seatbelt, I would tend to be okay with that decision.
• I worry that as a social technology, paternalism usually increases in scope and severity over time as the policy ratchet goes primarily in one direction.

I guess the real crux here is between hedonic vs preference utilitarianism. If you’re a hedonic utilitarian, paternalism is straightforwardly morally fine. If you’re a preference utilitarian, as I think most of us are, then it doesn’t.

The false vs real preferences justification

Heroin addicts want heroin. They don’t want to want heroin. Hence it’s okay to deny them heroin and force them into rehab because that matches their true underlying desires.

I find this argument fairly persuasive. I guess the only difficulty I have is in how we distinguish between real and false preferences, maybe let’s call them higher and lower order preferences from now on. A bad way to do this is to look at what most people do and want, assume this is normal and then judging deviations from this as increasingly likely to be false preferences. A better way is to see what people say they want when the immediate need for a thing is satisfied. There’s still some difficulty with different versions of a person having different desires and how far we value the preferences of short lived personas of a person verses their more normal persona, but hey that’s a tricky problem generally.

The future self justification

Some decisions you can make will unduly constrain your future self. E.g: living like a king for 10 years in return for being a slave for the next 50 and being abused constantly. This is wrong because you remove the agency of your future self, harming them. Consider a similar case where you live like a king for 10 years and then have to put on a mind-wipe headband before having a different person overwritten onto your brain who is then enslaved. That would also be bad for the same reason.

I’m not sure here. I think this argument raises a lot of genuinely thorny/difficult problems that are hard to resolve in isolation. Some thoughts:

• You’re not actually harming the future person because absent your actions, they would not exist in the fist place. (What’s the name for this? Is this true?)
• All actions you make constrain your future self.
• To what extent do your present choices constrain a future person vs determine which of many different future people will exist?
• The whole future person thing seems contingent on a view of identity that is not based on continuity of conscious experience.

So yeah, this is pretty messy and I think pretty much entirely depends on what your view is on identity + whether non-instantiated agents moral preferences matter. My view is that identity is basically mindspace similarity (aka how similar is your mind to the other agent) and and non-instantiated agents preferences matter an equal amount. I’d hence say your future self, provided it’s sufficiently similar to you, counts as pretty much the same agent and you can make decisions for the both of you. (Although you should probably have a discount rate of 0 or close to it for you’re future self’s utility).

All in all

I think paternalistic coercion is sometimes justified. Probably when someone has explicitly said they’d rather not have a preference/do an act and the version of them that says that is either reasonably similar to the version of them that does the act. In theory I’m not okay with “you’re wrong” type paternalism but in practice I think it’s often okay because the political reality is such that the choice is between “paternalism to stop dumb people killing themselves or teens trying super heroin” and no paternalism, not magical libertarian dangerous goods shops.

(Positive Srdjan) An argument I read years ago when first hearing about EA goes something like this.

• many people think that becoming a doctor is really morally good
• this actually is much less true than commonly assumed because the impact of becoming a doctor is fairly low
• yes a doctor saves many lives, but if you don’t become a doctor that doesn’t mean that the people you would have saved die. Rather it means that the place you would have taken at medical school is taken by someone else. In effect, the whole stack of applicants below you is shifted one place up.
• hence your impact if you work as a doctor is not the number of lives you save, but rather the extra lives you save vs the counterfactual world where someone else with extremely similar grades had taken your place.

I always found this argument persuasive. Of course there are counterarguments that poke at the calculus. It could be the case that moving the whole stack ranking down has a significant overall effect or that you as an EA doctor would be especially effective. Still, the core insight that what matters morally is counterfactual impact was interesting and seemed obvious.

(Negative Srdjan) What about drug dealers?

Let’s say the market for drug dealers is fairly efficient. By becoming a drug dealer, you only marginally change things. Or what about being a concentration camp guard in WW2. If you don’t do it, there’s an endless supply of soldiers who want to not go to the eastern front who would take the job. Your counterfactual impact by doing it is super low or maybe even zero. So it’s completely morally fine, right?

(Positive Srdjan) Hmmmmmm. So this get’s tricky. You’re right that there’s a clear difference here between the consequential impact of an action and the amount of moral blame/praise we assign for it. Many things we would ordinarily find good/bad seems to not really make sense under a consequential moral framework. I can think of a few different ways to resolve this:

1. Pure vs practical ethics. Counter-factual impact is the correct true moral standard for judging behaviour and choices and the one you should use when making decisions about what to do. Other norms (doctor good, concentration camp guard bad) are useful social norms to have. They encourage people to do pro-social things and lead to better equlibria. e.g: If we shame anyone who is a concentration camp guard, we get less concentration camp guards.
2. There’s a difference between judging moral acts and judging people. When judging which act is correct, counter-factual impact is fine. When judging people, we don’t just care about terminal impact. Otherwise attempting to poison someone but instead putting vanilla in their food would not be bad. Instead we care about assessing a persons character and their propensity to behave in various ways in the future. The kind of person who does certain things: e.g: becoming a firefighter, becoming a concentration camp guard, tends to be better/worse. Hence these actions are useful signals. The whole doctor/guard intuition mismatch is just the result of confusing two different questions: what’s the right act vs what does a given act tell you about someone’s character.
3. Our deontic moral intuitions are just wrong and being a concentration camp guard is fine if you’re not doing additional harm
4. Our consequential moral intuitions are just wrong and acts can be really bad even if the outcomes are net zero or positive
5. The “goodness” of an act is combination of various different moral values we hold. So partly determined by consequences, partly by other considerations.

I’m leaning most towards 5 but I think 1 and 2 offer some interesting avenues of exploration. E.g: how does ethics work when your actions have nonlinear effects on other agents (e.g: Act X has +50 utility if only you do it, but -50 if more than 50% of people do it). When doing things, in the real world how much of the actual impact comes from the direct outcome of the decision vs the effect sticking to or defecting has on overall norm stability?

(Negative Srdjan) Either way, I think that one thing is clear here. If you suspect that being a concentration camp guard is morally problematic, or that paying 5$ to kill a person who would be executed anyway the same hour is not great, maybe you should be a less quick to accept the EA argument for counterfactual impact.

(Positive Srdjan) I don’t know. For the examples you give I agree but I feel like there’s a lot of cherry-picking going on here. Yes in extreme cases you maybe shouldn’t do things that are deontologically bad or defect from agreed-upon equlibria. But the vast majority of decisions in real life aren’t like that. Most decisions are mundane and do not involve any serious moral transgressions. If you’re deciding between working on management consulting vs being a doctor, where to give your money or any other of the 99% of decisions you make that don’t involve starting a coup/killing babies, then the counterfactual-impact standard is probably the one I would advise you to go with.

Also, a question here is what is the average person like? If we lived in a world of committed act-utilitarians, then maybe I would be happy to advocate for more deontology. I think the reality is the opposite. Most people lean far too far towards act deontology and don’t really consider or understand the counterfactual impact of their actions. Hence the advice to consider that impact more is probably net benificial.

In this twitter post, Richard argues that in the distant future there will be far more zero or negative sum interactions because once all possible value has been wrung out of the physical world there’s noting left to compete over other than the distribution of that value. I wonder if this is true?

Imagine a far future where the following hold true

• the physical universe is divided among N agents
• each agent has fully exploited the physical resources in it’s territory to best satisfy its utility function (some agents turn everything into computronium and simulate endless minds experiencing pleasure, others turn everything into tessellating atom-scale paperclip replicas)
• we’ve reached the limits of scientific progress. Each agents ability to convert the matter/energy in their territory to utility is capped out and cannot be further improved

(Optimistic Srdjan): Even if agents know the same things, they can still plausibly gain from trade in a few ways.

• It could be the case that different kinds of agents are innately better at doing different kinds of things. Different kinds of minds have different strengths after all. In that case trade can be mutually beneficial as it allows for more specialisation and more stuff for all parties. Basically, as long as agents differ and those differences translate into some degree of difference in effectiveness in the material world, trade can make sense.
• Maybe scale has returns. Maybe a computronium matrix the size of 10 galaxies is marginally more efficient per unit of entropy than one the size of 1 galaxy. In that case trade again can be win-win.

(Pessimistic Srdjan) Hmmmm. So the argument here is basically that trade can still make sense even if knowledge is capped out provided either agents abilities differ or scale matters. Okay, lets grant that both of those conditions hold. I’m still not sure that argument works.

Sure, in the initial state the agents will trade. They’ll collaborate to build larger computers and split the compute time gains from doing so. They’ll specialise in producing things they’re better at and also split the gains from trade. Fine. But what happens once the agents do that for long enough to eat all the gains. Eventually the final equilibrium is:

• The agents have built a shared computronium matrix that spans the entire universe or hits scaling limits
• They’ve produced all the widgets they need in an optimal way and have stockpiles/futures contracts that will last until the heat-death of the universe. Surely at some point you exhaust all the gains from trade, right?

I guess the meta point here is something like this:

• You start with a pool of possible trades. Some, e.g: war, are negative sum. Some are neutral. Some are positive sum.
• You take the positive sum trades.
• Eventually you’ve taken all the positive sum trades and have only negative or zero sum ones left

(Optimistic Srdjan)

Hmmmmmm.

I’m tempted to start arguing the object level point here but I think that would be a mistake. Maybe it is the case that if you go sufficiently far into the future, all possible trades are made and all positive-sum potential is tapped. Still, that seems somewhat different from the question we were initially asking. The question was something akin to “Is there any need for trade in a world where material resources are all fixed and fully-owned and all agents have full knowledge”. I think the answer to that is a yes, possibly. The question of “Will there eventually be no need for trade once we finish creating the optimal pan-galactic compute cluster” or “Will AI’s 3 days after birth make all possible trades with precomittments spanning to the heat death of the universe, meaning that there is no real new trade from that point on” are harder to think about but also meaningfully different.

Imagine a bag of pebbles. It’s deep. You don’t know how deep. You draw a thousand pebbles. They’re almost all white. Some are grey. Each white pebble you draw makes you stronger, happier, better. Grey pebbles hurt you a little bit but it’s just a twinge. A thousand pebbles in you’re much better off than when you started. That being said, you know that it’s possible that the bag also contains black pebbles. You also know that if you draw a black pebble it will immediately kill you. Should you continue drawing pebbles? If not, at what point should you stop? 0 pebbles? 100? 1000? Does the fact that you’ve already drawn 1000 pebbles and not a single one has been black influence your probability assignment to probability of drawing a future black pebble?

The bag of pebbles analogy, or something like it, is what Bostrom uses to argue that technological progress may well lead to existential risk. I’ve thought about this a bit and I’m not sure what my conclusion is. I think the bag problem raises a bunch of interesting question both philosophically and practically.

Philosophically, there a few interesting problems here. The first one is that we don’t know what the % chance of drawing a black pebble is which makes any EV calculation really tricky to do. It can be tempting to take the naive approach and reason inductively here. “I’ve drawn 1000 pebbles and none have been black. Hence that’s strong evidence that future pebbles won’t be black and/or that the bag contains few black pebbles”. That’s doesn’t really work for a few reasons:

• The boring problem: the distribution isn’t fixed. The further down the tech tree you go the wackier things get. A medieval steel-smith finding a way to forge swords which results in swords that sharper and cheaper may upset a balance of power or disrupt some political equilibria, but it won’t kill everyone, wipe out the biosphere or cause a vacuum collapse which destroys spacetime. Future tech we find (engineered pandemics, vacuum decay bombs/plank worms, etc…) might. To link it back to pebbles, the more pebbles you pick the deeper into the sack you start going and further down layers have a different composition.
• The interesting problem: anthropic shadow. If we had picked a black pebble in the past, or even just a very bad tech, there would be far less people alive today. Hence you probably won’t have exist. Hence the fact that we look around and observe “hey, we’re in a world where no terrible tech was picked which wiped out civilisation” may not be actual evidence as much as it is a selection effect. Trying to do inductive reasoning without taking into account that the environment affects whether you exist and hence what you can observe is akin to concluding that god exists because the universe is fine tuned to support life. (Also, on a meta level I’m really uncertain about anthropic shadow and anthropic reasoning more generally. I plan to do some work at some point to study it and come to a conclusion but until then I’ll leave this here) So if we can’t inductively reason about the distribution of future tech and we don’t have any way of choosing a reasonable prior what then? How do you even approach the problem?

Ok. Putting aside the epistemic confusion there are a bunch of interesting pragmatic considerations:

• Is it possible to control tech progress?
  • It seems practically impossible. We’re in a multi-actor system where every actor benefits hugely from increasing tech. One view is that absent an incredibly repressive global super state, tech will continue to advance. Still we’ve stopped techs dead in the past. Human cloning. Embryo selection. Eugenics. Inadvertently nuclear power.
  • What about avoiding certain specific techs but still making overall progress? Maybe we just blacklist specific technologies or applications, This may work but I’m sceptical for a few reasons. 1: Time. Over the next 1000 years I’m sceptical the genie will stay in the bottle. 2: Ease of access. As the tech level generally rises it becomes easier and easier to research/implement previous techs. What takes a nation state in 2020 may take a team of PHD’s in 2050 and a lone smart teenager in 2200.
• Can we see it coming?
  • It’s really, really hard to predict long term trends in technology. It’s also really hard to predict the social effects of new forms of tech. Who would have thought that access to ultra violet lithography would be an important strategic capabilities for nations in the 21st century? Which Chinese chemist in 900AD would have predicted that gunpowder would contribute to the end of Feudalism and centralisation of power by making castles less and professionalized armies more important. Can we even tell in advance which techs are dangerous?
  • We can certainly do some prediction. If tomorrow there are simultaneous news reports of a new way to synthesise silicone based bacteria + a new way to make mechanical keyboard keycaps I’m pretty confident assessing which has a greater risk. Still, maybe our predictions on average only become accurate enough 5 years out from really bad tech being realised. Maybe some really bad techs aren’t obviously bad and their dangers only become apparent when they become widespread or hit a crucial threshold. (e.g: fast takeoff AI scenarios)
• What’s the tradeoff? What do we give up by slowing tech progress or banning techs?
  • There are natural X-Risks. If we magically stopped tech development at our current level it would just be a matter of time until an asteroid strike, plague or eventually the sun dying would end consciousness in our galaxy.
  • Tech development also increases the number of humans alive at any given time. 10 billion humans for 100 yeas > 100’000 humans for 10’000 years.

I talked to Andrea last night. Turns out I misunderstood what the St Petersburg paradox was.

• I thought it was that accepting an infinite series of 50/50 bets to double or go to 0 leads to an infinitely small chance of a non-0 outcome.
• She says it’s actually that if you have an infinite number of trades, you never cash out your winnings and instead just end up trading and trading.
• Chat GPT and wikipedia say it’s about expected outcome calcs leading to different answers than what people will actually do (the original paradox was about bets in money, not utility, so risk aversion makes sense given diminishing marginal returns).

Of the three interpretations, I find none particularly convincing.

• The fact that an infinite series of 50/50 bets leads to an outcome space where value is very concentrated in one tiny part of the utility space isn’t particularly interesting. Whether making that choice is sane or not depends entirely on the agents utility function and risk attitude. (If presented with multiple options with the same EV but different degrees of variance, what would the agent prefer)
• Being trapped forever in trading is silly. If trades take time then the disutility of a trade is something you need to account for. If they don’t you don’t. Also, the word “trade” here doesn’t have the same meaning as “trade” in real life. It’s not a commercial transaction but the act of choosing from among a few options. The idea that you’re wasting your life by trading instead of touching grass isn’t really sensical. You’re either saying that people shouldn’t waste time making choices or that trade-bot 3000 who exists only to transact should have a different utility function. Both are silly.
• The oldest version of the paradox dealing with real money trades is dissolved by the realisation that money and utility are not interchangeable at a 1:1 ratio. For the vast majority of people, money has steeply diminishing marginal utility. Hence it makes sense to be risk averse even when offered trades that are EV positive in $ terms.

I read a lot of blogs. Around 200 articles per month get streamed to me via RSS. A lot of those are read and forget but a few are truly great. In fact, they’re so good that certain collections of articles, often times from a single author, rival many of the good books I own in quality. Therin lies the problem.

Books are great for a few reasons. They’re pretty. I can give them to friends easily. They usually form a collection of one authors thoughts. I would like to have an affordable, not super manually intensive way to take a collection of articles or a source, e.g: a substack or blog, and a few days or weeks later get a book shipped to my door. This is something I can do right now but it requires a huge amount of manual faff with formatting, downloading posts, selecting a printer etc…

I would love a website where I can

• put in a link to a substack, blog etc…
• get a UI showing a list of articles by publication date
• select/unselect articles
• get a printed book delivered to my door a bit later with correct formatting, images intact

I can imagine a bookshelf filled with collections of essays from The Diff, Slate Star Codex, Robin Hanson, etc…

Why would this not work:

• how many people actually want this? Is there a market?
• how much does it cost per book? Low volume = high unit price. + the fixed costs of typesetting and making substack posts pretty.
• how does this work with copywright law?
  • printing private articles from behind a paywall and then distributing them is illegal I think. Is this also true if you give a customer the ability to put in a URL and then print 1 book only for them. Or maybe some set number like max 5?
  • If it is illegal, we could get specific authors to sign up? Does this then turn into basically a publishing house?

Just like there are rules for writing non fiction, so there are rules for writing fiction. One thing I notice when I look at stories, even good ones, is how few of them do justice to the other. Pan’s labyrinth has an evil Facist officer and a heroic child and freedom fighters. It never stops to make the officer human. It never stops to ask if maybe the Facists were better than the communists for Spain.

Almost any story or fiction, aside from rationalist fiction, shows people like this. Cardboard cutouts where the author is certain of their own view on who is right and wrong, good an evil. It’s almost only in rationalist fiction that I’ve consistently found writing where the opponents of the protagonists have goals that are coherent and self-contained. Where enemies have integrity and think for themselves.

Hmmmm. On the other hand, not every story has to depict a conflict between people who earnestly believe in competing ideals. It’s fine to have stories about low-minded people for whom ideas either mean nothing (opportunists) or who don’t really interrogate their own beliefs. Most people in the world are like that and stories can and should reflect them. It’s fine to have stories showing the evil in an ideology or system. Still, the fact that these things can exist doesn’t mean they should be the norm.

In Srdjan-Land, most writers would adhere to or strive to write according to roughly https://yudkowsky.tumblr.com/writing and https://yudkowsky.tumblr.com/writing/level3intelligent. When writing about an ideology opposed to their own beliefs, especially an ideology that was socially taboo or they strongly personally disliked, writers would try as hard as possible to make sure that their presentation of that ideology passed the ideological turing test.

I’m going to start reading a fair bit about anthropics because I suspect that understanding anthropic reasoning is key to various interesting real world problems including:

• the fermi paradox (should you buy the grabby aliens/humans are early argument?)
• X risk base rates (does CERN have a 99% chance of destroying the universe every time it’s run? What about a light switch being flipped?)

I’m writing this a bit differently. It’s less of a “here’s what I think about this problem and which solution I think is best/what I have to add” and more of a “here’s stuff I’m really confused about going into this”.

Okay. First thing first. What is anthropic reasoning? In short, it’s reasoning about situations where there are observation selection effects. Observer selection effects meaning that you as an observer will only exist to observe some subset of possible outcomes. A classic example of this is the fine tuning argument.

• theist: god exists
• atheist: what’s your evidence for that?
• theist: we live in a remarkably fine-tuned universe. If gravity was slightly stronger, the universe would have collapsed into a second big crunch shortly after the big bang. Stars, planets and intelligent life would never have formed. If gravity was weaker the expansion of the universe would have been faster and the universe would be endless sparse clouds of matter with no stars or planets. We happen to be precisely at the gravitational goldilocks zone where starts, planets and our existence is possible. This is very unlikely and should be taken as evidence of intelligent design.
• atheist: but we would only exist to observe a universe which has suitable conditions for sentient life to emerge. Even if 99.9999% of universes end up with no life, we should expect all observations to occur in the 0.0001% that have conditions that allow for life

Why is anthropic reasoning confuzzling? A few reasons:

• It seems importantly different from our usual reasoning. Usually we look at the evidence the world presents, consider our priors and shift probability weight between the different possible worlds we could be in. Walking into my building and seeing my apartment door is broken open shifts weight towards me being in the “my flat has been robbed” world and to a lesser extent to the “there was a fire in my flat and the fire brigade had to break in” worlds etc… For anthropics we also need to consider the possibility of our own existence given very uncertain counter-factuals. This is weird and requires some additional scaffolding on top of standard baysianism as I use it day to day.
• I’m not sure where it ends. Let’s say I walk up to you and tell you you shouldn’t use light switches. Why not? Because each time you do there’s a 99% chance of a FTL vacuum collapse and the universe immediately ends. You could respond to this that that’s unlikely and you’ve never observed it happening but wait, no observations here are not evidence.

What’s my plan for learning about this

• reading [[Anthropic Bias]] by Bostrom
• reading sleepingbeautyproblem.com
• annoying Andrea until she explains things to me
• writing a few blog posts along the way