Thank goodness someone else is writing about FDT on substack! Just subscribed, looking forward to more of your work :)
If Newcomb's problem feels like a cheat to you, requiring predictions of what you would do in a hypothetical, Joe Carlsmith recommended (on the 80K hours podcast) thinking about twin prisoners' dilemmas as a more visceral and explicitly *fair* representation of the problem. I wrote about it here: https://jacktlab.substack.com/p/introduction-to-functional-decision
“Xing makes you poor” means “Xing will give you less money relative to the other options you had.” The two-boxer will not admit that getting a million is an option they had; they will say “My choice was really one between getting nothing and getting a thousand. I chose the option that makes me most rich.”
In other words, “this action makes you poor” only makes sense relative to a specification of alternative outcomes. But how we define those alternatives is basically just the same dispute as CDT vs EDT vs FDT.
Ah true, I guess they frame it differently. On the one hand, *being an EDT* makes you rich overall (and FDT is just the extreme of this I think), whereas CDT could be argued to "locally" make you rich (compared to the locally available alternatives)
Specifically, because once you know there million is in Box A, whether you one-box or two-box won’t be evidence for anything except whether you get the extra thousand.
The main issue I have with this is that the 'don’t pay people to do things you can do for free' principle doesn't tell against EDT or CDT. The more general idea the principle seems to motivate is that (ideal) rationality requires a capacity for binding (in the sense of Arntzenius, Elga, and Hawthorne (2004), 'Bayesianism, Infinite Decisions, and Binding'), such that, e.g., EDT agents should be able to bind themselves to one-boxing in the transparent Newcomb problem, instead of needing to shell out to stay ignorant of the content of the boxes, and likewise (mutatis mutandis) for CDT agents in other cases. But FDTers go further than that in advocating (what is in effect) a kind of *pre-emptive* binding which the principle doesn't seem to support or make especially plausible. (Wolfgang Schwarz made this point first: https://www.umsu.de/blog/2022/772.) I should also note that a different principle which reflection on the shoes anecdote and the transparent Newcomb problem might suggest is that the right decision theory should (in some appropriate way) value knowledge, and it turns out that on the most natural precisification of that idea, CDT (more precisely: a particular class of causal decision theories) is the only candidate which meets this desideratum, as Michael Nielsen show in 'Only CDT Values Knowledge': https://academic.oup.com/analysis/article/84/1/67/7283000
The thing is the example Nielsen cites from Skyrim to establish this is of the form Omega offers to show you what is in the opaque box but warns you this will not change the chances his prediction are correct. But remember Omega's predictions are supposed to hold for thinkers much like you but with significant variation (as Nozick puts it you know that lots of people like you have faced the challenge and tried all the sorts of tricks and such you might and not reduced the predictive accuracy). I value somewhat being a contrarian, I would derive some satisfaction from showing up Omega (showing he was wrong), and it seems to me within the normal range of variation that there is someone who values that a lot more. Such that they would derive $1000 of utility from showing Omega wrong. In which case if they saw that the opaque box was empty they would (if they could) one box and thus Omega would have been wrong (they would be out the $1000 in the transparent box but would derive a compensatory amount of utility from showing up Omega's prediction), if they saw the opaque box had a million dollars they would (if they could) two box and thus get $1 001 000 and the satisfaction of making Omega wrong (equivalent to another $1000). This would clearly significantly degrade the accuracy of Omega's predictions if it could happen.
For Omega to be telling the truth that revealing the contents of the opaque box does not affect the accuracy of his predictions, it must be that something interferes with doing as one would in these cases. Either something interferes with one's perceptions such that you do not visually acquire knowledge of the actual contents of the box (preventing the contrarian from beating Omega) or something interferes with motor control (or thought process etc.) preventing one from doing as one would (something forces the contrarian to two box when they would one box and one box when they would two box). In which case by not gaining the extra "information" under the condition that you know it will not impact the outcome if you did gain it, you actually give yourself some small evidence that your senses will not be impaired or your body high jacked by whatever it is that ensures the accuracy of Omega's predictions in such cases. To me this sounds like valuing knowledge and maximizing the chances that Omega is thwarting your ability to use new information is to disvalue knowledge, although it is small comfort because there is no telling how Omega achieves his accuracy in cases where the box remains opaque, maybe he hijacks motor control in those cases also. Still it might be worth paying Omega not to hijack your body and doesn't seem to devalue knowledge.
Note about binding. If your behaviour is predictable it is clearly bound by something. It may be bound by something external (optical illusions, something taking control of your body etc.), something geographically internal but beyond your conscious knowledge or control (unconscious Freudian drives say) or it may be bound by your judgements, desires, knowledge and so on that are under your direct mental observation and control. FDT just seems to take seriously that the binding might be completely in terms of endogenous mental predispositions and that choices are (in at least some cases) about modifying these characteristics (that they can indeed be so modified), not about generating acts in the moment. If the binding of the type FDT posits is impossible than a significant subclass of Newcomb's problem are also impossible.
Schwarz blog post is interesting, but I don't think his use of the example of Newcomb's revenge is apt. Newcomb's problem is interesting because you lay out a clear set of conditions that are somehow known and yet get very different analysis of what to do if you can (people know they are in a Newcomb's problem type scenario, but disagree about what it means). Newcomb's revenge is not parallel to that as you don't know the conditions (you don't know that you are faced with Newcomb's revenge instead of Newcomb's problem). So the fact that you will get it wrong when you don't know what kind of problem you are in is not informative of strategy where somehow you do know what scenario you are in. However it does illustrate nicely that it would be very hard and indeed perhaps impossible to ever be confident you were in a Newcomb's problem type situation. If it's impossible to know (or even guess with a probability greater than chance etc.) you are in a Newcomb's problem type situation then it does not matter what any analysis suggests you do, because you would never have occasion to act on any such analysis.
Here's how I think the scenario would actually play out Edmond would not say what was suggested in the transparent case he would just take the 1 box and then rub his eyes and it would turn out the million dollars was in it the whole time.
Flo would says "But I thought you followed the evidence?"
Edmond respond "I do it was far more likely I was having a visual hallucination of an empty box than that Omega would be wrong, so taking just one box makes for way better evidence of the million dollars being there than my lying eyes can provide of it not being there."
Flo "But I saw the same thing, surely it multiples the improbability of a hallucination in one case by the same miniscule fraction making for an even smaller chance."
Edmond "Yes but Omega being wrong is very unlikely and indeed it is somehow the only thing I have good evidence about apparently, so even two of us suffering a collective hallucination is more credible than Omega being wrong. Also if some aspect of the transparent box was making me misapprehend its contents it would probably also effect your eyes, so we should not treat the chances of the two events as wholly independent."
Flo "Oh well, hmm the box appears empty but I know this game."
Flo trips and accidentally picks up both boxes to stop her fall.
"Drat now I only have $1000 but I was true to my principles I should have gotten the million bucks."
Cassandra "Yes but Omega predicts what you will do, not what you intend to do, Omega knew you would trip and accidentally pick up both boxes. Que sera sera."
Newcomb's problem and related decision theory depends on several assumptions some of which are tricky and ambiguous. A hugely important one is how and in what way Omega's predictions are better than chance (note they only have to be better than chance by like 1 in 1000 to create the force of Newcomb's problem) and how the participants have come to know this. For example if I think he's an accurate predictor just because in every case I know of but I only know like 5 cases or indeed a million I could never say it is a trillion to one he is wrong. Especially if for example he just always predicts two boxing and everyone two boxes. In that case I wouldn't conclude that Omega is good at predicting, I and everyone other participant would just conclude there is never a million dollars and so always take the $1000 and it would happen that Omega's "prediction" would always be right. If there were only 5 other participants that would be the best conclusion from the evidence, if there were a million it would be surprising no one picked one box and ruined Omega's perfect record, but still probably a better explanation than attributing predictive power to Omega.
So it is complicated to get evidence of Omega's success as a predictor and cases where he is ludicrously good (trillion-to-one he is wrong) I think tend to defy our understanding.
As described in my variations on the double transparent box thought experiment Omega can make his predictions in two separate ways. He can predict (will assume using normal computation) what you will do given that you will do what you intend to do given your principles and best judgement, in which case you should properly formulate your intentions (call this Endogenous Omega). Or Omega can predict (again via normal computation) what you do even if it is opposite to or otherwise independent of what you would intend to do given your principles and best judgement (Exogenous Omega).
If you are dealing with Exogenous Omega then Cassandra's causal diagnosis is correct, who gets the million dollars is luck, although in that case it might be that few of the participants do what they would normally intend and Cassandra might one box (due to tripping etc.) and be a millionaire and yet Omega would generally be correct in their predictions and Cassandra would be a "two boxer" in general.
If you are dealing with Endogenous Omega than Flo and Functional diagnosis is perhaps the most appropriate.
If you were not sure which and looking for evidence of whether Omega is making exogenous or endogenous predictions I'd say Edmond and Evidential decision theory is the best outlook.
As it is it is Omega would probably do a mix of endogenous and exogenous prediction.
However note that if Omega's predictions are purely endogenous (based on how the reasoner reasons, the evidence and principles available to them) then it is unlikely he can correctly predict in the transparent case. I'd say you can show that successfully placing money as prediction in the transparent case is equivalent to solving the halting problem (easy example if this program halts I take one box, if it does not halt I will take both boxes, no matter what, if Omega can predict that based on my endogenous mental states via normal prediction he can predict whether an algorithm will halt by analysis which is the halting problem but that is impossible). Omega can only predict what a diverse sample of humanity can do in the transparent case by relying on some kind of exogenous factors to prevent deliberate thwarting by contrarians (people who derive great utility from proving other people wrong) and others who would benefit from thwarting his predictions. Otherwise such malcontents would drive down his accuracy of prediction. Unless Omega does not use ordinary computation at all for prediction but instead some other mechanism (time travel, smoke and mirrors or something else).
Oh one other thing, I'd understand Functional Decision Theory to posit that: 1) all human decision making can be modeled as the computation of an appropriate Turing machine (computer). 2) that answering questions about hypotheticals should be modeled as stating which instruction sets you would program to achieve the intention to carry out an action within one's power in them. 3) In Newcomb's problems type cases what is meant by correctly predicting an action is correctly predicting an intended action (all intended actions within one's power will be carried out).
I think if you stipulate what would you do given 1-3 are true then EDT, CDT and FDT collapse into each other in terms of what they advise. What you decide is not what you will do at a given time t, but rather the principles you will follow at all times t, and finding out these principles is what cause Omega's prediction, therefore the decision (setting ones principles, programming oneself) is causal of the appearance of the money, so deciding to be a one boxer (in the range of Newcomb's problems that meet requirements 1-3) causes the million dollars to be present and so is causal and that is how you should program yourself according to CDT. Likewise the Evidential decision theorist will see that the best evidence of the million dollars being there in such cases would be that you programmed oneself to one-box in such cases so it advises you to do that.
I think one of the easiest to analyze and yet true to the spirt of Nozick's original presentation of Newcomb's problem is probably one that includes assumptions 1-3, but I don't know that it exhausts all possibly Newcomb Problem-type scenarios. I think there are plausible cases where Omega could be a completely exogenous predictor (indeed Nozick seems to broach one like that in his original presentation) and one would have to say either FDT does not apply or is simply wrong in those instances.
I'd say that assumptions 1-3 also suggest an upper bound on Omega's practical success in predicting far lower than 1 in a trillion, as in if someone built a robot to be either a 1 or 2 boxer then I think it would fail to do what it was programmed to do way more often than 1 in a trillion times and so even though you could predict what it should have done with very very high accuracy, it could not be with trillion-to-one accuracy in terms of what it actually does. If a robot would not be trillion-to-one reliable and so predictable then how could a prediction that relies on the warped wood of humanity to properly execute its principles be better?
Thank goodness someone else is writing about FDT on substack! Just subscribed, looking forward to more of your work :)
If Newcomb's problem feels like a cheat to you, requiring predictions of what you would do in a hypothetical, Joe Carlsmith recommended (on the 80K hours podcast) thinking about twin prisoners' dilemmas as a more visceral and explicitly *fair* representation of the problem. I wrote about it here: https://jacktlab.substack.com/p/introduction-to-functional-decision
'The “CDT makes you poor” argument is, ultimately, question-begging.'
Could you clarify what you mean by this?
“Xing makes you poor” means “Xing will give you less money relative to the other options you had.” The two-boxer will not admit that getting a million is an option they had; they will say “My choice was really one between getting nothing and getting a thousand. I chose the option that makes me most rich.”
In other words, “this action makes you poor” only makes sense relative to a specification of alternative outcomes. But how we define those alternatives is basically just the same dispute as CDT vs EDT vs FDT.
Ah true, I guess they frame it differently. On the one hand, *being an EDT* makes you rich overall (and FDT is just the extreme of this I think), whereas CDT could be argued to "locally" make you rich (compared to the locally available alternatives)
What would Edmund do in the Transparent situation if he saw the million dollars in Box A?
He would take both!
Specifically, because once you know there million is in Box A, whether you one-box or two-box won’t be evidence for anything except whether you get the extra thousand.
Edmund should've kept his eyes closed! Opening them seems bad according to EDT, right?
The main issue I have with this is that the 'don’t pay people to do things you can do for free' principle doesn't tell against EDT or CDT. The more general idea the principle seems to motivate is that (ideal) rationality requires a capacity for binding (in the sense of Arntzenius, Elga, and Hawthorne (2004), 'Bayesianism, Infinite Decisions, and Binding'), such that, e.g., EDT agents should be able to bind themselves to one-boxing in the transparent Newcomb problem, instead of needing to shell out to stay ignorant of the content of the boxes, and likewise (mutatis mutandis) for CDT agents in other cases. But FDTers go further than that in advocating (what is in effect) a kind of *pre-emptive* binding which the principle doesn't seem to support or make especially plausible. (Wolfgang Schwarz made this point first: https://www.umsu.de/blog/2022/772.) I should also note that a different principle which reflection on the shoes anecdote and the transparent Newcomb problem might suggest is that the right decision theory should (in some appropriate way) value knowledge, and it turns out that on the most natural precisification of that idea, CDT (more precisely: a particular class of causal decision theories) is the only candidate which meets this desideratum, as Michael Nielsen show in 'Only CDT Values Knowledge': https://academic.oup.com/analysis/article/84/1/67/7283000
The thing is the example Nielsen cites from Skyrim to establish this is of the form Omega offers to show you what is in the opaque box but warns you this will not change the chances his prediction are correct. But remember Omega's predictions are supposed to hold for thinkers much like you but with significant variation (as Nozick puts it you know that lots of people like you have faced the challenge and tried all the sorts of tricks and such you might and not reduced the predictive accuracy). I value somewhat being a contrarian, I would derive some satisfaction from showing up Omega (showing he was wrong), and it seems to me within the normal range of variation that there is someone who values that a lot more. Such that they would derive $1000 of utility from showing Omega wrong. In which case if they saw that the opaque box was empty they would (if they could) one box and thus Omega would have been wrong (they would be out the $1000 in the transparent box but would derive a compensatory amount of utility from showing up Omega's prediction), if they saw the opaque box had a million dollars they would (if they could) two box and thus get $1 001 000 and the satisfaction of making Omega wrong (equivalent to another $1000). This would clearly significantly degrade the accuracy of Omega's predictions if it could happen.
For Omega to be telling the truth that revealing the contents of the opaque box does not affect the accuracy of his predictions, it must be that something interferes with doing as one would in these cases. Either something interferes with one's perceptions such that you do not visually acquire knowledge of the actual contents of the box (preventing the contrarian from beating Omega) or something interferes with motor control (or thought process etc.) preventing one from doing as one would (something forces the contrarian to two box when they would one box and one box when they would two box). In which case by not gaining the extra "information" under the condition that you know it will not impact the outcome if you did gain it, you actually give yourself some small evidence that your senses will not be impaired or your body high jacked by whatever it is that ensures the accuracy of Omega's predictions in such cases. To me this sounds like valuing knowledge and maximizing the chances that Omega is thwarting your ability to use new information is to disvalue knowledge, although it is small comfort because there is no telling how Omega achieves his accuracy in cases where the box remains opaque, maybe he hijacks motor control in those cases also. Still it might be worth paying Omega not to hijack your body and doesn't seem to devalue knowledge.
Note about binding. If your behaviour is predictable it is clearly bound by something. It may be bound by something external (optical illusions, something taking control of your body etc.), something geographically internal but beyond your conscious knowledge or control (unconscious Freudian drives say) or it may be bound by your judgements, desires, knowledge and so on that are under your direct mental observation and control. FDT just seems to take seriously that the binding might be completely in terms of endogenous mental predispositions and that choices are (in at least some cases) about modifying these characteristics (that they can indeed be so modified), not about generating acts in the moment. If the binding of the type FDT posits is impossible than a significant subclass of Newcomb's problem are also impossible.
Schwarz blog post is interesting, but I don't think his use of the example of Newcomb's revenge is apt. Newcomb's problem is interesting because you lay out a clear set of conditions that are somehow known and yet get very different analysis of what to do if you can (people know they are in a Newcomb's problem type scenario, but disagree about what it means). Newcomb's revenge is not parallel to that as you don't know the conditions (you don't know that you are faced with Newcomb's revenge instead of Newcomb's problem). So the fact that you will get it wrong when you don't know what kind of problem you are in is not informative of strategy where somehow you do know what scenario you are in. However it does illustrate nicely that it would be very hard and indeed perhaps impossible to ever be confident you were in a Newcomb's problem type situation. If it's impossible to know (or even guess with a probability greater than chance etc.) you are in a Newcomb's problem type situation then it does not matter what any analysis suggests you do, because you would never have occasion to act on any such analysis.
Here's how I think the scenario would actually play out Edmond would not say what was suggested in the transparent case he would just take the 1 box and then rub his eyes and it would turn out the million dollars was in it the whole time.
Flo would says "But I thought you followed the evidence?"
Edmond respond "I do it was far more likely I was having a visual hallucination of an empty box than that Omega would be wrong, so taking just one box makes for way better evidence of the million dollars being there than my lying eyes can provide of it not being there."
Flo "But I saw the same thing, surely it multiples the improbability of a hallucination in one case by the same miniscule fraction making for an even smaller chance."
Edmond "Yes but Omega being wrong is very unlikely and indeed it is somehow the only thing I have good evidence about apparently, so even two of us suffering a collective hallucination is more credible than Omega being wrong. Also if some aspect of the transparent box was making me misapprehend its contents it would probably also effect your eyes, so we should not treat the chances of the two events as wholly independent."
Flo "Oh well, hmm the box appears empty but I know this game."
Flo trips and accidentally picks up both boxes to stop her fall.
"Drat now I only have $1000 but I was true to my principles I should have gotten the million bucks."
Cassandra "Yes but Omega predicts what you will do, not what you intend to do, Omega knew you would trip and accidentally pick up both boxes. Que sera sera."
Newcomb's problem and related decision theory depends on several assumptions some of which are tricky and ambiguous. A hugely important one is how and in what way Omega's predictions are better than chance (note they only have to be better than chance by like 1 in 1000 to create the force of Newcomb's problem) and how the participants have come to know this. For example if I think he's an accurate predictor just because in every case I know of but I only know like 5 cases or indeed a million I could never say it is a trillion to one he is wrong. Especially if for example he just always predicts two boxing and everyone two boxes. In that case I wouldn't conclude that Omega is good at predicting, I and everyone other participant would just conclude there is never a million dollars and so always take the $1000 and it would happen that Omega's "prediction" would always be right. If there were only 5 other participants that would be the best conclusion from the evidence, if there were a million it would be surprising no one picked one box and ruined Omega's perfect record, but still probably a better explanation than attributing predictive power to Omega.
So it is complicated to get evidence of Omega's success as a predictor and cases where he is ludicrously good (trillion-to-one he is wrong) I think tend to defy our understanding.
As described in my variations on the double transparent box thought experiment Omega can make his predictions in two separate ways. He can predict (will assume using normal computation) what you will do given that you will do what you intend to do given your principles and best judgement, in which case you should properly formulate your intentions (call this Endogenous Omega). Or Omega can predict (again via normal computation) what you do even if it is opposite to or otherwise independent of what you would intend to do given your principles and best judgement (Exogenous Omega).
If you are dealing with Exogenous Omega then Cassandra's causal diagnosis is correct, who gets the million dollars is luck, although in that case it might be that few of the participants do what they would normally intend and Cassandra might one box (due to tripping etc.) and be a millionaire and yet Omega would generally be correct in their predictions and Cassandra would be a "two boxer" in general.
If you are dealing with Endogenous Omega than Flo and Functional diagnosis is perhaps the most appropriate.
If you were not sure which and looking for evidence of whether Omega is making exogenous or endogenous predictions I'd say Edmond and Evidential decision theory is the best outlook.
As it is it is Omega would probably do a mix of endogenous and exogenous prediction.
However note that if Omega's predictions are purely endogenous (based on how the reasoner reasons, the evidence and principles available to them) then it is unlikely he can correctly predict in the transparent case. I'd say you can show that successfully placing money as prediction in the transparent case is equivalent to solving the halting problem (easy example if this program halts I take one box, if it does not halt I will take both boxes, no matter what, if Omega can predict that based on my endogenous mental states via normal prediction he can predict whether an algorithm will halt by analysis which is the halting problem but that is impossible). Omega can only predict what a diverse sample of humanity can do in the transparent case by relying on some kind of exogenous factors to prevent deliberate thwarting by contrarians (people who derive great utility from proving other people wrong) and others who would benefit from thwarting his predictions. Otherwise such malcontents would drive down his accuracy of prediction. Unless Omega does not use ordinary computation at all for prediction but instead some other mechanism (time travel, smoke and mirrors or something else).
Oh one other thing, I'd understand Functional Decision Theory to posit that: 1) all human decision making can be modeled as the computation of an appropriate Turing machine (computer). 2) that answering questions about hypotheticals should be modeled as stating which instruction sets you would program to achieve the intention to carry out an action within one's power in them. 3) In Newcomb's problems type cases what is meant by correctly predicting an action is correctly predicting an intended action (all intended actions within one's power will be carried out).
I think if you stipulate what would you do given 1-3 are true then EDT, CDT and FDT collapse into each other in terms of what they advise. What you decide is not what you will do at a given time t, but rather the principles you will follow at all times t, and finding out these principles is what cause Omega's prediction, therefore the decision (setting ones principles, programming oneself) is causal of the appearance of the money, so deciding to be a one boxer (in the range of Newcomb's problems that meet requirements 1-3) causes the million dollars to be present and so is causal and that is how you should program yourself according to CDT. Likewise the Evidential decision theorist will see that the best evidence of the million dollars being there in such cases would be that you programmed oneself to one-box in such cases so it advises you to do that.
I think one of the easiest to analyze and yet true to the spirt of Nozick's original presentation of Newcomb's problem is probably one that includes assumptions 1-3, but I don't know that it exhausts all possibly Newcomb Problem-type scenarios. I think there are plausible cases where Omega could be a completely exogenous predictor (indeed Nozick seems to broach one like that in his original presentation) and one would have to say either FDT does not apply or is simply wrong in those instances.
I'd say that assumptions 1-3 also suggest an upper bound on Omega's practical success in predicting far lower than 1 in a trillion, as in if someone built a robot to be either a 1 or 2 boxer then I think it would fail to do what it was programmed to do way more often than 1 in a trillion times and so even though you could predict what it should have done with very very high accuracy, it could not be with trillion-to-one accuracy in terms of what it actually does. If a robot would not be trillion-to-one reliable and so predictable then how could a prediction that relies on the warped wood of humanity to properly execute its principles be better?