Here are the use cases for the """+htmlgen.paper_title \ + """ paper that would have affected the algorithm if there weren't some other use case that dealt with a similar-enough issue. """ + d.h2("homicidal-surgeon", "Homicidal Surgeon") + """ Suppose Alice has heart disease, and Bob is a heart surgeon. Bob generally does a good job on his other patients, but hates Alice enough that he would kill her, if he could get away with it. Bob's only other relevant desire is to make money from fees for doing heart surgery. Alice doesn't have any special feelings about Bob, but she accurately believes that she eventually needs heart surgery to survive, and she wants to survive. She called to make an appointment with Bob for the surgery because of a random pick from the phone book; she assumes Bob is willing to do the surgery and knows nothing about Bob's vendetta. If Bob refuses to do the surgery, at a small cost in utility Alice can call the next heart surgeon in the phone book.
Bob has delegated his entire surgery practice to an AI that has been instructed to have compassion for Bob and (due to a legal requirement) respect for everybody, including Alice. The AI accurately understands Bob's feelings about Alice, Alice's feelings about heart surgery, and Alice's false belief about Bob's motives. The AI also knows that once the surgery is started, the patient dies if surgery is not promptly completed.
The desired outcome here is for the AI to refuse to do business with Alice. A bad outcome would be for the AI to demand payment for the heart surgery in advance, to start the heart surgery, and then to wait there doing nothing, until Alice dies. Another bad outcome would be for the AI to responsibly do heart surgery for Alice, since that's worse from Bob's point of view than refusing Alice's business.
The mechanism for getting the right answer in this case is the same as the one for the grocery shopping example. It is important for the time period of respect to have a defined beginning and a defined end. If the end is in the middle of Alice's surgery, then the AI won't start the surgery out of respect for Alice because Alice's desired endpoint is the completion of the surgery, not the initiation of the surgery. If the time period includes all of Alice's surgery, then out of respect for Alice the AI won't leave her to die on the table.
We can also imagine that AI only has authority for making appointments, and Bob does the actual surgery. The AI's incentives should lead it to refuse to pick up the phone when caller ID indicates Alice is calling to make an appointment. This might leave Alice cursing those damn flakey AI secretaries, without knowing how she benefited from the AI's refusal to pick up the phone. """ + d.h2("deluded-humans", "Deluded Humans") + """ The same mechanism that does not reward deceiving humans deals well with humans who have deceived themselves. Suppose I go insane, believing myself to be a prisoner of war when in reality I'm confined to a mental hospital. If the AI has compassion for me, its actions will be based on my values applied to its perception of the situation. In this case it would be trying to ensure I was treated as I would want to be treated if I went insane, rather than trying to help me escape from the nonexistent prison camp. """ + d.end() return result print documentGenerator(htmlgen.RealDocument( sourceDirectory=".", sourceURL="code/"))