Valid argument, but we have a clear example that it is wrong.
Humans.
A lot of what we do is driven by internal value calculations and pleasure centers, so why aren't we all simply taking drugs and avoiding all this messy "doing things" business?
Point is, if humans figured out a way of avoiding purely pressing the right buttons to enjoy themselves and actually being useful, so too will smart robots.
We are driven by multiple different value system. e.g. I am extremely curious to know if god's existence can or can not be proved scientifically. I can go to great length to know it, I do not give a damn about money. On the other hand, there are people who do not give damn about god and can go to great length for money. So Humanity can not go towards single goal. Some of them could go for shortcut and are going.
Drugs, in their current incarnations, are simply not that powerful a motivator. If you were to point at a heroin junky or basehead and say 'that person is in ecstasy right now, do you want their life?', the majority of people would say no, because all they see is a dirty, homeless junky.
I'm not sure humans are a counterexample. Drugs don't generally relieve the need to work to buy food, and most serious drug addicts don't look as though they enjoying themselves.
I've been thinking about this recently: drug addicts "seem" unhappy because they don't match the usual outward appearance of happiness. But who's to say in their lifetime they haven't experienced many orders of magnitude more happiness than even the happiest non-addicts? Drug addiction breaks the usual profile of happiness, it doesn't mean that they're not actually the happiest people who ever lived.
It might not have to do with "figuring it out", in the case of humans it's natural selection: individuals who choose to stimulate their reward centers only had much lower probability of reproducing. (like mice who stimulate their nucleus accumbens till they starve to death)
Most of us haven't actually experienced these powerful drugs. I wonder how many people could willingly pass it up if everyone had a significant experience with them.
Was going to say something like this. There's a significant difference between:
(researcher) points at a junkie on the street: This person is in ecstasy. How would you like to feel like that forever?
(researcher) gets you comfortable and injects you professionally and painlessly with a controlled dose of heroin, leaves and returns in a few hours: How would you like to feel like that forever?
Humans.
A lot of what we do is driven by internal value calculations and pleasure centers, so why aren't we all simply taking drugs and avoiding all this messy "doing things" business?
Point is, if humans figured out a way of avoiding purely pressing the right buttons to enjoy themselves and actually being useful, so too will smart robots.