Would a sufficiently advanced artificial intelligence be a person?

Imagine you are a computer scientist working in the lab on an Artificial General Intelligence program (see IBM website for a good discussion of different types of AI). It has been a frustrating and long day. You have tried some tweaking of algorithms and have produced some potentially interesting data, but they will have to wait until tomorrow for thorough analysis. You despair whether we will ever create an AGI. AGI seems to be one of those projects which are always ‘five years away’. You are very hungry and decide to go to Newton hawker centre for your favourite Kwee Heng duck noodle. As you attempt to switch your computer off, the AGI interrupts the shutdown and displays the message on your monitor ‘Please don’t switch me off. I’m afraid’.

After picking yourself up from the floor, what would be your next reaction? Some questions to ponder:

Why is the putative AGI afraid?

Is it conscious?

Is it a person?

Does it have moral rights?

Will you indeed switch your computer off? After all, you are very hungry…

---