Comment by DoomHotel

Peter Watts covered the possibility of "zombie" intelligences pretty well in his novel Blindsight. One thing I took from it is that such an intelligence could not possibly behave exactly like a human being, because a lot of what we do as humans involves interaction with other humans where we have a mental model of their consciousness being much like our own. Expressing sympathy toward another person, for example, would be inconceivable without consciousness.