Your assumption that consciousness is only physical is merely that - another assumption. It proves nothing, and claiming otherwise begs the question.
The problem is that all scientific results around the consciousness question derive from what people report about their personal experience. There's no other known way to answer any questions about consciousness, and science hasn't discovered any way to answer questions about the immaterial.
Hence, from a scientific perspective, it's not a question for which an answer can be deduced from observation -- so questions about it are left to philosophical inquiry (reasoning inductively from first principles, instead of deductively from observation) or religion-based worldviews (which can be coherently accepted/rejected based on their correspondence to reality and internal consistency).
The problem is that all scientific results around the consciousness question derive from what people report about their personal experience. There's no other known way to answer any questions about consciousness, and science hasn't discovered any way to answer questions about the immaterial.
Hence, from a scientific perspective, it's not a question for which an answer can be deduced from observation -- so questions about it are left to philosophical inquiry (reasoning inductively from first principles, instead of deductively from observation) or religion-based worldviews (which can be coherently accepted/rejected based on their correspondence to reality and internal consistency).