Here's a thought.

Perhaps we are not physical bodies that occasionally go to a spiritual place.

Perhaps we are spiritual beings that occasionally inhabit physical bodies.