2000 Honda Accord: Weird problem with my speakers
 

My left front door speaker was not working so I took the door apart and removed the speaker. I tried putting it on the right side but it doesn't work. I took the right speaker and tried putting it left, but that didn't work either. I am not understanding what this could mean. Is it like the stereo is damaged or both the stereo and speaker are damaged?

Any suggestions?