Could be a cable has come lose. The USB port is connected by a cable to a Bluetooth module. All of this sits behind the glovebox. If you remove the glove box (4 screws at the front and a couple of bolts inside in the roof of the glovebox) you will be able to see the cable running from the USB port.
The Bluetooth module is bolted to the dash but to see it it's easier to lie in the footwell and look back toward the back of the car. Or since you already have the USB, follow the cable
Does handsfree calling work in your car? If it's not a lose cable, could be a duff Bluetooth module. Was your car factory fitted with Bluetooth or was it retro-fitted?
I retrofitted USB to my car using the genuine Ford parts and the USB stick slots into the USB port quite snug, it certainly doesn't fell like it's going to fall out.