I've seen this happen many times. Windows doesn't like using two seperate output devices at the same time. As one is plugged via the 3.5mm Analog connections (most all desktop speakers) and the other is USB, windows picks which one is active, and odds are, the speakers are the default output device.
To change which is active, go to the sounds tab of the control panel. Under output devices, there will probably be a device called USB audio (or USB headset, or something like that). Set that as the output device instead of the speakers. You can also set the headset to the default output device, so the headphones override the speakers automatically when you plug them in.