Some people reported that their on-board AC97 had noise, and I found a similar situation on my new PC. When you move your mouse, or when the hard drive begins to spin, you can hear the subtle noise, especially at night when the environment is relatively quiet. Therefore, I proceed to buy an external USB sound card in the hope to fix the problem.
However, the problem persists.
Then I know why, my speakers are drawing power from the USB port of the motherboard, and those were the real source of the noise. I tried to power the speaker from some other sources and the noise was gone!