Gizmodo's Prof. Dealzmodo takes a similar approach to buy PCs as I do. Buy a low-end machine and then buy third-party components if/when required. You can read the Gizmodo article to get an idea, but I'll also throw in some of my own thoughts.
First, you must get at least 2 GB of RAM. If you can still get Windows XP, then you can stop at 2 GBs. If you are stuck with Vista, add at least 1 more GB -- if you add 2, giving you 4 total, 0.5 GB will go to waste unless you go with a 64-bit Vista. This introduces potential compatibility problems with older hardware and games, so likely you will want to stick with the non-64-bit versions.
For monitors, it's likely you already have 1 or more. If it's time to get a new one, look for 3rd party deals. Sometimes running 2 side-by-side can be more effective than one large display -- putting them at an angel can be easier to view than turning your head from side to side (avoid the 30" displays).
In terms of a video card to drive that monitor, if you are getting a laptop, you must buy what you need at time of purchase. For desktops, you can pick it up through a 3rd party. Video cards have come a long way due to gaming. Unless you're a hardcore gamer (which would make this blog mute), getting a 12 to 18 month old card should be sufficient and much cheaper. If you do some gaming, such as 1st-person shooters, I would look for 512 MB minimum -- letting your budget determine whether to get more RAM. If you do little to no gaming, 256 MB would probably be just fine. The other item to look for is the number of outputs. If you are going to run 2 monitors, make sure you have 2 outputs. Likewise, digital is the predominant output now -- make sure your video card and monitors take the same output/input. (You can get digital to analog converters to support older monitors.)
Input devices, mouse and keyboard -- you should already have ones that meet your needs. Buy these independent of your PC, if and when you need them. Don't pay extra while you order your new PC.
For audio out, unless your an audiophile, what you get standard should be good enough. Like the input devices, buy what meets your needs independent of the PC, them move them to the new PC as required.
For networking, all your PCs should have wired 10/100 ethernet connections. If you're running wireless and buying a laptop, you should be covered too. For desktop users who want to go wireless, buy an 802.11N ethernet card from the same manufacturer that makes your router -- it will likely always be cheaper to buy it third-party and add yourself. This is the safest way to make sure you will be compatible. If you want the fastest throughput, though easier, do not get the USB version, buy the add-on card version.
So far we've covered video, audio, and networking that may all require additional slots in your computer. Be sure to validate that the new desktop has slots to accept this add-ons.
Remaining are your media devices, harddrive, CD/DVD, and perhaps blue-ray. For all of these, they are going to come with the machine regardless, so do some comparison shopping to see if you should get bigger/better from the PC manufacturer or 3rd party, as typically if you replace what comes with the PC, you're going to have unused parts left-over -- this can actually lead to higher costs.
To dig deeper into the harddrive, if you buy small and add a second in the PC box, now you have to manage 2 drives. If you're doing a lot of picture or video editing, this is probably desirable, for the rest of us, probably not. With that in mind, I would likely just get the largest drive I can from the PC manufacturer. I would still get a 2nd drive, but I would make this either a USB or network drive and use this for backup -- mirroring my data from my primary drive. This can be bought through a third-party and configured to run backups while I'm asleep.
One last though on harddrives. If you are running a home network, you can always add some storage capacity to your old PC and configure it to be centralize media storage and/or backups for the entire family. This is a completely different topic that can be discussed at some future time.
There you have it... my recommendation for buying a new PC. By the way, my current home PC is a 3 year old Compaq that I have added better video, preferred keyboard and mouse, additional RAM, a wireless network card, and more harddrive storage. My next upgrade will come after Windows 7 is released, and I will at least keep the video card, keyboard and mouse, likely the wireless network card, and my external backup drive. Oh, and my speakers -- they are about 6 years old.