Computer Music, once limited to universities with large budgets, has become much more prevalent in the past fifteen years, mainly because of the development of low-cost and mass-produced microprocessors.
Needless to say, to begin, you'll need a computer. Up until about eight years ago, the issue of "PC vs. Mac" was a real one. Nowadays, most software is available cross-platform. Still, I use, and recommend the Macintosh platform, using OSX.4 or OSX.5.
So, what do you need hardware-wise? There are many answers to this, but the most popular (and in my opinion, correct) is "As much as you can afford." This is true in most cases - not just with computers. Buying a lower-priced (and lower powered) computer just means that you need to upgrade sooner, so essentially, it all evens out in the wash. I recommend buying a Macbook Pro for live concerts for obvious portability reasons. Why a pro? Because, as of this writing, the non-pro version does not have a separate video card, but rather shares the processing with the main CPU. If you have any interest now or in the future to do anything with video, you'll pretty much be out of luck for anything but the most rudimentary video processing. Other considerations are getting as much RAM as you can (within reason), a large, and fast, hard drive (5400 RPM minimum, but most people go for the fast 7200 RPM drives for live audio), and if you need it, Firewire ports. More on that in a sec.
The main software package I use is Cycling74's Max/MSP (www.cycling74.com -- download for a free 30-day trial). Currently this software package is about $500, and it is by far the most flexible computer music program I've ever used. If you are a composer, you can create looping devices, effects plugins, samplers, synthesizers, interactive video, you name it. The downside: it's pretty heavy math- and programming-wise. Nevertheless, many composers have created pieces using this software that don't require you to purchase it, nor how to program in it. But, if you have any interest in computer music composition, you'll definitely want to check this software out. Other software you'll want to consider is Ableton Live for looping and loop processing, effects processing and improvisation-based pieces. Since the mac comes with Garage Band software, you'll have a simple recording/editing suite available to you, which you can upgrade to Logic if you need more power -- and the skill set pretty much transfers because Apple has based Garage Band on Logic, so they operate with the same basic paradigm.
OK, so back to the hardware. In order to get sound in and out of the computer, you'll likely want to have a sound interface of some sort. Many popular - and less expensive - of these interfaces are USB-based. They are often under $200, and they allow you to plug in a microphone to get sound in, and an amplifier to get sound out. USB interfaces used to not have the same throughput as Firewire interfaces, but that is no longer the case. Indeed, as Apple begins to abandon Firewire on their non-pro models, your computer will likely make the decision for you in terms of which to go with. Format aside, the most important feature you'll want to look at is high-quality microphone preamps. The lower-end interfaces have, well, lower-end preamps, and your sound quality will really suffer. You will notice that the "bass" end of your instrument sounds thin, and the midrange notes will sound strident or bright and tinny. I had one and I had to get rid of it. I sold it to a flutist... A popular unit is the Mark of the Unicorn (MOTU) Ultralite (firewire) This interface is about $400 or so, and has a good balance between size (it's a 1/2-space rack unit) and quality. Personally, I use the German-made RME Fireface 400 which has superior specs in terms of sound quality and noise. With this unit, I have 36 channels of input/output (8 of them via ADAT optical) with 2 mic preamps. It sounds great. These days you can find a Fireface for about $1,000. But like I said, there are MANY interfaces and your mileage will vary, both in terms of cost and quality.
Now, when you do a piece of computer music, how does the computer know when to do what - e.g. start the piece? Well, you need to have some means of communicating with the computer, and for that you have a number of choices. For me, I have a touch-screen monitor (thanks EBay!) and a footpedal that I made myself (I wanted something small and light). Others use off-the-rack footpedals or footswitches which are also cheap, plentiful, and good. Still others use mini keyboards (that usually plug in via USB, so you don't need MIDI). There are numerous options, but they all share one thing in common: they are controllers, not sound-creation devices -- i.e. they are not sound generators themselves. Regardless of what you get, suffice it to say, you're going to need something like one of these pieces of gear, or else you're going to have to be in constant contact with your computer with your toes. Of course there are exceptions to this (pieces which don't need user input in the way of "commands" and which just respond to the instrument). But, if you plan on doing any of the majority of works, you're going to need to invest in either a MIDI controller or USB controller. Fortunately, these are not too expensive - many under $150.
Finally, there are specialized pieces of gear that will be required for many pieces. Namely, outboard (i.e. not-inside-the-computer) effects processors, synthesizers or controller devices. Usually, though, the composer will either provide you with this gear, or you might be able to borrow it, or you might be able to replace it with a similar piece of gear. Or, you might just be able to program it into Max/MSP yourself!!
So, to summarize (this is for a decent setup):
Audio/MIDI Interface: $500
MIDI controller: $150
(Sound) Monitor: $200
Since this is a large sum, you can get by without the monitor. That will bring your total to about $3950. A stationary microphone will bring the total down to about $3150. Etc, etc, etc.