On the exhaust....
If you're looking at an early WRX (02-05) with the 2.0L, there are some things you should be aware of. First, there are three cats on these cars. There is one in the up-pipe that goes from the exhaust manifolds to the turbo, and two in the downpipe that comes off the turbo. First thing to do here is ditch that up-pipe cat by either gutting th estock one or replacing the pipe with a catless aftermarket unit. If the car sees any more boost than stock or hasn't been taken care of, the material in the cat will come apart and go though the turbo! And yes, it has happened may times, and it's no joke. Also, don't cheap out here if you are replacing it. Ask Pseudosport about the "metal butthole"! His cheap Ebay flex-sectioned up-pipe decided to implode and come apart.
Second is that the exhaust flange on the back of the turbo that goes to the downpipe is a non-conventional, weirdo Subaru-only flange. Because of this, there are pretty much three types of downpipes: the stock style (which sucks), the bell mouth design (which is good and cheap), and the divorced design (which is marginally better than the bellmouth, and usually expensive). The Subaru turbos have a internal wastegate, and from the factory, there's a plate that blocks it off to force anything that comes out of it back into the turbo. This isn't the best for performance, as you can imagine. The bell mouth and the divorced setups allow for that exhaust to leave, and help to increase boost. I say get a cheap bellmouth. I had (and so does Pseudosport) a Maddad catless 3'' bellmouth downpipe, and it is a nice piece. I'm not sure if he's in business anymore, but his stuff was nice. One thing to watch out for here is that some aftermarket downpipes may interfere with a drainage line for the a/c system. It can be either trimmed or moved if this is an issue. If you're replacing everything, make sure you get something that's 3'' with no taper at the end that meets with the catback.
Ok, onto catbacks...
On these cars, catbacks really don't add too much power (5-10hp), but I would still replace it anyway. There's nothing like the sound of one of these cars with a nice exhaust. Pick your poison here, because there are hundreds of different catbacks on the market. Among the most common are the big angled fart cannon ones, stock appearing twin-tips, and stock appearing single tips. Most of the cheap ones on Ebay (Megan Racing, Tsudo, etc...) are decent but LOUD. On my car, I had an Autospeed twin-tip, but I can't reccommend it because they went out of business for shady business practices. If you find one used, they are super rare and they sound good and a bit loud, but don't expect to find a new one. If you want something on the quiet side, I'd check out the Tanabe Medallion Touring, the Borla Hush, and if he's still in business, the Maddad Whisper catbacks.
If you can, try to go used here. I got a killer deal on my old catback that was barely used, and I got it for hundreds less than it went for new.