I see this debate a lot on the internet. I think this documentation is some of the best out there:

http://web.tampabay.rr.com/redroby/3liter/exhaust.html

Backpressure reduction will result in high end HP gains and loss in low end torque. If the engine breathes too freely down low, there is less resistance and loss of torque.

Case and point: a friend of mine has a Subaru 2.5RS with turbo, sometimes for drag racing we would unbolt the exhaust just past the test pipe (so effectively exhaust would come out right from the downpipe into the atmosphere). When we did this, the car would bog a lot more out of the hole and launches weren't nearly as hard (keep in mind it is AWD). However, near redline there was a good increase in acceleration. If we left the entire exhaust system in place and launched the car it would take off much harder but would not make the high end pull that it did with exhaust disconnected. If we were running an 1/8 mile the times were always better if the exhaust were connected, but if it was a 1/4 mile then times were better disconnected (about 0.2 sec consistently).

Case and point #2: I had a GTI 1.8t before my S2k. I put an aftermarket DP/Exhaust on the car that was about 2.75in. It would pull pretty hard to redline. One day the DP cracked and I put the stock one back on until I could get a new one. Once I did that (I hadn't driven the car like this in months) I was shocked at the huge lowend pull the car had (stock DP 2.25 in) however it sucked up top.