Simply put, sprung weight is everything from the springs up, and unsprung weight is everything from the springs down. Wheels, shock absorbers, springs, knuckle joints and tyres contribute to the unsprung weight. The car, engine, fluids, you, your passenger, the kids, the bags of candy and the portable Playstation all contribute to the sprung weight. Reducing unsprung weight is the key to increasing performance of the car. If you can make the wheels, tyres and swingarms lighter, then the suspension will spend more time compensating for bumps in the road, and less time compensating for the mass of the wheels etc.
The greater the unsprung weight, the greater the inertia of the suspension, which will be unable to respond as quickly to rapid changes in the road surface.
As an added benefit, putting lighter wheels on the car can increase your engine's apparent power. Why? Well the engine has to turn the gearbox and driveshafts, and at the end of that, the wheels and tyres. Heavier wheels and tyres require more torque to get turning, which saps engine power. Lighter wheels and tyres allow more of the engine's torque to go into getting you going than spinning the wheels. That's why sports cars have carbon fibre driveshafts and ultra light alloy wheels.