Does this always turn into a big problem if you do not tend to it when making the clicking noise?

Is there any good way to tell if the clicking is the nuts vs., say, brake pads shifting? Mine just clicks when I am going from reverse to forward, or vice versa, at slow speeds.