Howdy. If by the shut off you are referring to the pressure switch, the air will probably be leaking out the little dooey (technical term) on one side of the switch.
If that is the case, check out the page on CHECK VALVE, as that is likely your problem.