We develop theoretical methods to predict the effects of paramagnetic impurities on the microwave response of conventional spin-singlet superconductors. Our focus is on superconducting devices and resonators with low concentrations of impurities and exchange interactions with conduction electrons. We connect the sub-gap quasiparticle spectrum generated by pair-breaking to the frequency and temperature dependence of the conductivity for superconductors operating at microwave frequencies. We report theoretical results for superconducting device performance - dissipation, quality factor and frequency shift anomalies - based on self-consistent calculations of the current response and penetraion of the electromagnetic field at the vacuum-superconducting interface. Key results include the prediction of a non-monotonic anomaly in the low-frequency superfluid fraction and penetration depth at very low temperatures related to the sub-gap quasiparticle spectrum. Dissipation of microwave power is predicted from intra- and inter- impurity band transitions at GHz frequencies at low temperatures, including a physical mechanism responsible for residual resistance. We predict anomalies in the resonant frequency, f (T ), and quality factor, Q(T ), of high-Q SRF cavities operating in the GHz range at low-temperatures that are sensitive to non-magnetic and paramagnetic impurity disorder.