We report several technical approaches that significantly improve the performance of a vapor-cell atomic electrometer operating in the quasi-DC frequency domain ($\ll$ 1 kHz). With a very small active volume of approximately 11 mm${}^3$ inside a 2$$2$$2cm$^3$ vapor cell, we demonstrated a noise floor for electric field (E-field) sensitivity ranging from 0.2 to 7.7 (mV/m)/$\sqrt{\rm Hz}$ for a frequency band of 1–100 Hz. Our work utilizes only a bare vapor cell for electrometry, without any metal parts or electrodes, to ensure minimal distortion of the measured E-field and to minimize the effective sensing volume for high spatial resolution. The E-field-sensitive atomic state (Rydberg state) is excited and read out optically, maximizing the simplicity of the system design and enabling the miniaturization of quasi-DC E-field sensors for potential applications, such as diagnostics of electronics without physical contact, communications in and below the super-low frequency (SLF) band, proximity detection, remote activity surveillance, tracing charge signatures, and research in bioscience and geoscience.