We investigate the interplay between lattice symmetry, phonons, and magnetism in the quasi-onedimensional ladder compound BaFe2S3 by combining polarized synchrotron infrared spectroscopy, hybrid-functional density functional theory calculations, and inelastic neutron scattering. Latticedynamics analysis reveals that the crystal symmetry is lower than previously proposed and is consistent with a P 1 space group at low temperature. Several infrared-active phonon modes exhibit pronounced anomalies at both the structural transition temperature TS ≈ 125–130 K and the N´eel temperature TN ≈ 95 K. First-principles calculations show that the modes affected at TS predominantly involve displacements that modulate magnetic exchange pathways. Neutron scattering demonstrates that below TN the magnetic order is three-dimensional, long-ranged, and static. Between TN and TS, the system displays three-dimensional short-range dynamic magnetic correlations, which disappear above TS. The structural transition thus coincides with the onset of magnetic fluctuations rather than with static magnetic order. Our results point toward the fact that the BaFe2 S3 compound could be another example where short-range, dynamical magnetic correlations are sufficient to drive a static structural instability, providing a magnetically driven mechanism reminiscent of the iron-pnictide 122 family. These findings highlight the central role of magnetoelastic coupling in iron-based superconductors beyond the itinerant regime.