The sound energy from marine mammal populations vocalizing over extended periods of time adds up to quasi-continuous “choruses” which create characteristic peaks in marine sound spectra. We present an approach to estimate animal distribution that uses chorus recordings from very sparse unsynchronized arrays in ocean areas that are too large or remote to survey with traditional methods. To solve this underdetermined inverse problem, we use simulated annealing to estimate the distribution of vocalizing animals on a geodesic grid. This includes calculating a transmission loss matrix, which connects all grid nodes and recorders. Geometrical spreading and the ray trace model BELLHOP were implemented. The robustness of the proposed method was tested with simulated marine mammal distributions in the Atlantic sector of Southern Ocean using both drifting acoustic recorders (Argo floats) and a moored array as acoustic receivers. The results show that inversion accuracy mainly depends on the number and location of the recorders and can be predicted using the entropy and range of the estimated source distribution. Tests with different transmission loss models indicated that inversion accuracy is affected only slightly by inevitable inaccuracies in transmission loss models. The presented method could also be applied to bird, crustacean and insect choruses.