Various glaciological topics require observations of horizontal velocities over vast areas, e.g., detecting acceleration of glaciers, as well as for estimating basal parameters of ice sheets using inverse modelling approaches. The quality of the velocity is of high importance; hence, methods to remove noisy points in remote sensing derived data are required. We present a three-step filtering process and assess its performance for velocity fields in Greenland and Antarctica. The filtering uses the detection of smooth segments, removal of outliers using the median and constraints on the variability of the flow direction over short distances. The applied filter preserves the structures in the velocity fields well (e.g., shear margins) and removes noisy data points successfully, while keeping 72–96% of the data. In slow flowing regions, which are particularly challenging, the standard deviation is reduced by up to 96%, an improvement that affects vast areas of the ice sheets.