Here we evaluate five atmospheric reanalyses in an Arctic gateway during late summer. The reanalyses include ERA5, ERA-Interim, JRA-55, CFSv2 and MERRA-2. We use observations from 50 radiosondes launched in the Fram Strait around 79-80˚N, between 25 August – 11 September 2017. Crucially, data from 27 radiosondes were not transmitted to the Global Telecommunications System (GTS), and therefore not assimilated into any reanalysis. In most reanalyses, the magnitude of wind speed and humidity errors are similar for profiles with and without data assimilation. In cases without data assimilation, correlation coefficients (R) exceed 0.88 for temperature, wind speed and specific humidity, in all reanalyses. Overall, the newly released ERA5 has higher correlation coefficients than any other reanalyses as well as smaller biases and root mean square errors, for all three variables. The largest improvements identified in ERA5 are in its representation of the wind field, and temperature profiles over warm water.