Expressive communication in the arts often involves deviations from stylistic norms, which can increase the aesthetic evaluation of an artwork or performance. The detection and appreciation of such expressive deviations may be amplified by cultural familiarity and expertise of the observer. One form of expressive communication in music is playing “out of time,” including asynchrony (deviations from synchrony between different instruments) and non-isochrony (deviations from equal spacing between subsequent note onsets or metric units). As previous research has provided somewhat conflicting perspectives on the degree to which deviations from synchrony and isochrony are aesthetically relevant, we aimed to shed new light on this topic by accounting for the effects of listeners' cultural familiarity and expertise. We manipulated (a)synchrony and (non-)isochrony separately in excerpts from three groove-based musical styles (jazz, candombe, and jembe), using timings from real performances. We recruited musician and non-musician participants (N = 176) from three countries (UK, Uruguay, and Mali), selected to vary in their prior experience of hearing and performing these three styles. Participants completed both an aesthetic preference rating task and a perceptual discrimination task for the stimuli. Our results indicate an overall preference toward synchrony in these styles, but culturally contingent, expertise-dependent preferences for deviations from isochrony. This suggests that temporal processing relies on mechanisms that vary in their dependence on low-level and high-level perception, and emphasizes the role of cultural familiarity and expertise in shaping aesthetic preferences.