The unclear picture of the sex difference in color preference might result from personality variations. We invited 359 Chinese university students (166 men and 193 women) to undergo a color preference test and the Zuckerman-Kuhlman Personality Questionnaire (ZKPQ), a five-factor model test. Depressive trends were measured by the Plutchik-van Praag Depression Inventory (PVP). There was no significant difference between men and women regarding either ZKPQ or PVP scale scores. However, men preferred blue and green significantly more, and their preference order of yellow was negatively correlated with ZKPQ Sociability. Women preferred purple, pink and white significantly more, their preference order of gray was positively correlated with Neuroticism-Anxiety, and the order of orange negatively with Aggression-Hostility. Our results suggest that, partly from a biological layout, men as hunters and women as gatherers prefer some different colors on the one hand, but from a social structural layout, they might try to adjust some personality traits by preferring other colors on the other, in order to attain a sex-equality but polychromatic world.