Females typically report higher social phobia levels than males in community samples, and this may be due to sex bias in assessment measures. This study aims to establish whether patterns of responding to social phobia diagnostic criteria in the Composite International Diagnostic Interview (CIDI) are comparable across males and females. A subsample of participants in the Australian National Survey of Mental Health and Wellbeing (1997) reporting at least one social fear were selected (n=1755). Analyses were conducted using a series of multi-group confirmatory factor analyses for categorical data, with unique steps to model invariance of residual variances. Partial, but not full, invariance was established, as males and females differed in their responses to items assessing physical anxiety symptoms at low levels of social fear. Whilst these differences were statistically significant, they are likely not to affect clinical practice or rates of social phobia diagnosis. This supports differences on this measure being interpreted as genuine, and strengthens findings females are more vulnerable to social phobia than males.