From my experience this can be due to the coarse texture of the mohair (Angora goat) fibre. I've been looking for some alternatives for those more sensitive types...
As a result, I've become quite the fan of De Rerum Natura's Bérénice. There is something about the 20% Merinos d'Arles wool in the mix that makes it softer. Merino d'Arles wool has one of the strongest crimps of any merino. Hence, it is more loosely spun and appears to be puffier. The result is a wonderfully lightweight yarn that is soft as can be.
Do you have a favourite silk mohair alternative?