Speak to the children of Israel, and let them take to you a perfect red cow, with no blemish, which has never worn a yoke. (Bamidbar 19:2)
That the cow have no blemish is a well-known condition for sacrifices. Why, though, the requirement of never having worn a yoke?
Ever since the first-ever sacrifice in Breishit, where Hevel's offering of firstborn sheep was accepted by God, it has been preferred that "the first" of something be used for sacrifices. This is reflected in numerous later examples, such as bikurim for fruits, the laws of bechor for animals and people, and perhaps even Jericho (the first city conquered in Israel) becoming "herem... la-hashem" (Yehoshua 6:17) rather than being plundered by the conquerers.
I think the rule about the yoke is also a case of sacrificing "the first". In this case, that means that the first work done with the cow should be the act of sacrifice. The word for field labor is "avodah", and a sacrifice is also considered a form of "avodah". Both field labor and sacrifice are part of the service/servitude which might be expected of a cow.
I haven't thought about why exactly this requirement is absent for most sacrifices (one can come up with all sorts of guesses why). But where it is present (for parah adumah, and also for eglah arufah), I think the reason is in order to sacrifice "the first".