Here's what I did (see the post below):
1) I text paged my coworker. Yes text paged her -- this was one of those crazy days where I just couldn't escape the cath lab, and wouldn't run into her otherwise. I said, "Thanks so much for the Indian dinners, that was really sweet of you. Unfortunately, since I'm a vegan, I can't eat them because they have milk. Sorry!" I might whip up a batch of chocolate chip cookies as a token of my gratitude, because it was nice of her to think of me.
2) I took the sweater. I'd rather re-use something that would otherwise be given away or end up in a landfill. Further, this way I didn't buy a sweater, which would involve using up resources, energy, and probably in some manner isn't good for the environment. Besides, who other than me will know it's cashmere?