Brass or aluminium: It’s a weight difference of around 0,7g per nipple. You do the math. I too do ride wheels with aluminium nipples on my gravel bike. If they’re of good quality, like Sapim Polyax for instance, it’s likely that you’ll never have a problem. On my commuter though, I would never use them. I rarely clean that bike cuz I’m lazy. Winter salt builds up on it etc. With brass nipples, corrosion is simply out of the question.
Also, there is the possibility of galvanic corrosion between badly anodised nipples and carbon rims. IIRC Enve had this happen to them quite a while back. The nipples just disintegrated at some point.
But I think none of that is an issue anymore today if you’re using top quality stuff. And there’s really not much money to be saved by going cheap on nipples.