Sand it flush, my OCD will thank you. Wet sand, so that you dont breathe in the dust which is very bad for your lungs. in 2min if will be flush, it's worth it. Necessary? no. Worth doing? yes.
You can also then flex to the ladies, saying how flush your big expander is to the lip, all around, just flush and nice, like it's been machined real smooth, real nice. It might, or might not have an effect.