Heres what I found out:
1) My steerer tube is cut a tad too long likely.
2) The velobuild compression plug was slipping as I torqued the top cap down, causing the top cap to hit the top of the plug and the stack to not have enough compression.
My fix was to get a $19 FSA compression plug, these things are great since you can adjust how deep the plug sits and you should always be able to create room to compress the stack. Worked perfectly with a bit of carbon paste.
https://shop.fullspeedahead.com/en/star-nut-and-compression-device/road-mtb/compression-device-3819For future builders the proper order to tightening everything is:
1) Tighten compression plug to 6-9nm.
2) Push down with your hands on the bars and make sure all headset alignment pins are fully engaged
3) Tighten top cap until headset is fully compressed and no play remains.
4) Tigten stem bolts