I would like to say that I'm not at my first build, almost 10 bikes (road, MTB, chinese, branded, Dengfu, Cannondale, OPEN Cycle, etc...)
To resume, I cut the fork steerer under the top of the stem and when I put the expander there is more or less 3mm free, to the top of the stem, so enought to compress the headset.
There is no spacer installed under the stem.
For me this flex is coming from the stem
Apart from that, a photo of the bike under construction (size 56cm, color blue chameleon)
And a little hack on the seat post (gain 28gr)