+1 on the loose headset. Be careful, because that sounds like a bad case of headset play. The sort of thing that prematurely wears out a carbon steerer. If you're really heavy and really unlucky, you end up snapping the steerer, jump over the bike, and have a really bad day.
Given you're on a lightcarbon frame, it's probably user error rather than bad tolerances, so should be an easy fix once you find the issue.