you need to have several mm of space, before you compress the top car, between the steerer and the top of the stem / top cap. it's that space that gets compressed to tighten the column.
if you tighten the top cap endlessly, unscrew and check: the plug probably moved up inside the steerer, instead of compressing the whole system.
If that's the case, then torque the plug more (i routinely do 8-10nm), consider carbon paste (i dont think it helps, but maybe). Do NOT compress to these levels if the steerer is NOT in situ surrounded by the stem, as the steerer deforms and i have no idea how much torque it would take to crack it. But if the stem & bearings are resisting the expansion, you can do 8-10 w/o problems.

If your problem is different, then there's probably something wonky with your setup somewhere else, like spacers contacting the frame when they shouldnt or something like that.

I ought to try picking up a girl at my next date talking about compressing steering columns. Along with different hardness of aluminum, I feel this should be very interesting to ladies.