I now have three bikes with Shimano 12 speed, one each with full 105, Ultegra and Dura Ace, so feel well qualified to anwer this!
Functionally 105 is absolutely fine, there is very little difference between it and the others there are just these things that IMO is well worth going higher up the range:
- The shifters don't have the bonus buttons at the top, which once you have and have set up to control your computer, you will miss massively. You can set up Front Shift Next and get a free button back on 105 to control using tap, double tap, long press - but its nowhere near as good
- The rear mech on 105 has bushings in the jockey wheels. Which feels ludicrously cheap in a groupset that costs about £900 / 1000 dollars / euros. Very quick and cheap fix in paying for Ultegra jockey wheels, but you shouldn't need to do that
Then...
- As mentioned, the front mech for 105 is a big bulbous thing, whereas the Ultegra and Dura Ace look much smaller and neater, if that bothers you. They also work faster, but you won't notice that unless bench testing and specifically looking for it
- 105 cassettes are absolutely fine and work just as well as all the others, as long as you don't want 11-30. And if you wanted 11-36, you need 105. And all work with all, whatever Shimano says
And IMO these things Shimano have fallen behind aftermarket parts here:
- Cranksets. Shimano chainrings are lovely things and the front shifting is super slick, but it should not be possible to save c200g off a top of the range Dura Ace crankset with an aftermarket unit. I changed my Ultegra for a Leese when I moved to 165 cranks, keeping the original rings, which is a good compromise
- Rotors. I know some people swear by Ice Tech but I can't get on with anything above 105 level, the CL800/900s just seem to go out of true far too quickly for my liking
So if it were me... I would buy a 105 groupset, change the shifters for Ultegra, swap the jockey rings out of the 105 mech for Ultegra. And if at some point you want to save weight you can buy a carbon crankset and monoblock Spedao / Goldix cassette, and you'll end up with something lighter than Ultegra anyway.