Something that a VERY good tyre place might do is put the rim on, see how it balances, then mount the tyre with the high spot or the heaviest part of the tyre (is that the red or yellow dot?) furthest away. In theory at least it will give you the best chance of balancing the combo up correctly. The other thing that might be affecting the job is if the rim is not true. I remember seeing something online, about a home-made rig with a laser mounted to the rim/hub, to show multi plane balancing. Can't find it now, but I reckon that it was something like an amateur version of this.
This might be anecdotal, There was a place that used to pride itself on how little weight that they could add to a rim to balance the tyre, they used to spend ages when the shop was quiet getting ones that they had in for a day just right. I wish this was true and that it was local to me...
BB