Next to physique, your technique and playstyle play a role in this. Having a playstyle that's very wrist dependent, lots of flips, wrist-driven direction changes, stuff like that, can be really hard to maintain with a heavier bat. But also, if your technique is smooth and divides the stress along the core, shoulder, elbow and wrist, that's more manageable than a technique with not a lot of body movement.