In the way I just spelled out. Both theoretically and evidentially.
You seemed to have spelled out your belief but not your reasoning. Try doing it in terms of physics. This is an equation of buoyant force (let me know if you need a proof for this equation). Buoyant force is the upward force due to gravity on a body immersed in a fluid.
Buoyant force = (weight of body)*(density of fluid)/(density of body)
For easy comparison to keep the "weight of body" the same, we can compare (1) a 175-pound man with (2) a 170-pound man, the same as the 175-pound man but slightly scaled down in height and he is wearing five pounds of gold chains. Both systems each weigh 175 pounds, so the weight of body is the same between them, the density of fluid is the same between them, and the only difference between them is the density of body. Person #2 combined with the gold chains he is wearing can be considered a single system. As the gold chains are very dense (19 g/cc), as in the same mass but less volume, that means system #2 has greater density than system #1. As the "density of body" is a denominator, system #2 has less buoyant force than system #1. It is essentially no different than if gold chains were no part of it and person #2 has a greater body density due to bone density, lung size and/or muscle mass.