We investigate the shear elastic modulus of soft polymer foams loaded with hard spherical particles and we show that, for constant bubble size and gas volume fraction, strengthening is strongly dependent on the size of those inclusions. Through an accurate control of the ratio λ that compares the particle size to the thickness of the struts in the foam structure, we evidence a transition in the mechanical behavior at λ ≈ 1. For λ < 1, every particle loading leads to a strengthening effect whose magnitude depends only on the particle volume fraction. On the contrary, for λ > 1, the strengthening effect weakens abruptly as a function of and a softening effect is even observed for λ ≳ 10. This transition in the mechanical behavior is reminiscent of the so-called “particle exclusion transition” that has been recently reported within the framework of drainage of foamy granular suspensions [Haffner B, Khidas Y, Pitois O. The drainage of foamy granular suspensions. J Colloid Interface Sci 2015;458:200-8]. It involves the evolution for the geometrical configuration of the particles with respect to the foam network, and it appears to control the mechanics of such foamy systems.