Characterizing the elastic properties of carbonate rocks is a crucial parameter in geotechnical and reservoir engineering. However, these natural materials exhibit a wide variety of microstructures, even when they belong to the same facies, leading to a significant dispersion of their elastic properties. We focus on six selected rocks, for which global porosity is shown to be not the only factor controlling elasticity. This study aims to understand which additional microstructural parameters impact the rocks' elastic properties by means of nanoindentation coupled with microscopic observations and analyses. A complete procedure is developed to combine nanoindentation measurements with Scanning Electron Microscopy (SEM) - Energy Dispersive Spectroscopy (EDS) analyses to identify volume fractions and elastic properties of the mineral phases in each rock. The macroscopic Young's moduli are estimated using analytical homogenization and are finally compared with the values obtained from macroscopic experiments. We find that the mineral composition and mechanical properties are critical parameters involved in carbonate rocks' elastic behavior. However, while capturing the overall trend, homogenization is found to overestimate Young's modulus. A discussion on the possible impact of cracks and non-spherical pores is conducted to explain this overestimation. Besides, microindentation tests are carried out to explore higher scales, which remain too small, however, to characterize the macroscopic Young's modulus of carbonate rocks.