The influence of the addition of silica fume (SF) on the basic mechanical and thermal parameters of cement concrete have been presented in this paper. Numerous studies show that the specific properties of silica fume allows to produce concretes with increased strength and durability properties. However, the results of the authors’ own studies have demonstrated that in specific circumstances, the SF addition can negatively affect the strength of cement concrete. In this paper, concretes were tested that were made from two types of coarse aggregate, 4-8 mm fraction: reference ordinary gravel aggregate (Ref) and broken basalt aggregate (Bas). Comprehensive tests (microstructural, strength and thermal) were carried out on three formulas: the first one (Ref-0) did not contain any additives or admixtures. Silica fume and superplasticizer were used in the second formula (Ref-8), introduced in the amount of 8% and 0.75% of the cement’s weight respectively. The third formula, based on basalt aggregate (Bas-8), also contained silica fume and a superplasticizer, dosed in the same amount as in the second formula. Porosimetric studies showed that the use of silica fume in the formula Ref-8 resulted in a decrease in the number of pores smaller than 0.15 μm in comparison to the reference concrete Ref-0 without the silica fume. On the other hand, numerous additional pores with diameters ranging from 0.05 to 300 μm were found in the basalt-based concrete Bas-8. In turn, optical porosimetry proved that the basalt-based concrete had numerous pores in the range above 70 μm, which were not observed in the case of concretes with ordinary pebble aggregate. In the case of normal concrete, the addition of silica fume resulted in an increase in the compressive strength. However, the basalt-based concrete, due to its much higher porosity, achieved significantly lower strength values. The results showed that the diagnosed changes in porosity, caused by the presence of silica fume, resulted in lower values of thermal conductivity and specific heat of mature composites that it had been added to. This was particularly evident for the basalt-based concrete.