Soil acidification is a critical factor leading to low soil fertility and continuous nutrient loss, severely affecting agricultural production. In this study, a typical acidic soil area in Sichuan, China, was selected as the research region. Based on a systematic analysis of the physicochemical properties of the acidic soils, a compound soil amendment consisting of Ca(OH)2-modified biochar (CBC) and fermented chicken manure was prepared to improve soil fertility and reduce nutrient leaching. Four application rates of CBC (0.5%, 1%, 2%, and 4.5%, based on dry soil weight) were investigated through a soil column leaching experiment. The results showed that the majority of soils were rather poor in fertility under acidic conditions, with several nutrients, including available phosphorus, being insufficient. In addition, the combined application of CBC and fermented chicken manure effectively improved soil fertility and reduced nutrient losses. Compared with the control, all amended treatments increased soil pH and enhanced the contents of ammonium nitrogen, nitrate nitrogen, available phosphorus, available potassium, and organic matter. Among them, the CBC(2%) treatment exhibited the highest available phosphorus content, while the CBC(4.5%) treatment showed superior performance in improving available potassium and organic matter contents. The lowest total nitrogen loss (139.42 mg/L) was observed in the CBC(2%) treatment, whereas the lowest total phosphorus loss (0.90 mg/L) and organic matter loss (224.62 mg/L) were recorded in the CBC(4.5%) treatment. These findings indicate that the combination of Ca(OH)2-modified biochar and fermented chicken manure can effectively enhance nutrient retention and improve the fertility of acidic farmland soils. The proposed amendment provides a simple, low-cost, and sustainable approach for acidic soil remediation and nutrient management in agricultural ecosystems.