We present resistivity and AC calorimetry measurements of single crystalline CeNiGe2 under high pressure and low temperature in a diamond anvil cell. CeNiGe2 is a highly anisotropic, antiferromagnetic Kondo system at ambient pressure, located close to the boundary with non-magnetic systems. It has two ordering temperatures, T = 3.9 K and T = 3.2 K, which are suppressed under pressure. Between 0.7 and 1.4 GPa only a single transition was identified, with T reaching 1.9 K by 1.4 GPa. At 1.9 GPa two transitions could again be seen by AC calorimetry at 2.0 and 1.3 K, accompanied by a qualitative change in the behaviour of the resistivity. The Sommerfeld coefficient showed a maximum of 755 mJ mol K at 1.4 GPa, up from 448 mJ mol K at close to ambient pressure. The residual resistivity was found to decrease sharply above 1.4 GPa after an initial increase with pressure.