The mineral gaudefroyite, Ca4Mn3O3(BO3)3CO3, contains chains of edge-linked MnO6 octahedra such that the Mn3+ ions are located on a Kagomé lattice (perpendicular to the chains), which frustrates antiferromagnetic order. In the temperature range 6 – 24 K, small changes in magnetic field cause large entropy changes and give excellent magnetocaloric properties. Neutron powder diffraction data have therefore been collected in various magnetic fields over this temperature range to reveal the nature of the low temperature structural and magnetic properties. At temperatures below 10.2(1) K, the magnetic moments on the Mn3+ ions (3.28(5) μB at 100 mK) order to give a typical Kagomé antiferromagnetic q = 0 structure, according to a 2-dimensional XY scaling law. The magnetic frustration causes the magnetic order to be very sensitive to applied magnetic fields: a field of only 1 T rotates the moments such that 90% of the overall moment corresponds to ferromagnetic order. The properties are discussed in relation to the exceptional low field magnetocaloric properties of this mineral.