Taylor's swimming sheet is a classical model of microscale propulsion and pumping. Many biological fluids and substances are fibrous, having a preferred direction in their microstructure; for example cervical mucus is formed of polymer molecules which create an oriented fibrous network. Moreover, suspensions of elongated motile cells produce a form of active oriented matter. To understand how these effects modify viscous propulsion, we extend Taylor's classical model of small-amplitude zero-Reynolds-number propulsion of a `swimming sheet' via the transversely-isotropic fluid model of Ericksen, which is linear in strain rate and possesses a distinguished direction. The energetic costs of swimming are significantly altered by all rheological parameters and the initial fibre angle. Propulsion in a passive transversely-isotropic fluid produces an enhanced mean rate of working, independent of the initial fibre orientation, with an approximately linear dependence of energetic cost on the extensional and shear enhancements to the viscosity caused by fibres. In this regime the mean swimming velocity is unchanged from the Newtonian case. The effect of the constant term in Ericksen's model for the stress, which can be identified as a fibre tension or alternatively a stresslet characterising an active fluid, is also considered. This stress introduces an angular dependence and dramatically changes the streamlines and flow field; fibres aligned with the swimming direction increase the energetic demands of the sheet. The constant fibre stress may result in a reversal of the mean swimming velocity and a negative mean rate of working if sufficiently large relative to the other rheological parameters.