1-DREAM: 1D recovery, extraction and analysis of manifolds in noisy environments

Filamentary structures (one-dimensional manifolds) are ubiquitous in astronomical data sets. Be it in particle simulations or observations, filaments are always tracers of a perturbation in the equilibrium of the studied system and hold essential information on its history and future evolution. However, the recovery of such structures is often complicated by the presence of a large amount of background and transverse noise in the observation space. While the former is generally considered detrimental to the analysis, the latter can be attributed to measurement errors and it can hold essential information about the structure. To further complicate the scenario, one-dimensional manifolds (filaments) are generally non-linear and their geometry difficult to extract and model. Thus, in order to study hidden manifolds within the dataset, particular care has to be devoted to background noise removal and transverse noise modelling, while still maintaining accuracy in the recovery of their geometrical structure. We propose 1-DREAM: a toolbox composed of five main Machine Learning methodologies whose aim is to facilitate manifold extraction in such cases. Each methodology has been designed to address particular issues when dealing with complicated low-dimensional structures convoluted with noise and it has been extensively tested in previously published works. However, for the first time, in this work all methodologies are presented in detail, joint within a cohesive framework and demonstrated for three particularly interesting astronomical cases: a simulated jellyfish galaxy, a filament extracted from a simulated cosmic web and the stellar stream of Omega-Centauri as observed with the GAIA DR2. Two newly developed visualization techniques are also proposed, that take full advantage of the results obtained with 1-DREAM. This contribution presents the toolbox in all its details and the code is made publicly available to benefit the community. The controlled experiments on a purposefully built data set prove the accuracy of the pipeline in recovering the real underlying structures.