Smoothing and the environmental manifold

How the observed occurrences of a species relate to environmental gradients is a fundamental question in community ecology. In this paper, we present a new approach to address this question, using the smoothing function, a method not previously recruited into this ecological context. Using simulation techniques, we explore its accuracy in recovering known species distributions from simulated noisy data, and we compare the smoothing function's predictive abilities to two widely used methods in this field, the generalised linear model (GLM) and random forest machine learning. In studying the smoothing function, we are led to consider a new analytical tool for ecology, which we call the environmental manifold. It is given by the shape of the data cloud of sampled environmental predictor variables, and has deep relevance to ecological niche theory. Hitherto not considered in ecological analyses, it plays a fundamental role in understanding the species-environment relationship, and we utilise it to compare the performance and behaviour of these three methods. The results of our analysis find both random forest and smoothing to be robust to the complexities of the species-environment relationship, and also, to a degree, the shape of the environmental manifold. In contrast, the GLM's accuracy depends heavily on the complexity of the species-environment relationship, and is also affected by the geometry of the environmental manifold. Furthermore, the smoothing function is seen to be more accurate than random forest in every combination of species-environment relationship and environmental manifold shape, and also less affected by sampling bias. This suggests the promising role that such smoothing functions can have in ecological analyses. Our results also support the robustness of random forest machine learning to nonlinearity in both the species-environment relationship, and for the first time, the complexity of the shape of the environmental manifold. We conclude by discussing the implications and uses of the environmental manifold in ecological practice and theory, including its importance for niche theory, understanding species distributions, and conservation policy.