Empirical Bayes improves assessments of diversity and similarity when overdispersion prevails in taxonomic counts with no covariates

The assessment of diversity and similarity is relevant in monitoring the status of ecosystems. The respective indicators are based on the taxonomic composition of biological communities of interest, currently estimated through the proportions computed from sampling multivariate counts. In this work we present a novel method to estimate the taxonomic composition able to work even with a single sample and no covariates, when data are affected by overdispersion. The presence of overdispersion in taxonomic counts may be the result of significant environmental factors which are often unobservable but influence communities. Following the empirical Bayes approach, we combine a Bayesian model with the marginal likelihood method to jointly estimate the taxonomic proportions and the level of overdispersion from one set of multivariate counts. We also present an extension of the methodological framework to the case of more than one sampling collection. Our proposal is compared to the classical maximum likelihood method in an extensive simulation study with different realistic scenarios. As an exemplary case, a comparison with real data from aquatic biomonitoring is also presented. In both the simulation study and the comparison with real data, we consider communities characterized by a large number of taxonomic categories, such as aquatic macroinvertebrates or bacteria which are often observed in overdispersed data. The applicative results demonstrate an overall superiority of the empirical Bayes method in almost all examined cases, for both assessments of diversity and similarity. We would recommend practitioners in biomonitoring to use the proposed approach in addition to the traditional procedures. The empirical Bayes estimation allows to better control the error propagation due to the presence of overdispersion in biological data, with a more efficient managerial decision making.