Evaluation of software tools for Bayesian estimation on population models with count and continuous data.
Objectives: In recent years, Bayesian modelling techniques have received increasing attention. Different tools have been developed to perform Bayesian estimation using Markov Chain Monte Carlo (MCMC) methods. The aim of this work is to compare five tools in order to evaluate their performances and limitations on both algebraic and ordinary differential equation (ODE) population models..
Methods: WinBUGS 1.4.3 (with BlackBox Component Builder 1.5 and WBDiff interface), Stan 2.5.0 (with RStan 2.5.0 interface), OpenBUGS 3.2.3, NONMEM 7.3.0 and JAGS 3.4.0 were compared. Two models were selected for this purpose: a Poisson count model concerning a randomized clinical trial of an anticonvulsant for epilepsy treatment , and a two-compartment PK ODE model with linear and non-linear elimination, already used for a Phase I dose escalation study of a monoclonal antibody for epilepsy [2,3]. For the first model, data and uninformative priors from a published study were used to perform the estimates . For the second model, PK data were simulated using the Simulx function of the R package mlxR and informative priors were defined using literature data . The tools were tested on a single platform and the Effective Sample Size (ESS, computed via R coda package) per execution time (T) was used as a performance index for comparison.
Results: Similar posterior distributions were obtained with all the tools for the Poisson count model. OpenBUGS and Stan showed superior performance in terms of ESS/T. In NONMEM, the implementation of the model requires the objective function to be written explicitly, resulting in a less user-friendly model encoding than the other tools. NONMEM also has a limited distribution choice (Normal and InverseWishart), hence parameter transformations were required to encode the chosen models. The ODE model could be implemented only with NONMEM and WinBUGS, since JAGS does not include an ODE solver, OpenBUGS gives errors solving population models with ODEs and Stan could not finish the estimation process. NONMEM and WinBUGS with BlackBox showed comparable performance for the ODE model, while WinBUGS alone showed, on average, a considerably lower ESS/T.
Conclusions: The performed tests highlighted tool-specific limitations and performance differences. The evaluation on models with different features, like the ones used in this work, will support the choice of the most suitable tool for Bayesian estimation tasks in several contexts.
This work was supported by the DDMoRe project (www.ddmore.eu).
 R.Lledo-Garcia, F.Strimenopoulou, R. Oliver, M.Zamacona. Dose escalation studies for mAb: prior distributions selection and software comparison. Proceedings of the PAGE meeting (2012), June 5-8, Venice, Italy.
 F. Strimenopoulou, R. Oliver, M.Zamacona. Bayesian non-linear PK modelling applied to dose escalation studies using WinBUGS. Proceedings of the Bayes 2012 meeting, May9-11, Basel, Switzerland.
Full Reference: E. Borella, L. Carrara, S.M. Lavezzi, E. Mezzalana, L. Pasotti, G. De Nicolao, P. Magni. Evaluation of software tools for Bayesian estimation on population models with count and continuous data. PAGE 24 (2015) Abstr 3452 [www.page-meeting.org/?abstract=3452].
Link to full text: https://www.page-meeting.org/?abstract=3452