Quantifying Uncertainty: Evaluating Trading Algorithms using Probabilistic Programming (Thomas Wiecki) 

There exist a large number of metrics to evaluate the performance and risk of a trading strategy. Although those metrics have proven to be useful tools in practice, most of them require a large amount of data and yield unstable results on shorter timescales. Quantopian allows users to develop and launch trading algorithms that invest in the stock market. As we have launched live trading less than a year ago, estimating performance with few data points becomes critical. Bayesian modeling is a flexible statistical framework well suited for this problem as uncertainty can be directly quantified in terms of the posterior distribution.

In this talk I will briefly provide an overview of Bayesian statistics and how Probabilistic Programming frameworks like PyMC can be used to build and estimate complex statistical models. I will then show how several common financial risk metrics like Alpha and Beta can be expressed as a probabilistic program. Finally, I will apply this type of Bayesian data analysis to evaluate the performance of anonymized real-world trading algorithms running on Quantopian.