How to handle uncertainty in future projections?

How to handle uncertainty in future projections?
Samu Mäntyniemi, Fisheries and Environmental Management group (FEM), University of Helsinki Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

My background PhD in biometry Bayesian inference in fisheries science
Dynamic models Alternative model structures Lots of uncertain parameters Interest in latent variables: the true stock size has never been observed Role in the FEM group Supervising the use of statistical methods Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

Purpose and outline of the talk
Food for thought Basic principles illustrated with simple examples Three parts Part I: There is no uncertainty Part II: Correlation is good for you Part III: Weight the average Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

Part I: There is no uncertainty
Uncertainty does not physically exist Uncertainty exists in mind Uncertainty only exists in the context of a person Subjective object by definition Uncertainty of a ”collective mind” Scientific community Panel of experts Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

Uncertainty about the future
When future becomes observed, it is not the future anymore We can only have beliefs about the future The belief can be well documented, consistently updated and scientifically sound But it is still a (scientific) belief Varies between scientists Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

Is my uncertainty more correct than yours?
Uncertainty is what it is: everyone is right Being wise afterwards: a reality check? Predict the future Observe what really happened Compare and learn : ”calibration” Some statements of uncertainty fit to observations better than others How to take this into account? Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

Sources of uncertainty: Why I am uncertain?
Unexplained natural variability Can be reduced by better modeling Uncertainty about causalities Alternative models Uncertainty about model parameters More data -> less uncertainty Uncertainty about implementation of management actions How will governments, industry and consumers react? Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

How to handle uncertainty systematically?
Probability: measure of uncertainty Interpretation of probability: Relative frequency Or degree of belief? RF is not defined for unique events can not be repeated Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

Handling uncertainty : statistical inference
Classical (frequentist) statistics Uncertainty about potential data sets under a fixed hypothesis: long run relative frequencies Assumes a repeatable experiment to define the concept of probability Bayesian statistics Uncertainty about hypotheses given a fixed set of data Probability : personal degree of belief Data consists of facts, everything else is a matter of believing Mathematical logic for handling beliefs Transparent and consistent framework Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

Learning from experience : location of a fish school
1. Prior knowledge 1 0.8 2. New observation 0.6 Probability 0.4 0.2 3. Learning Estuary Offshore 4. Posterior knowledge Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

The Bayes’ rule: ”Artificial intelligence”
h : hypothesis x : observed data Likelihood: interpretation of observed data Prior: initial knowledge Posterior: updated knowledge Normalizing constant: predictive probability of data Learning: use posterior as prior for the next case

Part I : Summary I Everyone is right about their own uncertainty/beliefs Best we can do: Systematic approach to our own uncertainty Honest account of uncertainty Emphasize clear communication of uncertainty But some people’s beliefs are more consistent with data than others : higher ”expertise” Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

Part I : Summary II Classical statistics : Bayesian statistics:
No direct statements of uncertainty about past and future states of nature, no parameter/structural uncertainty Bayesian statistics: Consistent framework for handling beliefs Probability as measure of uncertainty Baye’s rule for learning from data Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

Part II: Correlation is good for you
”Correlation of parameter estimates is a problem, as it indicates that the parameters are difficult to identify” On the other hand: Higher correlation = more information Less correlation = less information Example from linear regression Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

Bayesian linear regression: predicting surface runoff using precipitation
Priors Intercept : between -50 and 0 Slope: betweem 0 and 1 Add one data point at a time Show predictions with and without correlation of parameters Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

Part II: Summary Correlation is information
Include correlation when predicting, not just the marginal distributions Report the correlation of your parameter estimates, so that others can use it When using expert knowledge to formulate priors for prediction, try to assess the combinations of variables. Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

Part III: model averaging
Multiple models Each model gives different prediction Not sure which model is ”true” How to take this uncertainty into account? Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

Bayesian Model Averaging
Each model represents a hypothesis Baye’s rule: probability of a hypothesis | data Just use it! Give prior weights to models before seeing the data Predict the next time step Update the weights based on observed data Go to (2) Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

Example: three models for an environmental Index
Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

Uncertainty: which model is ”true”?
Bayesian approach: assing prior probability for each model before seeing the data In this example: pretend that ”nothing” is know a priori P(model=white noise) = 1/3 P(model=white noise | data) = ? Demonstration Predict upcoming data Observe data Update P( model | data), use as prior for next case Go to (1) Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

How it works? P(data)=ΣP(data | model)P(model) In other words: Predictive distribution of data is weighted average of model specific predictive distributions Weights are updated based on the predictive performance of individual models Just using the Bayes’ rule Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

The next step Prediction using updated weights P(model | data)
Update the weights again using a new observation Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

And so on… Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

Part III: summary Bayesian Model Averaging : consistent way of accounting for model uncertainty Theory is very simple Practice can be difficult Typically analytically intractable Approximations needed Ideal: parameters of each model should be updated after each new data point : ”Correlation is information” Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

Methods to estimate uncertainties of scenario simulations
Bayesian framework Structural & parameter uncertainty Expert knowledge and data Fit all models to all data using Baye’s rule : full uncertainty accounted for, including the effect of correlations of parameter estimates Computational tools Monte Carlo methods: parameter estimation & model uncertainty Bayesian networks : Decision anlysis, integration of simulation output, interactive format Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

Possibilities to combine Bayesian risk analysis and climate change ensemble modelling?
If the models in the ensemble provide probabilistic prediction: Should be straightforward, in theory Computational problems may arise If the models are deterministic Not possible without changes to models Predict a data set, learn residual variation -> use as uncertainty in further predictions And/or use expert judegement to estimate the uncertainty Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

Monthly Weather Review, 133, 1155-1174
Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

Thank you! Biotieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

How to handle uncertainty in future projections?

Samankaltaiset esitykset

Esitys aiheesta: "How to handle uncertainty in future projections?"— Esityksen transkriptio:

Samankaltaiset esitykset

Projektista

Palaute

Kirjaudu sisään

Kirjaudu sisään sosiaaliverkostojen kautta:

How to handle uncertainty in future projections?

Samankaltaiset esitykset

Esitys aiheesta: "How to handle uncertainty in future projections?"— Esityksen transkriptio:

Samankaltaiset esitykset

Projektista

Palaute