CMP.9:4.3 - Derive the estimator and its error sources

For independent draws from p, the mean of f(X_i) estimates E_p[f(X)]. Its expectation equals the target when the expectation exists. With finite variance σ², the mean’s variance is σ²/n.

For independent draws from q, use contributions Y_i=f(X_i)*p(X_i)/q(X_i) when the ratio is computable and q is positive wherever f*p contributes. Then E_q[Y_i]=E_p[f]. Inspect the second moment under q: a poor proposal can greatly increase variance. If an unknown normalizing constant leads to a self-normalized ratio of sums, its finite-sample bias and uncertainty require their own account.

For dependent draws, the variance of a mean includes covariance terms:

Var(mean Y_i)=(sum_i Var(Y_i)+2*sum_(i<j) Cov(Y_i,Y_j))/n².

This identity does not require stationarity. A chain’s approximate equilibrium, correlation and initial transient cannot be assessed just by counting iterations. Use an applicable mixing argument, a justified dependence-aware uncertainty method, or retain a weaker empirical conclusion.

Keep computational variation distinct from uncertainty in the target model. Sampling an assumed distribution accurately improves its computed consequences; it does not by itself validate that distribution for the subject.