Road four

Expectation propagation, one site at a time

EP approximates the posterior by a product of Gaussian “sites”, one per observation, and improves them one at a time. Step through every update the original EP.logit makes, in either version of the analysis.

The same four R functions, fed the data the assignment describes: 7 doses, 10 patients each, and logit p = β₀ + β₁·dose. All four roads should arrive at nearly the same posterior.

Initial state

update 0 / 35

One site update

Before the first update

Every site starts as a unit-precision Gaussian centred at zero, so the global approximation is N(0, I/N). Step forward to watch the first update.

Press play or step forward

Global approximation

β₀ and β₁ after this update

Solid ellipse: 95% region of the current EP approximation; faint: before this update. Grey contours: the exact posterior. The dashed line is where this site's tilted mean puts the linear predictor.

β₀ mean ± sd: 0 ± 0.378
β₁ mean ± sd: 0 ± 0.378

Stopping rule

Largest relative change per sweep

After each full pass over the sites, EP.logit compares the summed natural parameters with the previous pass and stops once every entry has moved by less than 10⁻⁶.

The sites

Precision each site contributes

On the linear-predictor scale. Greyed sites still hold their initial identity matrix.

dose 0I
dose 1I
dose 2I
dose 3I
dose 4I
dose 5I
dose 6I

The update

Write the approximation as $g(\beta) \propto \prod_i g_i(\beta)$ with each $g_i$ Gaussian. To refresh site i:

form the cavity $g_{-i} = g / g_i$ by subtracting natural parameters;
project it onto the linear predictor $\eta = x_i^\top \beta$ , a one-dimensional normal;
compute the mean and variance of the tilted density $g_{-i}(\eta)\, p(y_i \mid \eta)$ with three calls to R’s integrate();
choose the new site so that cavity × site has exactly those moments.

Faithful to the last digit

Because R’s integrate() stops at a fairly loose tolerance, a generic quadrature routine would give slightly different moments and a different final answer. The browser port is a line-by-line translation of QUADPACK’s DQAGS, the routine R calls, so the EP mean and covariance match R to about 10⁻¹².

In the corrected setting EP converges in 5 sweeps and lands within 0.002 of the HMC posterior mean for the dose slope. See it next to the other methods in the explorer.