SyncAI.news, a Varaisys broadcasting
How Should Parallel Langevin Chains Share Noise?
RF

Ryan Farell

· 1 min read

ResearcharXiv cs.LG

How Should Parallel Langevin Chains Share Noise?

arXiv:2611.19784v1 Announce Type: new Abstract: Parallel Langevin sampling usually draws an array of independent Gaussian vectors, one row per chain. The average row drives the ensemble mean, with variance $1/K$ per coordinate for $K$ chains. Yet the unadjusted Langevin algorithm (ULA) specifies only each row's distribution. Subtracting the column means and rescaling, as in classical antithetic sampling, makes that variance zero. Each chain keeps its ordinary ULA path law for any finite run. For quadratic targets we derive the joint covariance of linear estimates computed from the ensemble mean, including coefficient estimates, time averages and predictions. The identity separates contributions from initial states and Gaussian draws. The coupling changes the second through one scalar, which is zero whenever the draws sum to zero across chains. For nonquadratic targets we bound the remaining variance. Centering has a price: for a quadratic form in the mean-subtracted Gaussian state, the ensemble average has variance $1/(K-1)$ relative to one chain, compared with $1/K$ under independent draws. On three tempered logistic posteriors, eight centered chains reduced the variance of the posterior-mean estimate, summed over coordinates, by two to three orders of magnitude compared with eight independent chains. They also beat 128 independent chains on variance and on variance times sampling time. Under Metropolis correction the reduction shrank to about three- to eightfold: rejected draws no longer balance accepted ones. The measured variance ratios were consistent with a leading-order prediction from rejection frequency and noise energy.

Original source

This story was published by arXiv cs.LG and written by Ryan Farell. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News