Principal component analysis (PCA) is one of the most widely used unsupervised
dimension reduction techniques. We study PCA for data from multiple related
domains. Since principal components generally differ across domains, one way
to obtain a shared low-rank embedding is to perform PCA on the pooled data.
However, this approach can focus on spurious directions that exhibit high variation
in only a few domains. To find a robust embedding that still explains most
variance in unseen but similar domains, we propose instead to focus on shared
directions of variation. To this end, we introduce Anchor PCA which trades off
overall explained variance with agreement between the shared and domain-specific
low-rank embeddings. Anchor PCA amounts to PCA on a modified target matrix
and thus can be solved efficiently. Moreover, we show that Anchor PCA recovers
a maximal invariant subspace and admits a minimax reconstruction interpretation
under bounded domain-specific covariance inflations. On simulated and real-world
data with temporal distribution shift, we demonstrate, respectively, that Anchor PCA
recovers the maximal invariant subspace and yields embeddings that explain
more variance on unseen domains than the pooling baseline and a worst-case
alternative. Taken together, these findings establish Anchor PCA as a promising
approach to robust unsupervised dimension reduction from multi-domain data.
@inproceedings{seiter2026anchorpca,title={Anchor PCA},author={Seiter, Benedikt and Fries, Anya and {von Kügelgen}, Julius and Peters, Jonas},year={2026},archiveprefix={arXiv},primaryclass={stat.ML},booktitle={Advances in Neural Information Processing Systems (NeurIPS)},note={Camera-ready version in preparation.},}