python - scikit KernelPCA unstable results -

i'm trying use kernelpca reducing dimensionality of dataset 2d (both visualization purposes , further data analysis).

i experimented computing kernelpca using rbf kernel @ various values of gamma, result unstable:

(each frame different value of gamma, gamma varying continuously 0 1)

looks not deterministic.

is there way stabilize it/make deterministic?

code used generate transformed data:

def pca(x, gamma1):     kpca = kernelpca(kernel="rbf", fit_inverse_transform=true, gamma=gamma1)     x_kpca = kpca.fit_transform(x)     #x_back = kpca.inverse_transform(x_kpca)     return x_kpca 

kernelpca should deterministic , evolve continuously gamma.
it different rbfsampler have built-in randomness in order provide efficient (more scalable) approximation of rbf kernel.

however can change in kernelpca order of principal components: in scikit-learn returned sorted in order of descending eigenvalue, if have 2 eigenvalues close each other order changes gamma.

my guess (from gif) happening here: axes along plotting not constant data seems jump around.

could provide code used produce gif?

i'm guessing plot of data points along 2 first principal components see how produced it.

you try further inspect looking @ values of kpca.alphas_ (the eigenvectors) each value of gamma.

hope makes sense.

edit: noted looks points reflected against axis, plausible explanation 1 of eigenvector flips sign (note not affect eigenvalue).

i put in a simple gist reproduce issue (you'll need jupyter notebook run it). can see sign-flipping when change value of gamma.

as complement note kind of discrepancy happens because fit several times kernelpca object several times. once settled particular gamma value , you've fit kpca once can call transform several times , consistent results. classical pca the docs mention that:

due implementation subtleties of singular value decomposition (svd), used in implementation, running fit twice on same matrix can lead principal components signs flipped (change in direction). reason, important use same estimator object transform data in consistent fashion.

i don't know behavior of single kernelpca object fit several times (i did not find relevant in docs).

it not apply case though have fit object several gamma values.