While training, the GSC tries to balance quantization and optimization. This is done by balancing the two processes through lambda-diffusion:
In pseudo-code the dynamics D at any given training step t is equal to:
D(t) = lambda_t * Optimization + (1-lambda_t) * Quantization
lambda progressively decreases from 1 to 0, starting with pure Optimization and landing at pure Quantization. The important issue here is how we calculate and update lambda.
The original code by Goldrick et al. uses this matlab function:
eigMin = min(real(eig(net.Wc))); % Find min eigenvalue of c-space weight matrix.
lambda = 1/(1+4*abs(eigMin-domain.q));
which I translated in Python as follows:
import numpy as np
min_eigenvalue = np.min(np.real(np.linalg.eigvals(self.Wc)))
l = 1 / (1 + 4*np.abs(min_eigenvalue - self.domain.q))
where self.Wc is the weight matrix and self.domain.q is the bowl-parameter q.
The authors state that this formula
THIS LOGIC MAY NOT BE CORRECT SINCE THE DRIFT TERM OF THE DIFFUSION PROCESS IS NOT EQUAL TO THE PARTIAL DERIVATIVES IN C-SPACE.
I've left the formula as in the original lacking a better idea, but I would be glad if anyone could help to improve this.
While training, the GSC tries to balance quantization and optimization. This is done by balancing the two processes through lambda-diffusion:
In pseudo-code the dynamics
Dat any given training steptis equal to:D(t) = lambda_t * Optimization + (1-lambda_t) * Quantizationlambda progressively decreases from 1 to 0, starting with pure Optimization and landing at pure Quantization. The important issue here is how we calculate and update lambda.
The original code by Goldrick et al. uses this matlab function:
which I translated in Python as follows:
where
self.Wcis the weight matrix andself.domain.qis the bowl-parameter q.The authors state that this formula
I've left the formula as in the original lacking a better idea, but I would be glad if anyone could help to improve this.