ConvergenceWarning probelm in findtailthreshold.py

[taochou-cogsci]

When fitting the model, I encountered the following warning:

~/.conda/envs/taoenv-glmsingle/lib/python3.10/site-packages/sklearn/mixture/_base.py:275: ConvergenceWarning: Best performing initialization did not converge. Try different init parameters, or increase max_iter, tol, or check for degenerate data.

Upon investigation, I found that the issue originates in the file findtailthreshold.py, specifically at Line 55:

# fit mixture of two gaussians
gmfit = gmdist(n_components=2, tol=1e-10, reg_covar=0, n_init=numreps).fit(v2.reshape(-1, 1))

After reviewing the use of sklearn.mixture.GaussianMixture, I noted that the default values for the parameters are: tol = 1e-3, reg_covar = 1e-6, max_iter = 100. The warning indicates that the model is not converging during initialization, which suggests the chosen parameter values might need adjustment.

i). I understand that the goal of setting tol to such a small value is likely to improve precision in convergence.

ii). By default, reg_covar is set to 1e-6, which applies a small amount of regularization to the covariance matrix to ensure stability and prevent issues such as singular covariance matrices. Setting reg_covar = 0 disables this regularization, which could lead to convergence issues, particularly if the data is degenerate or lacks variability. Can you clarify why reg_covar was set to 0 in this case?

iii). Given that the model is not converging, I was considering increasing max_iter to 200, which may provide more iterations for the algorithm to converge, especially if the initial parameters are far from the optimal solution. I tried setting reg_covar by default and max_iter as 200, and it did solve the problem! But I'm worried about the overfitting problem.

I’d appreciate your feedback or any recommendations for improving the parameter settings to ensure better convergence!

[kendrickkay]

To better understand what's going on, it would be helpful to see the exact distribution of values that the code is attempting to fit the GMM to. Have you looked into what that looks like?

[kendrickkay]

One additional remark: The reason that "reg_covar was set to 0" was to achieve equivalence with the MATLAB version (where by default, fitgmdist.m has a parameter that appears to be equivalent and is 0 by default). Using 0 seems like a safe/conservative route. Since GM fitting in this context (just to a distribution of R^2 values) is fairly simple, it would seem important to try to understand why convergence is being problematic. It likely points to a data scenario that has close to no perceptible signal, in which case that seems like the real underlying problem to address.