Is there any method to tune the regulation parameters to get a good optimization result?

[KarryMa]

Here are my codes, I really need help to tune the two regulation parameters tau1 and tau2 to let both re1 and re2 < 0.2, I have tried many many values, but still can’t get the optimal results of re1 and re2. Is there any method to search for proper regulation parameters?

clc;clear
m = 128;
n = m;
ind_s = floor(m*0.05);
ind_c = ind_s;
x0 = zeros(n,1);
x10_spt = randperm(n);
x0(x10_spt(1:ind_c)) = randn(ind_c,1);
v0 = zeros(m,1);
v0_spt = randperm(m);
v0(v0_spt(1:ind_c)) = randn(ind_c,1);
Phi = 1/sqrt(m) * randn(m,n);
data = real(Phi * x0);
N = 200;
p = (1+data)/2; 
s0 = rand(numel(data),N);
P = repmat(p,1,N);
s = P > s0; 
s1 = sum(s,2); 
s2 = s1/N;
edata = 2*s2-1; 
y = edata + v0 ;
% these two parameters I need to tune, I have tried for loop, but can't get
% good results of re1 and re2
tau1=10; 
tau2=0.05;

yalmip('clear')

                % Define variables
                x = sdpvar(n,1);   
                v = sdpvar(m,1);
                % Define constraints
                Constraints = [x >= 0]; 
                % Define an objective
                %%% 
                Objective = 0.5*norm(y - Phi * x - v,2)^2 + tau1*norm(x,1)+ tau2*norm(v,1);
                sol = optimize(Constraints,Objective);
                % Analyze error flags
if sol.problem == 0
 % Extract and display value
x = value(x);
v = value(v);
else
 disp('Hmm, something went wrong!');
 sol.info
 yalmiperror(sol.problem)
end          

re1 = norm(x - x0, 2)/ norm(x0, 2);
re2 = norm(v - v0, 2)/ norm(v0, 2);
% need re1 and re2 < 0.2
re1
re2

[johanlofberg]

Unclear what you are asking about as we already concluded you cannot solve this, so how can you talk about improving the results

Also, the model is hilariously out of touch with reality. X is d x d where d = 2^n so it will have 2^2n elements, each taking 8 bytes, so in total 2^(2n+3) bytes to store a solution (and actually representing it using YALMIP symbolics is far worse). Let us assume you store it using memory swaps on a hard disk cluster where each harddisk takes 1 TB, i.e. 10^12 bytes. Let us assume they build 10^4 such disks per second and you buy them all. How long time will it take you to have bought sufficiently many hard disks, measured in ages of the universe? 2^(3+2 * 128)/(10^4 * 3600 * 24 * 365 * 13.7 * 10^9). We arrive at 2 * 10^56 ages of the universe...

[KarryMa]

Sorry, I copied the wrong codes in the module of yalmip. I made a change.

[johanlofberg]

There is no magic way to tune regularization parameters

[KarryMa]

Well, thanks. I would tackle the problem from other perspectives.

[KarryMa]

BTW, I used the genetic algorithm to search for the regularization parameters by set the values of x, v (in this situation, x, v are known, and I use them to solve regularization parameters) , but while I get the regularization parameters, them I bring them back to the objective function, I can't get the original x, v (or their approximation), why?

[johanlofberg]

I have no idea what you are asking, but it sounds a bit like you are asking why a heuristic method does not compute the exact solution to a hard cardinality constrained problem, and the answer to that question is of course that it is a heuristic method which has no guarantees in general.