Commit 920ff882 by Hafiz Imtiaz

initial git with functions and readme

parents
# author
This project is primarily maintained by Hafiz Imtiaz, Graduate Student at the Department of Electrical and Computer Engineering at Rutgers, the State University of New Jersey.
## Primary author
* [Hafiz Imtiaz](https://gitlab.com/u/hafizimtiaz)
## Additional contributors
* [Anand D. Sarwate](https://gitlab.com/u/asarwate)
# DP-OTD
DP-OTD is a MATLAB code collection for several helper functions for computing differentially-private orthogonal tensor decomposition. Currently it contains 6 functions:
* DProbustTPM - implementation function for differentially private robust TPM due Wang and Anandkumar 2016
* my_multilinear_tensor - function for computing multi-linear projection of a tensor onto some subspaces. For the mathematical explanation, see Anandkumar et al. 2014. Tensor Decomposition for Learning Latent Variable Models.
* myEigValCheck - function for checking a particular eigenvalue against a set of given eigenvalues
* myEigVecCheck - function for checking a particular eigenvector against a set of given eigenvectors
* mySymmTensor - function for creating a symmetric tensor from a list of values
* myVectorNoise - function for generating vector valued noise due Sarwate et al. 2011. Differentially private Empirical Risk Minimization.
## Contact
* Hafiz Imtiaz (hafiz.imtiaz@rutgers.edu)
## Dependencies
The codes are tested on MATLAB R2016a. The “tensorlab toolbox” is required for the functions.
## Downloading
You can download the repository at https://gitlab.com/hafizimtiaz/dp-otd.git, or using
```
$ git clone https://gitlab.com/hafizimtiaz/dp-otd.git
```
## Installation
Copy the functions in your working directory or add the directory containing the functions in MATLAB path.
## License
MIT license
## Acknowledgements
This development of this collection was supported by support from the
following sources:
* National Institutes of Health under award 1R01DA040487-01A1
Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of National Institutes of Health.
function [lambda, V] = DProbustTPM(T, K, L, R, epsilon, delta)
% this code implements the recently proposed differentially-private robust
% tensor power method by Wang & Anandkumar 2016
% lambda - recovered eigenvalues (K-by-1)
% V - matrix of eigenvectors, each column is an eigenvector (N-by-K)
% T - input tensor (should be orthogonally decomposable symmetric)
% K - desired rank
% L,R - number of iterations
% epsilon,delta - privacy parameters
N = size(T,1); % dimension
M = length(size(T)); % mode
% initialization
lambda = zeros(K,1);
V = zeros(N,K);
D = zeros(size(T));
capK = K*L*(R+1);
eps = epsilon/sqrt(capK*(4+log(2/delta)));
del = delta/(2*capK);
neu = (6/eps)*sqrt(2*log(1.25/del));
for k = 1:K
U = rand(N,L);
nrms = diag(U'*U);
U = U*diag(1./nrms);
U_new = zeros(N,L);
for r = 1:R
for l = 1:L
U_new(:,l) = my_multilinear_tensor(T-D,0,eye(N),U(:,l),U(:,l));
z = randn(N,1);
U_new(:,l) = U_new(:,l) + neu*norm(U(:,l),Inf)^2*z;
U_new(:,l) = U_new(:,l)/norm(U_new(:,l));
end
U = U_new;
end
lam = zeros(L,1);
for l = 1:L
lam(l) = my_multilinear_tensor(T-D,0,U(:,l),U(:,l),U(:,l)) + neu*norm(U(:,l),Inf)^3*randn(1);
end
[~,l_opt] = max(lam);
lambda(k) = lam(l_opt);
V(:,k) = U(:,l_opt);
D = D + lambda(k)*outprod(V(:,k),V(:,k),V(:,k));
end
\ No newline at end of file
function err = myEigValCheck(w, w_rec)
% this function computes the error in estimating a set of eigenvalues. the
% error is computed in the following way: pick one entry of w_rec, compare it
% with each entry of w and find the one with minimum absolute difference. this
% difference is the score for this entry of w_rec. sum all scores to get err.
% w - original set of eigenvalues. K-by-1
% w_rec - recovered eigenvalues (to be tested). K-by-1
% err - error between w and w_rec
% Author - Hafiz Imtiaz
K = length(w);
score = zeros(K,1);
for k = 1:K
test = w_rec(k);
d = w - repmat(test,K,1);
scores = abs(d);
score(k) = min(scores);
end
err = sum(score);
\ No newline at end of file
function err = myEigVecCheck(V, V_rec, preprocess)
% this function computes the error in estimating a set of eigenvectors. the
% error is computed in the following way: pick one col of V_rec, compare it
% with each col of V and find the one with minimum Euclidean distance. this
% distance is the score for this col of V_rec. sum all scores to get err.
% V - original set of eigenvectors. each eigenvector is in a column. N-by-K
% V_rec - recovered eigenvectors (to be tested). N-by-K
% preprocess - 1 for normalization of columns of V_rec treating those as
% probability vectors
% err - error between V and V_rec
% Author - Hafiz Imtiaz
[~,K] = size(V);
score = zeros(K,1);
if (nargin == 3) || (preprocess == 1)
V_rec = myPreProcessProbabilityVects(V_rec);
end
for k = 1:K
test = V_rec(:,k);
d = V - repmat(test,1,K);
scores = sqrt(sum(d.^2));
score(k) = min(scores);
end
err = (1/K)*sum(score);
function Vout = myPreProcessProbabilityVects(Vin)
% this function takes in a matrix with probability vectors as columns and
% checks if all elements in a column is >= 0. It also ensures the column
% sum is 1.
[N,K] = size(Vin);
Vout = zeros(N,K);
for k = 1:K
temp = Vin(:,k);
temp(temp<0) = 0;
temp = temp/sum(temp);
Vout(:,k) = temp;
end
\ No newline at end of file
function X = mySymmTensor(list, N, M)
% list - list of entries that will make the symmetric tensor
% N - dimension in each mode of the tensor
% M - number of modes of the tensor
% X - resulting symmetric tensor
% Author - Hafiz Imtiaz
if length(list) < nchoosek(N+M-1,M)
disp('not enough entries in the input list');
return;
end
temp_list = list; % back-up the input entries
dims = N*ones(1,M);
X = zeros(dims);
flag = zeros(size(X)); % flag for which entries I already entered
% first input the diagonal elements
diag_subs = diag(1:N)*ones(N,M);
diag_inds = tt_sub2ind(dims,diag_subs);
for n = 1:N
X(diag_inds(n)) = temp_list(n);
flag(diag_inds(n)) = 1;
end
temp_list(1:N) = [];
ids = find(flag==0);
% now off-diagonal entries
while numel(ids) > 0
ind = ids(1);
sub = tt_ind2sub(dims,ind);
P = perms(sub);
all_inds = tt_sub2ind(dims,P);
[num_perms,~] = size(P);
entr = temp_list(1);
for p = 1:num_perms
if flag(all_inds(p))==0
X(all_inds(p)) = entr;
flag(all_inds(p)) = 1;
end
end
temp_list(1)=[];
ids = find(flag==0);
end
% disp(['Entries left: ',num2str(find(flag==0))]);
% disp(['Is output symmetric: ',num2str(issymmetric(X))]);
\ No newline at end of file
function b = myVectorNoise( beta , N )
% this function generates a vector valued noise sample from the density
% f(b) = (1/alpha) * exp(-beta * norm(b))
% the density is proposed in Sarwate et al. 2011 Differentially Private Empirical Risk Minimization paper
% the generation procedure is described in the blog post:
% https://ergodicity.net/2013/03/21/generating-vector-valued-noise-for-differential-privacy/
% b = output noise vector
% beta = scale parameter (of the density)
% N = dimension of the vector b
% Author - Hafiz Imtiaz
r = normrnd(0,1,N,1); % generating a standard normal vector (for direction)
r = r/norm(r); % normalizing the vector
rand_norm = gamrnd(N, 1/beta, 1 ); % generate the norm of noise according to gamma distribution
b = rand_norm*r;
return
\ No newline at end of file
function T_new = my_multilinear_tensor( T , rep , varargin)
% T - input tensor, M-way N-by-N-by ... -by-N
% varargin - multiplying vectors or matrices in each mode
% rep - set to 1 if you want to multiply T with V in every mode, set to 0
% and list all different V's if different matrix should be used in
% different modes
% T_new = T(V_1, V_2, ..., V_M)
% requires "tensorlab toolbox"
% Author - Hafiz Imtiaz
[N,~] = size(T);
M = length(size(T)); %number of modes
V = varargin;
if M > (nargin-2)
for m = (nargin-2+1):M
if rep
V{m} = V{1};
else
V{m} = eye(N);
end
end
end
temp = T;
for m = 1:M
temp = tmprod(temp,V{m}',m);
end
T_new = temp;
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment