ldap filter does not recognize UTF-8 (base64)

Summary

We cannot filter on attributes that are UTF-8 (base64) encoded.

In our organization we store user information in a openldap db (that I cannot control by the way). We filter out users depending on which "ou" they belong to (we have users in several ou's that should have access).

For this we use a simple OR filter, eg. user_filter: '(|(ou=Avdelning 1)(ou=Avdelning 2)(ou=IT))'

This works well for ou's that only contain ascii characters but not so well for a ou with umlauts like 'Avdelning ÅÄÖ'.

Steps to reproduce

  1. Create ldap entry with a attribute that "becomes" base64 encoded, eg. ou=åäö.
  2. create a user_filter that should match the above, user_filter: ou=åäö

What is the current bug behavior?

the user_filter does not produces a match even if the (base64 decoded) values returned from ldapsearch match the filter string

What is the expected correct behavior?

The filter should produce a match.

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info 
gitlab-rake gitlab:env:info


System information
System:		Ubuntu 14.04
Current User:	git
Using RVM:	no
Ruby Version:	2.3.6p384
Gem Version:	2.6.13
Bundler Version:1.13.7
Rake Version:	12.3.0
Redis Version:	3.2.11
Git Version:	2.14.3
Sidekiq Version:5.0.5
Go Version:	unknown


GitLab information
Version:	10.4.2
Revision:	b1c501c
Directory:	/opt/gitlab/embedded/service/gitlab-rails
DB Adapter:	postgresql
URL:		https://git.cs.kau.se
HTTP Clone URL:	https://git.cs.kau.se/some-group/some-project.git
SSH Clone URL:	git@git.cs.kau.se:some-group/some-project.git
Using LDAP:	yes
Using Omniauth:	no


GitLab Shell
Version:	5.11.0
Repository storage paths:
  • default: /mnt/git-repos/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check 
gitlab-rake gitlab:check SANITIZE=true
Checking GitLab Shell ...

GitLab Shell version >= 5.11.0 ? ... OK (5.11.0)
Repo base directory exists?
default... yes
Repo storage directories are symlinks?
default... no
Repo paths owned by git:root, or git:git?
default... yes
Repo paths access is drwxrws---?
default... yes
hooks directories in repos are links: ...
4/2 ... ok
9/3 ... ok
4/10 ... ok
10/11 ... ok
15/12 ... ok
10/13 ... ok
12/18 ... ok
12/19 ... ok
26/26 ... ok
12/27 ... ok
10/28 ... ok
10/29 ... ok
30/31 ... ok
26/33 ... ok
15/34 ... ok
12/36 ... ok
9/37 ... ok
20/39 ... ok
20/40 ... ok
2/42 ... ok
2/43 ... ok
28/44 ... ok
10/45 ... ok
21/46 ... ok
39/47 ... ok
20/48 ... ok
48/50 ... ok
28/51 ... ok
2/52 ... ok
19/54 ... ok
50/57 ... ok
54/58 ... ok
56/59 ... ok
56/60 ... ok
59/61 ... ok
2/62 ... ok
66/63 ... ok
15/64 ... ok
19/66 ... ok
48/67 ... ok
73/72 ... ok
73/73 ... ok
15/74 ... ok
14/75 ... ok
37/76 ... ok
37/77 ... ok
78/78 ... ok
78/79 ... ok
56/80 ... ok
15/84 ... ok
56/86 ... ok
54/87 ... ok
21/90 ... repository is empty
67/92 ... ok
21/93 ... ok
82/94 ... ok
83/95 ... ok
41/96 ... ok
9/97 ... ok
22/98 ... ok
22/99 ... ok
89/100 ... ok
12/103 ... ok
74/104 ... ok
56/106 ... ok
25/107 ... ok
25/108 ... ok
41/111 ... ok
65/112 ... ok
19/116 ... repository is empty
20/117 ... ok
15/119 ... ok
111/120 ... ok
39/122 ... repository is empty
12/126 ... ok
25/129 ... ok
58/130 ... ok
41/131 ... ok
41/133 ... ok
2/134 ... ok
78/135 ... ok
19/137 ... ok
71/139 ... ok
21/142 ... ok
71/143 ... ok
128/147 ... ok
37/150 ... ok
9/151 ... ok
129/152 ... ok
71/153 ... ok
76/154 ... ok
130/156 ... ok
15/157 ... ok
21/159 ... ok
86/163 ... ok
28/164 ... ok
2/167 ... ok
19/171 ... ok
128/173 ... ok
86/174 ... ok
67/175 ... ok
41/176 ... ok
22/177 ... ok
71/178 ... ok
71/179 ... ok
67/181 ... ok
28/183 ... ok
4/185 ... ok
15/186 ... ok
41/187 ... ok
41/188 ... repository is empty
74/189 ... ok
16/190 ... repository is empty
128/191 ... ok
144/192 ... ok
145/193 ... repository is empty
41/194 ... ok
74/195 ... ok
22/196 ... repository is empty
2/198 ... ok
94/199 ... ok
25/201 ... ok
33/203 ... ok
65/206 ... repository is empty
41/207 ... ok
41/209 ... ok
130/211 ... ok
12/212 ... ok
33/213 ... ok
33/214 ... ok
136/215 ... ok
86/217 ... ok
56/218 ... ok
152/222 ... ok
149/223 ... ok
86/224 ... ok
33/225 ... ok
153/227 ... ok
153/228 ... ok
33/229 ... ok
76/230 ... ok
67/231 ... ok
2/232 ... ok
20/233 ... ok
86/234 ... ok
102/236 ... ok
58/239 ... ok
86/240 ... ok
102/241 ... ok
41/242 ... ok
121/243 ... ok
136/244 ... ok
71/246 ... ok
121/247 ... ok
22/248 ... ok
144/249 ... ok
2/250 ... ok
37/253 ... ok
37/254 ... ok
121/259 ... ok
171/260 ... ok
171/270 ... ok
171/271 ... ok
171/272 ... ok
171/273 ... ok
171/274 ... ok
171/275 ... ok
171/276 ... ok
171/277 ... ok
171/278 ... ok
171/279 ... ok
171/280 ... ok
171/281 ... ok
171/282 ... ok
171/283 ... ok
171/284 ... ok
171/285 ... ok
171/286 ... ok
171/287 ... ok
171/288 ... ok
171/289 ... ok
171/290 ... ok
171/291 ... ok
171/292 ... ok
171/293 ... ok
171/294 ... ok
171/295 ... ok
171/296 ... ok
171/297 ... ok
171/298 ... ok
171/299 ... ok
171/300 ... ok
171/301 ... ok
171/302 ... ok
171/303 ... ok
2/305 ... ok
37/306 ... ok
66/307 ... ok
2/308 ... ok
2/309 ... ok
153/311 ... ok
149/312 ... ok
41/313 ... ok
25/314 ... ok
66/315 ... ok
41/316 ... ok
128/317 ... ok
67/318 ... ok
71/319 ... ok
86/321 ... ok
58/322 ... ok
89/323 ... ok
121/324 ... ok
128/325 ... ok
37/326 ... ok
86/327 ... ok
86/328 ... ok
144/329 ... ok
102/331 ... repository is empty
22/332 ... ok
67/333 ... ok
37/335 ... ok
144/336 ... ok
21/337 ... ok
67/339 ... ok
121/341 ... ok
66/342 ... ok
67/344 ... ok
121/345 ... ok
19/347 ... ok
86/348 ... ok
189/349 ... ok
189/350 ... ok
66/351 ... ok
109/352 ... ok
109/353 ... ok
20/354 ... ok
74/355 ... repository is empty
152/356 ... ok
39/357 ... ok
39/358 ... ok
39/359 ... ok
71/360 ... ok
41/361 ... ok
74/362 ... ok
20/363 ... ok
14/366 ... ok
171/367 ... ok
171/368 ... ok
171/369 ... ok
171/370 ... ok
171/371 ... ok
171/372 ... ok
25/373 ... ok
171/374 ... ok
171/375 ... ok
171/376 ... ok
171/377 ... ok
171/378 ... ok
171/379 ... ok
171/380 ... ok
171/381 ... ok
171/382 ... ok
171/383 ... ok
171/384 ... ok
171/385 ... ok
171/386 ... ok
171/389 ... ok
171/390 ... ok
171/392 ... ok
78/393 ... ok
128/394 ... ok
20/395 ... ok
25/396 ... ok
194/398 ... ok
194/399 ... ok
111/400 ... ok
41/401 ... ok
121/403 ... ok
41/404 ... ok
37/405 ... ok
14/406 ... ok
54/408 ... ok
41/409 ... ok
39/410 ... ok
16/411 ... ok
195/412 ... ok
67/413 ... ok
201/414 ... ok
182/415 ... ok
130/416 ... ok
86/417 ... ok
121/418 ... ok
201/419 ... ok
76/420 ... ok
203/421 ... repository is empty
16/423 ... ok
48/424 ... ok
152/425 ... ok
66/426 ... ok
28/427 ... ok
71/428 ... ok
71/429 ... ok
2/430 ... ok
195/431 ... ok
144/432 ... ok
25/433 ... repository is empty
71/434 ... ok
121/436 ... ok
212/437 ... ok
212/438 ... ok
39/439 ... ok
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Check GitLab API access: OK
Redis available via internal API: OK


Access to /var/opt/gitlab/.ssh/authorized_keys: OK
gitlab-shell self-check successful


Checking GitLab Shell ... Finished


Checking Sidekiq ...


Running? ... yes
Number of Sidekiq processes ... 1


Checking Sidekiq ... Finished


Checking Reply by email ...


IMAP server credentials are correct? ... yes
Init.d configured correctly? ... skipped
MailRoom running? ... skipped


Checking Reply by email ... Finished


Checking LDAP ...


Server: ldapmain
not verifying SSL hostname of LDAPS server 'XXX.YYY.se:389'
LDAP authentication... Anonymous. No bind_dn or password configured
LDAP users with access to your GitLab server (only showing the first 100 results)
DN: uid=user1,ou=people,dc=YYY, dc=se	 uid: user1
DN: uid=user2,ou=people,dc=YYY,dc=se	 uid: user2



Checking LDAP ... Finished


Checking GitLab ...


Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ...
4/2 ... yes
9/3 ... yes
4/10 ... yes
10/11 ... yes
15/12 ... yes
10/13 ... yes
12/18 ... yes
12/19 ... yes
26/26 ... yes
12/27 ... yes
10/28 ... yes
10/29 ... yes
30/31 ... yes
26/33 ... yes
15/34 ... yes
12/36 ... yes
9/37 ... yes
20/39 ... yes
20/40 ... yes
2/42 ... yes
2/43 ... yes
28/44 ... yes
10/45 ... yes
21/46 ... yes
39/47 ... yes
20/48 ... yes
48/50 ... yes
28/51 ... yes
2/52 ... yes
19/54 ... yes
50/57 ... yes
54/58 ... yes
56/59 ... yes
56/60 ... yes
59/61 ... yes
2/62 ... yes
66/63 ... yes
15/64 ... yes
19/66 ... yes
48/67 ... yes
73/72 ... yes
73/73 ... yes
15/74 ... yes
14/75 ... yes
37/76 ... yes
37/77 ... yes
78/78 ... yes
78/79 ... yes
56/80 ... yes
15/84 ... yes
56/86 ... yes
54/87 ... yes
21/90 ... yes
67/92 ... yes
21/93 ... yes
82/94 ... yes
83/95 ... yes
41/96 ... yes
9/97 ... yes
22/98 ... yes
22/99 ... yes
89/100 ... yes
12/103 ... yes
74/104 ... yes
56/106 ... yes
25/107 ... yes
25/108 ... yes
41/111 ... yes
65/112 ... yes
19/116 ... yes
20/117 ... yes
15/119 ... yes
111/120 ... yes
39/122 ... yes
12/126 ... yes
25/129 ... yes
58/130 ... yes
41/131 ... yes
41/133 ... yes
2/134 ... yes
78/135 ... yes
19/137 ... yes
71/139 ... yes
21/142 ... yes
71/143 ... yes
128/147 ... yes
37/150 ... yes
9/151 ... yes
129/152 ... yes
71/153 ... yes
76/154 ... yes
130/156 ... yes
15/157 ... yes
21/159 ... yes
86/163 ... yes
28/164 ... yes
2/167 ... yes
19/171 ... yes
128/173 ... yes
86/174 ... yes
67/175 ... yes
41/176 ... yes
22/177 ... yes
71/178 ... yes
71/179 ... yes
67/181 ... yes
28/183 ... yes
4/185 ... yes
15/186 ... yes
41/187 ... yes
41/188 ... yes
74/189 ... yes
16/190 ... yes
128/191 ... yes
144/192 ... yes
145/193 ... yes
41/194 ... yes
74/195 ... yes
22/196 ... yes
2/198 ... yes
94/199 ... yes
25/201 ... yes
33/203 ... yes
65/206 ... yes
41/207 ... yes
41/209 ... yes
130/211 ... yes
12/212 ... yes
33/213 ... yes
33/214 ... yes
136/215 ... yes
86/217 ... yes
56/218 ... yes
152/222 ... yes
149/223 ... yes
86/224 ... yes
33/225 ... yes
153/227 ... yes
153/228 ... yes
33/229 ... yes
76/230 ... yes
67/231 ... yes
2/232 ... yes
20/233 ... yes
86/234 ... yes
102/236 ... yes
58/239 ... yes
86/240 ... yes
102/241 ... yes
41/242 ... yes
121/243 ... yes
136/244 ... yes
71/246 ... yes
121/247 ... yes
22/248 ... yes
144/249 ... yes
2/250 ... yes
37/253 ... yes
37/254 ... yes
121/259 ... yes
171/260 ... yes
171/270 ... yes
171/271 ... yes
171/272 ... yes
171/273 ... yes
171/274 ... yes
171/275 ... yes
171/276 ... yes
171/277 ... yes
171/278 ... yes
171/279 ... yes
171/280 ... yes
171/281 ... yes
171/282 ... yes
171/283 ... yes
171/284 ... yes
171/285 ... yes
171/286 ... yes
171/287 ... yes
171/288 ... yes
171/289 ... yes
171/290 ... yes
171/291 ... yes
171/292 ... yes
171/293 ... yes
171/294 ... yes
171/295 ... yes
171/296 ... yes
171/297 ... yes
171/298 ... yes
171/299 ... yes
171/300 ... yes
171/301 ... yes
171/302 ... yes
171/303 ... yes
2/305 ... yes
37/306 ... yes
66/307 ... yes
2/308 ... yes
2/309 ... yes
153/311 ... yes
149/312 ... yes
41/313 ... yes
25/314 ... yes
66/315 ... yes
41/316 ... yes
128/317 ... yes
67/318 ... yes
71/319 ... yes
86/321 ... yes
58/322 ... yes
89/323 ... yes
121/324 ... yes
128/325 ... yes
37/326 ... yes
86/327 ... yes
86/328 ... yes
144/329 ... yes
102/331 ... yes
22/332 ... yes
67/333 ... yes
37/335 ... yes
144/336 ... yes
21/337 ... yes
67/339 ... yes
121/341 ... yes
66/342 ... yes
67/344 ... yes
121/345 ... yes
19/347 ... yes
86/348 ... yes
189/349 ... yes
189/350 ... yes
66/351 ... yes
109/352 ... yes
109/353 ... yes
20/354 ... yes
74/355 ... yes
152/356 ... yes
39/357 ... yes
39/358 ... yes
39/359 ... yes
71/360 ... yes
41/361 ... yes
74/362 ... yes
20/363 ... yes
14/366 ... yes
171/367 ... yes
171/368 ... yes
171/369 ... yes
171/370 ... yes
171/371 ... yes
171/372 ... yes
25/373 ... yes
171/374 ... yes
171/375 ... yes
171/376 ... yes
171/377 ... yes
171/378 ... yes
171/379 ... yes
171/380 ... yes
171/381 ... yes
171/382 ... yes
171/383 ... yes
171/384 ... yes
171/385 ... yes
171/386 ... yes
171/389 ... yes
171/390 ... yes
171/392 ... yes
78/393 ... yes
128/394 ... yes
20/395 ... yes
25/396 ... yes
194/398 ... yes
194/399 ... yes
111/400 ... yes
41/401 ... yes
121/403 ... yes
41/404 ... yes
37/405 ... yes
14/406 ... yes
54/408 ... yes
41/409 ... yes
39/410 ... yes
16/411 ... yes
195/412 ... yes
67/413 ... yes
201/414 ... yes
182/415 ... yes
130/416 ... yes
86/417 ... yes
121/418 ... yes
201/419 ... yes
76/420 ... yes
203/421 ... yes
16/423 ... yes
48/424 ... yes
152/425 ... yes
66/426 ... yes
28/427 ... yes
71/428 ... yes
71/429 ... yes
2/430 ... yes
195/431 ... yes
144/432 ... yes
25/433 ... yes
71/434 ... yes
121/436 ... yes
212/437 ... yes
212/438 ... yes
39/439 ... yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.3.5 ? ... yes (2.3.6)
Git version >= 2.7.3 ? ... yes (2.14.3)
Git user has default SSH configuration? ... yes
Active users: ... 138


Checking GitLab ... Finished

Possible fixes

I did a quick check and you use the filter function in several places but I guess this is the correct place: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/ldap/authentication.rb#57

I am no ruby expert but other people have had similar issues when using ruby-net-ldap and did a work-around by forcing UTF-8 encoding. https://github.com/ruby-ldap/ruby-net-ldap/issues/4

Edited by 🤖 GitLab Bot 🤖