Commits · abartlet-ldb-1.4.0 · Catalyst Samba Development / samba

28 May, 2018 11 commits

Andrew Bartlett authored Mar 20, 2018



* New LMDB backend
* Comprehensive tests for index behaviour
* Enforce transactions for writes
* Enforce read lock use for all reads
* Fix memory leak in paged_results module.
  We hold at most 10 outstanding paged result cookies
  (bug #13362)
* Fix compiler warnings
* Python3 improvements
* Restore --disable-python build
* Fix for performance regression on one-level searches
  (bug #13448)
* Samba's subtree_rename could fail to rename some entires
  (bug #13452)

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

77c93c15

selftest: Add test to show that sam.ldb does not do a full scan in startup · 124c0d66

Andrew Bartlett authored May 23, 2018



We should add some other more complex operations here.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13448

124c0d66

ldb: Add tests for when we should expect a full scan · 35d6ea92

Andrew Bartlett authored May 23, 2018



Signed-off-by: Andrew Bartlett <abartlet@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13448

35d6ea92

ldb: One-level search was incorrectly falling back to full DB scan · f58e32ce

Andrew Bartlett authored May 21, 2018 and

Andrew Bartlett committed May 28, 2018



When no search filter is specified, the code falls back to using
'(|(objectClass=*)(distinguishedName=*)'. ltdb_index_dn() then failed
because matching against '*' is not indexed. The error return then
caused the code to fallback to a full-scan of the DB, which could have a
considerable performance hit.

Instead, we want to continue on and do the ltdb_index_filter() over the
indexed results that were returned.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13448

f58e32ce

ldb: Explain why an entry can vanish from the index · 4b87c4d6
Andrew Bartlett authored May 28, 2018
```
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
```
4b87c4d6
ldb: Indicate that the ltdb_dn_list_sort() in list_union is a bit subtle. · c4ab8090
Andrew Bartlett authored May 28, 2018
```
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
```
c4ab8090

ldb: Save a copy of the index result before calling the callbacks. · 5b4546ea

Andrew Bartlett authored May 28, 2018

Otherwise Samba modules like subtree_rename can fail as they modify the
index during the callback.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13452



Signed-off-by: Andrew Bartlett <abartlet@samba.org>

5b4546ea

subtree_rename: Correct comments · 3495e2e9
Andrew Bartlett authored May 28, 2018
```
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
```
3495e2e9

dsdb: Remove sort from subtree_delete and add comments. · 0ded555c

Andrew Bartlett authored May 28, 2018



The sort was written back when the module did not operate recursivly
over the tree. Now it is just confusing, so replace with useful
comments.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>

0ded555c

selftest: Lock down the expected parents in BasicTreeDeleteTests · 8244256e
Andrew Bartlett authored May 28, 2018
```
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
```
8244256e

selftest: Rework BasicDeleteTests.test_all() into setUp() and a test · 46acd164

Andrew Bartlett authored May 28, 2018

This will allow running multiple tests against the same tree. This tree
is very similar to the tree produced by the KCC test that simply does a
tree_delete, and I want to lock down the tree_delete behaviour.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>

46acd164

25 May, 2018 29 commits

vfs_gpfs: Use full_path_tos instead of talloc_asprintf · 1531fcde

Christof Schmitt authored May 22, 2018 and

Jeremy Allison committed May 26, 2018



full_path_tos avoids the talloc call for most cases; use that instead of
talloc_asprintf.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat May 26 01:43:09 CEST 2018 on sn-devel-144

1531fcde

vfs_gpfs: Remove get_full_smb_filename from is_offline check · 1843a2d6
Christof Schmitt authored May 22, 2018 and Jeremy Allison committed May 25, 2018
```
No stream information is required here.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
```
1843a2d6

vfs_gpfs: Remove wrong get_full_smb_filename from ntimes function · 4a29a949

Christof Schmitt authored May 22, 2018 and

Jeremy Allison committed May 25, 2018



Updating the timestamps requires the path to the file, but no stream
information.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

4a29a949

vfs_gpfs: Adjust debug level when get_winattrs returns EBADF · 01a6aef5

Christof Schmitt authored May 23, 2018 and

Jeremy Allison committed May 25, 2018



This is returned for a get_winattrs call against a non-gpfs file system.
This can happen for the .. entry when listing a share on the file system
root.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

01a6aef5

smbd: Move dfree_info struct · 9a79a61a

Christof Schmitt authored May 16, 2018 and

Jeremy Allison committed May 25, 2018



As the struct is no longer used as part of connection_struct, move it to
dfree.c.

This is not backported, as it would change the VFS ABI.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

9a79a61a

smbd: Flush dfree memcache on service reload · e30d0c0e

Christof Schmitt authored May 18, 2018 and

Jeremy Allison committed May 25, 2018

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13446



Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

e30d0c0e

smbd: Cache dfree information based on query path · 8f121747

Christof Schmitt authored May 16, 2018 and

Jeremy Allison committed May 25, 2018

Sub directories in a SMB share can have different free space information
(e.g. when a different file system is mounted there). Caching the dfree
information per SMB share will return invalid data. Address this by
switching to memcache and store the cached data based on the query path.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13446



Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

8f121747

memcache: Add new cache type for dfree information · f5d05562

Christof Schmitt authored May 16, 2018 and

Jeremy Allison committed May 25, 2018

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13446



Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

f5d05562

selftest: Add test for 'dfree cache' · a55b3d2f

Christof Schmitt authored May 23, 2018 and

Jeremy Allison committed May 25, 2018

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13446



Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

a55b3d2f

selftest: Add dfq_cache share with 'dfree cache time' set · 7ffcbd5c

Christof Schmitt authored May 23, 2018 and

Jeremy Allison committed May 25, 2018

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13446



Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

7ffcbd5c

s3: vfs_fake_acls: Correctly implement the chmod/fchmod algorithm on fake acls. · 52778afd

Jeremy Allison authored May 18, 2018



We now pass samba3hide(nt4_dc), so remove it from knownfail.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri May 25 21:29:32 CEST 2018 on sn-devel-144

52778afd

s3: smbd: Make map_acl_perms_to_permset() extern. · e85a662e

Jeremy Allison authored May 18, 2018



The vfs_fake_acl module will need it to implement chown/fchown.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

e85a662e

s3: smbd: Make unix_perms_to_acl_perms() extern. · 109d94d1

Jeremy Allison authored May 18, 2018



The vfs_fake_acl module will need it to implement chown/fchown.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

109d94d1

s3: posix_acls: Remove unused 'connection_struct *conn' parameter to map_acl_perms_to_permset(). · a41155b0
Jeremy Allison authored May 18, 2018
```
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
```
a41155b0
s3: posix_acls: Remove dead functions fchmod_acl()/chmod_acl(). · cfbe3048
Jeremy Allison authored May 17, 2018
```
No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
```
cfbe3048

s3: VFS: Remove SMB_VFS_FCHMOD_ACL(). · 5bbb8310

Jeremy Allison authored May 17, 2018



No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

5bbb8310

s3: torture: Remove cmd_fchmod_acl(). · a3d7544b

Jeremy Allison authored May 17, 2018



No longer needed.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

a3d7544b

s3: modules: vfs_ceph: Remove FCHMOD_ACL in cephwrap_fchmod(). · 55f013ed

Jeremy Allison authored May 17, 2018



Now I understand the use of the mask in POSIX ACLs
this extra step is no longer needed.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

55f013ed

s3: modules: vfs_default: Remove FCHMOD_ACL in fchmod. · 0c65f6b8

Jeremy Allison authored May 17, 2018



Now I understand the use of the mask in POSIX ACLs
this extra step is no longer needed.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

0c65f6b8

s3: VFS: Remove SMB_VFS_CHMOD_ACL(). · 7b8fa17f

Jeremy Allison authored May 17, 2018



No longer used.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

7b8fa17f

s3: torture: Remove cmd_chmod_acl(). · 7d3059cd

Jeremy Allison authored May 17, 2018



No longer needed.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

7d3059cd

s3: modules: vfs_ceph: Remove CHMOD_ACL in cephwrap_chmod(). · b8477abb

Jeremy Allison authored May 17, 2018



Now I understand the use of the mask in POSIX ACLs
this extra step is no longer needed.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

b8477abb

s3: modules: vfs_ceph: Remove CHMOD_ACL in cephwrap_mkdir(). · d50bb460

Jeremy Allison authored May 17, 2018



Now I understand the use of the mask in POSIX ACLs
this extra step is no longer needed.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

d50bb460

s3: modules: vfs_default: Remove CHMOD_ACL in chmod. · c5060e1a

Jeremy Allison authored May 17, 2018



Now I understand the use of the mask in POSIX ACLs
this extra step is no longer needed.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

c5060e1a

s3: modules: vfs_default: Remove CHMOD_ACL in mkdir. · 90117f25

Jeremy Allison authored May 17, 2018



Now I understand the use of the mask in POSIX ACLs
this extra step is no longer needed. If the mkdir
succeeded it's already set the correct mode.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

90117f25

s3: smbd: Use FCHMOD call, not FCHMOD_ACL call if mode bits reset needed. · aaed6b4e

Jeremy Allison authored May 17, 2018



This is a behavior change, it will modify the POSIX ACL mask
from a value of rwx instead of modifying the existing ACE
entries to be ANDed with the passed in mode. However it
will have no effect on the underlying permissions, and
better reflects the proper use of POSIX ACLs (i.e. I
didn't understand the use of the mask entry in the
ACL when I first wrote the POSIX ACL code).

In addition, the vfs_acl_common.c module already
filters these calls for all but POSIX opens, which
means the only place this change is exposed to the
client would be a cifsfs unix extensions client doing
posix acl calls (and they would expect the mask to
be set like this on chmod).

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

aaed6b4e

s3: smbd: Optimization. Only do the FCHMOD_ACL call if mode bits not equal. · 2f4e581c
Jeremy Allison authored May 17, 2018
```
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
```
2f4e581c
s3: smbd: optimization. Only do the FCHMOD call if needed. · 4f6c71ae
Jeremy Allison authored May 17, 2018
```
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
```
4f6c71ae

s3: smbd: Remove use of SMB_VFS_FCHMOD_ACL() in overwrite case. · d03e9a86

Jeremy Allison authored May 17, 2018



We have potentially called SMB_VFS_FCHMOD() here in
the file_set_dosmode() call associated with the comment
/* Overwritten files should be initially set as archive */
at line 3755 above, so there is no need to do any POSIX ACL
mask protection.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

d03e9a86