Resolve "Trash (never delete anything)"

Closes #142 (closed)

• How Trash Works in Baserow Summary
• Technical Details on the this MR's implementation
• Reviewing Notes
• Testing Notes

How Trash Works in Baserow Summary

Groups, Applications (databases), Tables, Fields and Rows now when deleted in Baserow go to the trash bin. Inside the trash bin you can see trashed "items" grouped by Group and Application. Trashed "items" can be restored which causes them to re-appear in Baserow as if they had not been deleted. After 3 days (configured by the HOURS_UNTIL_TRASH_PERMANENTLY_DELETED env variable) a trashed item will be permanently deleted. Or the user can instead go into the trash for a particular Group or Application and "empty" the trash which permanently deletes all items inside of the Group or Application (including the group or app itself if it is also trashed).

Technical Details on the this MR's implementation

A new TrashEntry table has been added which contains information on each trashed item.

A new Mixin TrashableModelMixin is introduced which:

1. Adds a trashed boolean field
2. Overrides the objects manager so it no longer returns rows where trashed=True
3. Adds a trash manager which returns rows where trashed=False
4. Adds a objects_and_trash manager

Various new ParentXXXTrashableModelMixin have been introduced for models which:

1. We don't want them to be trashable on their own (think a group token, view sort etc)
2. However if their 'parent' object is trashed they should no longer appear in Baserow
3. This mixin overrides the objects/trash/objects_and_trash like the TrashableModelMixin, however it joins via a FK to a parent object and delegates to that objects trashed field.

A new registry with a new registrable type is introduced called "TrashableItemType". For a model to be trashable it must have a TrashableItemType registered and also mixin the TrashableModelMixin.

A new TrashHandler exists with the following public methods:

1. trash
2. restore
3. get_trash_structure
4. mark_old_trash_for_permanent_deletion
5. empty
6. permanently_delete_marked_trash
7. permanently_delete (replaces everywhere that was deleting things permanently before)
8. get_trash_contents
9. item_has_a_trashed_parent (used in handlers to ensure we never return something that should be trashed as it's parent is trashed)

When something is trashed this is what happens:

1. It's trashed column is set to true
   1. Any related objects are also set to trashed as defined by the TrashableItemType.get_items_to_trash impl.
2. A TrashEntry is created for the item
3. All api endpoints etc will stop returning the now trashed item due to overridden managers filtered out trashed=True
4. The existing deletion code which performed validation checks and sent signals to front end clients still run as normal.
5. The item is not actually deleted from the DB.

When something is restored this is what happens:

1. It's trashed column is set to false
   1. Any related objects are also untrashed
2. The TrashableItemType.item_restored method is triggered which sends the appropriate front end signals based on the item's type
   1. Signals are also send for related objects

When something is perm deleted:

1. Its TrashableItemType.permanently_delete method is called which does the actual delete
2. No signals are sent as they were sent when it was trashed

Reviewing Notes

The first commit contains all the changes only for the backend and only to trash groups: https://gitlab.com/bramw/baserow/-/merge_requests/271/diffs?commit_id=69e141effb7ca5c75f98b505c4fd642884746565

After which I incrementally added Applications being trashable in: https://gitlab.com/bramw/baserow/-/merge_requests/271/diffs?commit_id=db537b1dbd088578567a66e3b7a31055d18fdbda

And then Tables, Rows and Fields in: https://gitlab.com/bramw/baserow/-/merge_requests/271/diffs?commit_id=81f8c4deca449d741f3b93c4dac023f1e5667e97

After which the commit sizes should be smaller other than a few larger refactor + cleanup's that came later on.

Testing Notes

0. Lower the trash duration setting at the bottom of base.py and watch the export celery worker to see it perm deleting trash
1. fill a table with add_columns and
   1. Can delete rows which are filtered / sorted / hidden and restore them across multiple clients
   2. Can delete fields which have filters / sorts / hidden in other views and restore them across multiple clients
2. Check out the new performance test i've added for perm deleting rows found in backend/tests/baserow/performance/test_trash_performance.py . With the --runslow option turned on you can simply run the test to check performance!
3. Can still export tables with deleted rows and fields
4. Deleting a group will return 404's when say trying to access its table pages directly
5. Can't restore row/field/table/app before it's parent has been restored
6. Can export and import templates still
7. Emptying a group will remove apps from the trash structure + actually remove the tables, fields, etc from the database
8. Use the patch below to fill the database with trashed and not trashed tables/groups/apps, great for testing scroll behaviour, speed by running ./baserow trash_test_data once applied in intellij using apply patch from clipboard

Index: backend/src/baserow/contrib/database/management/commands/trash_test_data.py
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/backend/src/baserow/contrib/database/management/commands/trash_test_data.py b/backend/src/baserow/contrib/database/management/commands/trash_test_data.py
new file mode 100644
--- /dev/null	(date 1625572824069)
+++ b/backend/src/baserow/contrib/database/management/commands/trash_test_data.py	(date 1625572824069)
@@ -0,0 +1,70 @@
+import sys
+from decimal import Decimal
+from math import ceil
+
+from django.contrib.auth import get_user_model
+from django.core.management.base import BaseCommand
+from django.db.models import Max
+from faker import Faker
+
+from baserow.contrib.database.fields.field_helpers import (
+    construct_all_possible_field_kwargs,
+)
+from baserow.contrib.database.fields.handler import FieldHandler
+from baserow.contrib.database.rows.handler import RowHandler
+from baserow.contrib.database.table.handler import TableHandler
+from baserow.contrib.database.table.models import Table
+from baserow.core.handler import CoreHandler
+from baserow.core.trash.handler import TrashHandler
+
+User = get_user_model()
+
+
+class Command(BaseCommand):
+    help = "Fills baserow with hundreds of trashed groups apps and tables for testing."
+
+    def add_arguments(self, parser):
+        parser.add_argument(
+            "--limit",
+            action="store",
+            type=int,
+            help="Amount of trashed stuff that need to be inserted.",
+        )
+
+    def handle(self, *args, **options):
+        limit = options["limit"] or 2
+
+        user = User.objects.first()
+        core_handler = CoreHandler()
+        table_handler = TableHandler()
+        tables_inserted = 0
+        for i in range(limit):
+            groupuser = core_handler.create_group(user=user, name=f"Group {i}")
+            group = groupuser.group
+            for j in range(10):
+                database = core_handler.create_application(
+                    user=user,
+                    group=group,
+                    type_name="database",
+                    name=f"Database {i}-{j}",
+                )
+                for k in range(100):
+                    table = table_handler.create_table(
+                        user=user,
+                        database=database,
+                        fill_example=True,
+                        name=f"Table {i}-{j}-{k}",
+                    )
+                    if k % 2 == 0:
+                        TrashHandler.trash(user, group, database, table)
+                    if tables_inserted % 20 == 0:
+                        self.stdout.write(
+                            self.style.SUCCESS(f"{tables_inserted} tables inserted")
+                        )
+                    tables_inserted += 1
+                if j % 2 == 0:
+                    TrashHandler.trash(user, group, database, database)
+            if i % 2 == 0:
+                TrashHandler.trash(user, group, None, group)
+
+        self.stdout.write(self.style.SUCCESS(f"{limit} things have been inserted."))