Draft: Integrate gatekeeper (part 2) - cleanup
Removes the app package. Requires !1543 (closed) to be merged first (this MR also contains commits from !1543 (closed)).
I split into part 1/2 so the part 1 can be reviewed more easily since part 2 just deletes the packages/app directory and regenerates yarn.lock