Slow query on users

SELECT COUNT(*) FROM "users" WHERE ("users"."state" IN ('active')) AND ((id IN (SELECT "approvers"."user_id" FROM "approvers" WHERE "approvers"."target_id" = 1447032 AND "approvers"."target_type" = 'Project' AND ("approvers"."user_id" != 638197)) OR id IN (SELECT "members"."user_id" FROM "members" WHERE "members"."source_type" = 'Project' AND "members"."type" IN ('ProjectMember') AND "members"."source_id" = 1447032 AND "members"."source_type" = 'Project' AND "members"."type" IN ('ProjectMember') AND "members"."requested_at" IS NULL AND (access_level > 20)  ORDER BY "members"."id" DESC)) AND id NOT IN (SELECT "approvals"."user_id" FROM "approvals" WHERE "approvals"."merge_request_id" = 978692) AND id != 638197);

QUERY PLAN                                                                   
             
------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------
 Aggregate  (cost=54447.39..54447.40 rows=1 width=0) (actual time=577.395..577.395 rows=1 loops=1)
   ->  Seq Scan on users  (cost=1655.92..53756.02 rows=276551 width=0) (actual time=319.819..577.384 rows=5 loops=1)
         Filter: ((NOT (hashed SubPlan 3)) AND (id <> 638197) AND ((state)::text = 'active'::text) AND ((hashed SubPlan 1) OR (hashed SubPlan 2)))
         Rows Removed by Filter: 740211
         SubPlan 3
           ->  Seq Scan on approvals  (cost=0.00..1631.65 rows=1 width=4) (actual time=10.325..10.325 rows=0 loops=1)
                 Filter: (merge_request_id = 978692)
                 Rows Removed by Filter: 81810
         SubPlan 1
           ->  Index Scan using index_approvers_on_target_id_and_target_type on approvers  (cost=0.00..8.33 rows=1 width=4) (actual time=0.051..0.051 rows=0
 loops=1)
                 Index Cond: ((target_id = 1447032) AND ((target_type)::text = 'Project'::text))
                 Filter: (user_id <> 638197)
         SubPlan 2
           ->  Sort  (cost=15.92..15.93 rows=2 width=8) (actual time=0.103..0.105 rows=6 loops=1)
                 Sort Key: members.id
                 Sort Method: quicksort  Memory: 25kB
                 ->  Index Scan using index_members_on_source_id_and_source_type on members  (cost=0.00..15.91 rows=2 width=8) (actual time=0.037..0.061 row
s=6 loops=1)
                       Index Cond: ((source_id = 1447032) AND ((source_type)::text = 'Project'::text))
                       Filter: ((requested_at IS NULL) AND (access_level > 20) AND ((type)::text = 'ProjectMember'::text))
 Total runtime: 577.471 ms

Particularly because of the high filtered rows.

cc/ @yorickpeterse @pacoguzman @ahmadsherif