LfsLinkService executes very large query
Follow-up from gitlab-com/gl-infra/production#1327 (comment 240909991):
The LfsLinkService has no bounds in terms of textual query size. This led to the linked incident above.
An example for the query pattern is:
SELECT "lfs_objects"."id" FROM "lfs_objects" WHERE "lfs_objects"."oid" IN (/* 30K oids */) ORDER BY "lfs_objects"."id" ASC LIMIT 1;
This originates from the any? call, however we also retrieve without a limit later.
This leads to problems parsing the query on the postgres side given its size. In this case (a project import) it also led to statement timeouts, clogging up the postgres logs (as the query is logged).
The idea here is to change this to batch the oids to find the ones that already exist.
Edited by Andreas Brandl