Skip to content

Add similar credit card counts to external pipeline validation service payload

Alex Buijs requested to merge pass-similar-cc-count-to-pvs into master

What does this MR do and why?

This MR adds the following counts to the payload of the external pipeline validation service:

  • similar credit cards
  • cards with similar holder names

Both queries do an index only scan: similar credit cards and similar holder names.

Issue: https://gitlab.com/gitlab-org/modelops/anti-abuse/team-tasks/-/issues/71

Database Migration

Up
main: == 20220829183356 ReplaceIndexOnCreditCardValidations: migrating ==============
main: -- transaction_open?()
main:    -> 0.0000s
main: -- index_exists?(:user_credit_card_validations, [:holder_name, :expiration_date, :last_digits, :credit_card_validated_at], {:name=>"index_user_credit_card_validations_meta_data_full_match", :algorithm=>:concurrently})
main:    -> 0.0049s
main: -- execute("SET statement_timeout TO 0")
main:    -> 0.0001s
main: -- remove_index(:user_credit_card_validations, {:name=>"index_user_credit_card_validations_meta_data_full_match", :algorithm=>:concurrently, :column=>[:holder_name, :expiration_date, :last_digits, :credit_card_validated_at]})
main:    -> 0.0031s
main: -- execute("RESET statement_timeout")
main:    -> 0.0002s
main: -- transaction_open?()
main:    -> 0.0000s
main: -- index_exists?(:user_credit_card_validations, "lower(holder_name), expiration_date, last_digits, credit_card_validated_at", {:name=>"index_user_credit_card_validations_meta_data_full_match", :algorithm=>:concurrently})
main:    -> 0.0011s
main: -- add_index(:user_credit_card_validations, "lower(holder_name), expiration_date, last_digits, credit_card_validated_at", {:name=>"index_user_credit_card_validations_meta_data_full_match", :algorithm=>:concurrently})
main:    -> 0.0012s
main: == 20220829183356 ReplaceIndexOnCreditCardValidations: migrated (0.0170s) =====

ci: == 20220829183356 ReplaceIndexOnCreditCardValidations: migrating ==============
ci: -- transaction_open?()
ci:    -> 0.0000s
ci: -- index_exists?(:user_credit_card_validations, [:holder_name, :expiration_date, :last_digits, :credit_card_validated_at], {:name=>"index_user_credit_card_validations_meta_data_full_match", :algorithm=>:concurrently})
ci:    -> 0.0023s
ci: -- execute("SET statement_timeout TO 0")
ci:    -> 0.0002s
ci: -- remove_index(:user_credit_card_validations, {:name=>"index_user_credit_card_validations_meta_data_full_match", :algorithm=>:concurrently, :column=>[:holder_name, :expiration_date, :last_digits, :credit_card_validated_at]})
ci:    -> 0.0036s
ci: -- execute("RESET statement_timeout")
ci:    -> 0.0002s
ci: -- transaction_open?()
ci:    -> 0.0000s
ci: -- index_exists?(:user_credit_card_validations, "lower(holder_name), expiration_date, last_digits, credit_card_validated_at", {:name=>"index_user_credit_card_validations_meta_data_full_match", :algorithm=>:concurrently})
ci:    -> 0.0010s
ci: -- add_index(:user_credit_card_validations, "lower(holder_name), expiration_date, last_digits, credit_card_validated_at", {:name=>"index_user_credit_card_validations_meta_data_full_match", :algorithm=>:concurrently})
ci:    -> 0.0015s
ci: == 20220829183356 ReplaceIndexOnCreditCardValidations: migrated (0.0119s) =====
Down
main: == 20220829183356 ReplaceIndexOnCreditCardValidations: reverting ==============
main: -- transaction_open?()
main:    -> 0.0000s
main: -- index_exists?(:user_credit_card_validations, "lower(holder_name), expiration_date, last_digits, credit_card_validated_at", {:name=>"index_user_credit_card_validations_meta_data_full_match", :algorithm=>:concurrently})
main:    -> 0.0042s
main: -- execute("SET statement_timeout TO 0")
main:    -> 0.0001s
main: -- remove_index(:user_credit_card_validations, {:name=>"index_user_credit_card_validations_meta_data_full_match", :algorithm=>:concurrently, :column=>"lower(holder_name), expiration_date, last_digits, credit_card_validated_at"})
main:    -> 0.0026s
main: -- execute("RESET statement_timeout")
main:    -> 0.0002s
main: -- transaction_open?()
main:    -> 0.0000s
main: -- index_exists?(:user_credit_card_validations, [:holder_name, :expiration_date, :last_digits, :credit_card_validated_at], {:name=>"index_user_credit_card_validations_meta_data_full_match", :algorithm=>:concurrently})
main:    -> 0.0010s
main: -- add_index(:user_credit_card_validations, [:holder_name, :expiration_date, :last_digits, :credit_card_validated_at], {:name=>"index_user_credit_card_validations_meta_data_full_match", :algorithm=>:concurrently})
main:    -> 0.0011s
main: == 20220829183356 ReplaceIndexOnCreditCardValidations: reverted (0.0148s) =====

ci: == 20220829183356 ReplaceIndexOnCreditCardValidations: reverting ==============
ci: -- transaction_open?()
ci:    -> 0.0000s
ci: -- index_exists?(:user_credit_card_validations, "lower(holder_name), expiration_date, last_digits, credit_card_validated_at", {:name=>"index_user_credit_card_validations_meta_data_full_match", :algorithm=>:concurrently})
ci:    -> 0.0039s
ci: -- execute("SET statement_timeout TO 0")
ci:    -> 0.0004s
ci: -- remove_index(:user_credit_card_validations, {:name=>"index_user_credit_card_validations_meta_data_full_match", :algorithm=>:concurrently, :column=>"lower(holder_name), expiration_date, last_digits, credit_card_validated_at"})
ci:    -> 0.0043s
ci: -- execute("RESET statement_timeout")
ci:    -> 0.0011s
ci: -- transaction_open?()
ci:    -> 0.0000s
ci: -- index_exists?(:user_credit_card_validations, [:holder_name, :expiration_date, :last_digits, :credit_card_validated_at], {:name=>"index_user_credit_card_validations_meta_data_full_match", :algorithm=>:concurrently})
ci:    -> 0.0030s
ci: -- add_index(:user_credit_card_validations, [:holder_name, :expiration_date, :last_digits, :credit_card_validated_at], {:name=>"index_user_credit_card_validations_meta_data_full_match", :algorithm=>:concurrently})
ci:    -> 0.0022s
ci: == 20220829183356 ReplaceIndexOnCreditCardValidations: reverted (0.0237s) =====

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Alex Buijs

Merge request reports