Write Arkose verification rate to redis stream

This MR adds the behavior of writing the arkose verification rate to Redis. A future MR will use this to detech verification rate anomalies.

Each 4-hour window maintains success/failure counters; when the window rolls over, the code computes the previous bucket’s verification rate and appends it to arkose:token_verification:vrates so it can later be used to compute a baseline.

NOTE: This MR needs to come after this one: !211653 (merged)

Resolves: https://gitlab.com/gitlab-org/gitlab/-/work_items/578300

Verification Script

# rails c

Feature.enable(:track_arkose_token_verification_results, :instance)

mod = AntiAbuse::IdentityVerification::ArkoseFailOpen

success_prefix = mod::COUNTER_SUCCESS_KEY_PREFIX
failure_prefix = mod::COUNTER_FAILURE_KEY_PREFIX
bucket_hours   = mod::BUCKET_DURATION_HOURS
bucket_secs    = mod::BUCKET_DURATION_SECONDS
stream_key     = mod::VERIFICATION_RATE_STREAM_KEY
min_attempts   = mod::MIN_ATTEMPTS_FOR_EVALUATION

def bucket_id(at:, bucket_hours:)
  "#{at.to_date.strftime('%Y%m%d')}-#{at.hour / bucket_hours}"
end

now       = Time.zone.now
prev_time = now - bucket_secs
prev_id   = bucket_id(at: prev_time, bucket_hours: bucket_hours)

succ_key  = "#{success_prefix}#{prev_id}"
fail_key  = "#{failure_prefix}#{prev_id}"

# Clean slate
Gitlab::Redis::SharedState.with { |r| r.del(stream_key, succ_key, fail_key) }

# ---- Seed previous bucket with >= 400 attempts ----
seed_success = 360
seed_failure =  60
raise "Seed total < #{min_attempts}" if (seed_success + seed_failure) < min_attempts

Gitlab::Redis::SharedState.with do |r|
  r.setex(succ_key, bucket_secs, seed_success)
  r.setex(fail_key, bucket_secs, seed_failure)
end

expected_vrate = (100.0 * seed_success.to_f / (seed_success + seed_failure)).round(6)

puts "\nSeeded previous bucket #{prev_id}:"
puts "  success=#{seed_success} key=#{succ_key}"
puts "  failure=#{seed_failure} key=#{fail_key}"
puts "  expected_vrate=#{expected_vrate}%"
puts "  stream_key=#{stream_key}"

# ---- Append previous bucket's vrate ----
mod.send(:track_previous_window_verification_rate!)

# ---- Verify last stream entry ----
entry = Gitlab::Redis::SharedState.with { |r| r.xrevrange(stream_key, '+', '-', count: 1) }&.first
abort "\n❌ No vrate entry was appended to stream #{stream_key}" if entry.nil?

entry_id, fields = entry
vrate_str  = fields['vrate']
bucket_str = fields['bucket']

puts "\nLast stream entry on #{stream_key}:"
puts "  entry_id: #{entry_id}"
puts "  fields:   #{fields.inspect}"

actual_vrate = vrate_str.to_f

puts "\nExpected VRATE: #{expected_vrate}%"
puts "Actual   VRATE: #{actual_vrate}%"
Edited by Ajay Thomas

Merge request reports

Loading