Skip to content

Log all GC stats when measurements are enabled

Matthias Käppler requested to merge 199103-full-gc-stats into master

What does this MR do?

In the import rake task, we currently print three GC stats when measurements are enabled:

  • total GCs
  • minor GCs
  • major GCs

I think we should log all GC stats instead; it's not that many so not overly noisy, and there are some very useful data points in there that we're currently throwing away.

I also think we should do a full mark+sweep before we run the program being instrumented. That way we prevent unrelated pending work from skewing results.

Finally I changed the output to be JSON instead. That prints more nicely, and is a proper DIF so we can copy and paste that and run e.g. jq over it. This is also better if for future iterations we may want to start collecting this data somewhere and e.g. plot changes over time.

References #199103 (closed)

New output:

{
  "count": {
    "before": 62,
    "after": 119,
    "diff": 57
  },
  "heap_allocated_pages": {
    "before": 5385,
    "after": 6759,
    "diff": 1374
  },
  "heap_sorted_length": {
    "before": 5385,
    "after": 7747,
    "diff": 2362
  },
  "heap_allocatable_pages": {
    "before": 0,
    "after": 988,
    "diff": 988
  },
  "heap_available_slots": {
    "before": 2194914,
    "after": 2754949,
    "diff": 560035
  },
  "heap_live_slots": {
    "before": 1395652,
    "after": 1802787,
    "diff": 407135
  },
  "heap_free_slots": {
    "before": 799262,
    "after": 952162,
    "diff": 152900
  },
  "heap_final_slots": {
    "before": 0,
    "after": 0,
    "diff": 0
  },
  "heap_marked_slots": {
    "before": 1395651,
    "after": 1572974,
    "diff": 177323
  },
  "heap_eden_pages": {
    "before": 5385,
    "after": 6759,
    "diff": 1374
  },
  "heap_tomb_pages": {
    "before": 0,
    "after": 0,
    "diff": 0
  },
  "total_allocated_pages": {
    "before": 5385,
    "after": 6759,
    "diff": 1374
  },
  "total_freed_pages": {
    "before": 0,
    "after": 0,
    "diff": 0
  },
  "total_allocated_objects": {
    "before": 8763739,
    "after": 15896223,
    "diff": 7132484
  },
  "total_freed_objects": {
    "before": 7368087,
    "after": 14093436,
    "diff": 6725349
  },
  "malloc_increase_bytes": {
    "before": 863272,
    "after": 13993400,
    "diff": 13130128
  },
  "malloc_increase_bytes_limit": {
    "before": 31581162,
    "after": 33554432,
    "diff": 1973270
  },
  "minor_gc_count": {
    "before": 47,
    "after": 99,
    "diff": 52
  },
  "major_gc_count": {
    "before": 15,
    "after": 20,
    "diff": 5
  },
  "remembered_wb_unprotected_objects": {
    "before": 17305,
    "after": 23571,
    "diff": 6266
  },
  "remembered_wb_unprotected_objects_limit": {
    "before": 34610,
    "after": 38326,
    "diff": 3716
  },
  "old_objects": {
    "before": 1351772,
    "after": 1542391,
    "diff": 190619
  },
  "old_objects_limit": {
    "before": 2703544,
    "after": 3015792,
    "diff": 312248
  },
  "oldmalloc_increase_bytes": {
    "before": 863272,
    "after": 14182296,
    "diff": 13319024
  },
  "oldmalloc_increase_bytes_limit": {
    "before": 45373937,
    "after": 104306648,
    "diff": 58932711
  }
}

Does this MR meet the acceptance criteria?

Conformity

Edited by 🤖 GitLab Bot 🤖

Merge request reports