Unify logs rendering code path (+ improve performance)
Right now there are essentially 2 different code paths that render logs on the job details page. First is inside the django templates and second one is inside javascript incremental logs.
However, this is suboptimal because it requires maintaining two separate code paths. Instead merge it in to one code path inside javascript.
For initial logs already present on the server use the json_script
filter provided by Django. It will convert the logs in to the JSON
object that can later be parsed by the javascript.
Move the action link list rendering to javascript meaning the
_pipeline_actions.html
template is no longer needed.
This improves performance because there will be less load on the server to render templates and reduces the required data transfer because logs will be compressed in to compact JSON instead of HTML.
Apache Bench 20 seconds:
Before:
Benchmarking localhost (be patient)
Finished 35 requests
Server Software: gunicorn
Server Hostname: localhost
Server Port: 37899
Document Path: /scheduler/job/4801718
Document Length: 1409743 bytes
Concurrency Level: 1
Time taken for tests: 20.020 seconds
Complete requests: 35
Failed requests: 0
Total transferred: 49356615 bytes
HTML transferred: 49341005 bytes
Requests per second: 1.75 [#/sec] (mean)
Time per request: 571.989 [ms] (mean)
Time per request: 571.989 [ms] (mean, across all concurrent requests)
Transfer rate: 2407.63 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 533 572 34.7 577 739
Waiting: 532 571 34.7 576 739
Total: 533 572 34.7 577 739
Percentage of the requests served within a certain time (ms)
50% 576
66% 581
75% 582
80% 584
90% 587
95% 602
98% 739
99% 739
100% 739 (longest request)
After:
Benchmarking localhost (be patient)
Finished 49 requests
Server Software: gunicorn
Server Hostname: localhost
Server Port: 37899
Document Path: /scheduler/job/4801718
Document Length: 1152380 bytes
Concurrency Level: 1
Time taken for tests: 20.373 seconds
Complete requests: 49
Failed requests: 0
Total transferred: 56488474 bytes
HTML transferred: 56466620 bytes
Requests per second: 2.41 [#/sec] (mean)
Time per request: 415.783 [ms] (mean)
Time per request: 415.783 [ms] (mean, across all concurrent requests)
Transfer rate: 2707.68 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 397 416 42.0 402 610
Waiting: 396 415 42.0 402 610
Total: 397 416 42.0 402 610
Percentage of the requests served within a certain time (ms)
50% 402
66% 405
75% 412
80% 424
90% 433
95% 444
98% 610
99% 610
100% 610 (longest request)
As you can see after the changes the requests are on average 150 milliseconds faster and 257 kB smaller.