B

benchmarking

Python toolkit for benchmarking RAG quality on Salesforce Data Cloud. Includes retrieval metrics (Hit Rate, MRR, NDCG), generation evaluation, and optional Claude Code skills.

ValidationOS is an open-source, local-first AI assurance platform that transforms AI performance into proven results. It provides tools for validating, optimizing, certifying, and monitoring AI models across edge hardware environments through automated testing, compatibility analysis, performance benchmarking, observability, and compliance workflows.

Automated LLM Benchmarking on GPU - tokens/sec, latency percentiles, VRAM profiling, multi-format support (HuggingFace, GGUF, GPTQ)

https://ayinedjimi-consultants.fr

A minimalistic and portable unit testing and benchmarking framework for C.

PTA 2026. Task Benchmarks

Benchmarks data processing tools

MultiNativQA is Multilingual Native question-answering (QA) dataset consisting of 64k QA pairs in seven extremely low to high resource languages, covering 18 different topics from nine different regions. Paper: https://arxiv.org/pdf/2407.09823. Project: https://nativqa.gitlab.io

Benchmarked columnar vs row-oriented databases using the Yahoo Cloud Server Benchmark (YCSB). Compared throughput, latency, and error rates under different workloads. Analyzed trade-offs between NoSQL (Cassandra) and relational (PostgreSQL) databases with supporting graphs and configuration files.

This C++ sample project uses meson to manage builds and already has auto-formatting, testing, linting, benchmarking, code coverage, static analysis and documentation generation.

Benchmarking framework for machine learning with fNIRS

https://doi.org/10.3389/fnrgo.2023.994969

QAPerf is a tool for evaluating the performance of quality assessment (QA) methods. It automates the generation of standard reports, including correlation tables, leave-one-group-out correlation boxplots, and other benchmarking visualizations. Designed for researchers and practitioners, QAPerf streamlines the analysis of QA metrics, ensuring reliable and reproducible evaluations.

Command-line benchmarking tool. Runs commands, times them, ranks them. Also reports memory used, context switches, and other metrics.

Server fibonacci benchmarks in three different technologies: node-js, vapor, php

A Live Evaluation of Computational Methods for Metagenome Investigation

SecurityPerf is a tool designed for benchmarking production workloads. In doing so, it makes measuring the impact of security programs on production workloads easy.

VTmark results, Ansible roles and other scripts.

Pew Pew is a simple HTTPS benchmarking tool written in Go.