Other packages > Find by keyword >

evaluatellm  

Statistical Inference for Language Model Evaluations
View on CRAN: Click here


Download and install evaluatellm package within the R console
Install from CRAN:
install.packages("evaluatellm")

Install from Github:
library("remotes")
install_github("cran/evaluatellm")

Install by package version:
library("remotes")
install_version("evaluatellm", "0.1.0")



Attach the package and use:
library("evaluatellm")
Maintained by
Charles Coverdale
[Scholar Profile | Author Map]
All associated links for this package
First Published: 2026-09-15
Latest Update: 2026-09-15
Description:
Treats language model evaluations as statistical experiments and supplies the inference they require. Provides central limit theorem and cluster-robust standard errors for evaluation scores, paired and unpaired model comparisons, variance decomposition when several responses are drawn per question, control-variate variance reduction, multiplicity adjustment across benchmark suites, and power and minimum detectable effect calculations for planning evaluations, following Miller (2024) <doi:10.48550/arXiv.2411.00640>. For evaluations scored by a model judge, implements agreement statistics against a human gold standard and prediction-powered inference (Angelopoulos et al. 2023) <doi:10.1126/science.adi6000> with the power-tuned estimator of Angelopoulos, Bates and Jordan (2023) <doi:10.48550/arXiv.2311.01453>, so a small set of human labels debiases a large set of judge scores. Leaderboards are supported through bootstrap rank intervals and Bradley-Terry ratings (Bradley and Terry 1952) <doi:10.2307/2334029>. Accepts scores from any evaluation harness.
How to cite:
Charles Coverdale (2026). evaluatellm: Statistical Inference for Language Model Evaluations. R package version 0.1.0, https://cran.r-project.org/web/packages/evaluatellm. Accessed 04 Oct. 2026.
Previous versions and publish date:
No previous versions
Other packages that cited evaluatellm R package
View evaluatellm citation profile
Other R packages that evaluatellm depends, imports, suggests or enhances
Complete documentation for evaluatellm
Functions, R codes and Examples using the evaluatellm R package
Full evaluatellm package functions and examples
Downloads during the last 30 days

Today's Hot Picks in Authors and Packages

skilljaR  
Connect to Your 'Skilljar' Data
Functions that simplify calls to the 'Skilljar' API. See < ...
Download / Learn more Package Citations See dependency  
r2resize  
In-Text Resize for Images, Tables and Fancy Resize Containers in 'shiny', 'rmarkdown' and 'quarto' Documents
Automatic resizing toolbar for containers, images and tables. Various resizable or expandable contai ...
Download / Learn more Package Citations See dependency  
plotbb  
Grammar of Graphics for 'base' Plot
Proof of concept for implementing grammar of graphics using base plot. The bbplot() function initial ...
Download / Learn more Package Citations See dependency  
rWishart  
Random Wishart Matrix Generation
An expansion of R's 'stats' random wishart matrix generation. This package allows the user to gener ...
Download / Learn more Package Citations See dependency  

28,905

R Packages

247,686

Dependencies

76,495

Author Associations

28,906

Publication Badges

© Copyright since 2022. All right reserved, rpkg.net.  Based in Cambridge, Massachusetts, USA