Other packages > Find by keyword >

evaluatellm  

Statistical Inference for Language Model Evaluations
View on CRAN: Click here


Download and install evaluatellm package within the R console
Install from CRAN:
install.packages("evaluatellm")

Install from Github:
library("remotes")
install_github("cran/evaluatellm")

Install by package version:
library("remotes")
install_version("evaluatellm", "0.1.0")



Attach the package and use:
library("evaluatellm")
Maintained by
Charles Coverdale
[Scholar Profile | Author Map]
All associated links for this package
First Published: 2026-09-15
Latest Update: 2026-09-15
Description:
Treats language model evaluations as statistical experiments and supplies the inference they require. Provides central limit theorem and cluster-robust standard errors for evaluation scores, paired and unpaired model comparisons, variance decomposition when several responses are drawn per question, control-variate variance reduction, multiplicity adjustment across benchmark suites, and power and minimum detectable effect calculations for planning evaluations, following Miller (2024) <doi:10.48550/arXiv.2411.00640>. For evaluations scored by a model judge, implements agreement statistics against a human gold standard and prediction-powered inference (Angelopoulos et al. 2023) <doi:10.1126/science.adi6000> with the power-tuned estimator of Angelopoulos, Bates and Jordan (2023) <doi:10.48550/arXiv.2311.01453>, so a small set of human labels debiases a large set of judge scores. Leaderboards are supported through bootstrap rank intervals and Bradley-Terry ratings (Bradley and Terry 1952) <doi:10.2307/2334029>. Accepts scores from any evaluation harness.
How to cite:
Charles Coverdale (2026). evaluatellm: Statistical Inference for Language Model Evaluations. R package version 0.1.0, https://cran.r-project.org/web/packages/evaluatellm. Accessed 25 Oct. 2026.
Previous versions and publish date:
No previous versions
Other packages that cited evaluatellm R package
View evaluatellm citation profile
Other R packages that evaluatellm depends, imports, suggests or enhances
Complete documentation for evaluatellm
Functions, R codes and Examples using the evaluatellm R package
Full evaluatellm package functions and examples
Downloads during the last 30 days

Today's Hot Picks in Authors and Packages

imputeTS  
Time Series Missing Value Imputation
Imputation (replacement) of missing values in univariate time series. Offers several imputation fu ...
Download / Learn more Package Citations See dependency  
RcmdrPlugin.aRnova  
R Commander Plug-in for Repeated-Measures ANOVA
R Commander plug-in for repeated-measures and mixed-design ('split-plot') ANOVA. It adds a new men ...
Download / Learn more Package Citations See dependency  
CTT  
Classical Test Theory Functions
A collection of common test and item analyses from a classical test theory (CTT) framework. Analyses ...
Download / Learn more Package Citations See dependency  
nlstools  
Tools for Nonlinear Regression Analysis
Several tools for assessing the quality of fit of a gaussian nonlinear model are provided. ...
Download / Learn more Package Citations See dependency  
quickcode  
Quick and Essential 'R' Tricks for Better Scripts
The NOT functions, 'R' tricks and a compilation of some simple quick plus often used 'R' codes to im ...
Download / Learn more Package Citations See dependency  
fstcore  
R Bindings to the 'Fstlib' Library
The 'fstlib' library provides multithreaded serialization of compressed data frames using the 'fst' ...
Download / Learn more Package Citations See dependency  

29,218

R Packages

247,686

Dependencies

77,281

Author Associations

29,219

Publication Badges

© Copyright since 2022. All right reserved, rpkg.net.  Based in Cambridge, Massachusetts, USA