Other packages > Find by keyword >

crawlee  

Tidy Interface for Reproducible Web Crawling
View on CRAN: Click here


Download and install crawlee package within the R console
Install from CRAN:
install.packages("crawlee")

Install from Github:
library("remotes")
install_github("cran/crawlee")

Install by package version:
library("remotes")
install_version("crawlee", "0.1.0")



Attach the package and use:
library("crawlee")
Maintained by
Andre Leite
[Scholar Profile | Author Map]
All associated links for this package
First Published: 2026-07-03
Latest Update: 2026-07-03
Description:
A tidy, pipe-friendly toolkit for reproducible web crawling and structured data collection, inspired by the architecture of the 'Crawlee' library. Provides a unified crawler with a deduplicating, resumable request queue, content-type aware handlers, structured storage backends and rich console logging via 'cli'. Supports crawling HTML pages, sitemaps, RSS and Atom feeds and PDF documents, with optional headless-browser rendering and helpers for retrieval-augmented generation.
How to cite:
Andre Leite (2026). crawlee: Tidy Interface for Reproducible Web Crawling. R package version 0.1.0, https://cran.r-project.org/web/packages/crawlee. Accessed 24 Oct. 2026.
Previous versions and publish date:
(2026-09-30 23:21), 0.1.0 (2026-07-03 14:10)
Other packages that cited crawlee R package
View crawlee citation profile
Other R packages that crawlee depends, imports, suggests or enhances
Complete documentation for crawlee
Functions, R codes and Examples using the crawlee R package
Full crawlee package functions and examples
Downloads during the last 30 days

Today's Hot Picks in Authors and Packages

gemlog  
File Conversion for "Gem Infrasound Logger"
Reads data files from the 'Gem infrasound logger' for analysis and converts to segy format (which is ...
Download / Learn more Package Citations See dependency  
climwin  
Climate Window Analysis
Contains functions to detect and visualise periods of climate sensitivity (climate windows) for a g ...
Download / Learn more Package Citations See dependency  
forecastHybrid  
Convenient Functions for Ensemble Time Series Forecasts
Convenient functions for ensemble forecasts in R combining approaches from the 'forecast' package. ...
Download / Learn more Package Citations See dependency  
FDRsampsize  
Compute Sample Size that Meets Requirements for Average Power and FDR
Defines a collection of functions to compute average power and sample size for studies that use the ...
Download / Learn more Package Citations See dependency  
givitiR  
The GiViTI Calibration Test and Belt
Functions to assess the calibration of logistic regression models with the GiViTI (Gruppo Italiano p ...
Download / Learn more Package Citations See dependency  
quickcode  
Quick and Essential 'R' Tricks for Better Scripts
The NOT functions, 'R' tricks and a compilation of some simple quick plus often used 'R' codes to im ...
Download / Learn more Package Citations See dependency  

29,080

R Packages

247,686

Dependencies

77,281

Author Associations

29,081

Publication Badges

© Copyright since 2022. All right reserved, rpkg.net.  Based in Cambridge, Massachusetts, USA