Other packages > Find by keyword >

ff  

Memory-Efficient Storage of Large Data on Disk and Fast Access Functions
View on CRAN: Click here


Download and install ff package within the R console
Install from CRAN:
install.packages("ff")

Install from Github:
library("remotes")
install_github("cran/ff")

Install by package version:
library("remotes")
install_version("ff", "4.5.2")



Attach the package and use:
library("ff")
Maintained by
Jens Oehlschlägel
[Scholar Profile | Author Map]
First Published: 2007-11-03
Latest Update: 2024-01-12
Description:
The ff package provides data structures that are stored on disk but behave (almost) as if they were in RAM by transparently mapping only a section (pagesize) in main memory - the effective virtual memory consumption per ff object. ff supports R's standard atomic data types 'double', 'logical', 'raw' and 'integer' and non-standard atomic types boolean (1 bit), quad (2 bit unsigned), nibble (4 bit unsigned), byte (1 byte signed with NAs), ubyte (1 byte unsigned), short (2 byte signed with NAs), ushort (2 byte unsigned), single (4 byte float with NAs). For example 'quad' allows efficient storage of genomic data as an 'A','T','G','C' factor. The unsigned types support 'circular' arithmetic. There is also support for close-to-atomic types 'factor', 'ordered', 'POSIXct', 'Date' and custom close-to-atomic types. ff not only has native C-support for vectors, matrices and arrays with flexible dimorder (major column-order, major row-order and generalizations for arrays). There is also a ffdf class not unlike data.frames and import/export filters for csv files. ff objects store raw data in binary flat files in native encoding, and complement this with metadata stored in R as physical and virtual attributes. ff objects have well-defined hybrid copying semantics, which gives rise to certain performance improvements through virtualization. ff objects can be stored and reopened across R sessions. ff files can be shared by multiple ff R objects (using different data en/de-coding schemes) in the same process or from multiple R processes to exploit parallelism. A wide choice of finalizer options allows to work with 'permanent' files as well as creating/removing 'temporary' ff files completely transparent to the user. On certain OS/Filesystem combinations, creating the ff files works without notable delay thanks to using sparse file allocation. Several access optimization techniques such as Hybrid Index Preprocessing and Virtualization are implemented to achieve good performance even with large datasets, for example virtual matrix transpose without touching a single byte on disk. Further, to reduce disk I/O, 'logicals' and non-standard data types get stored native and compact on binary flat files i.e. logicals take up exactly 2 bits to represent TRUE, FALSE and NA. Beyond basic access functions, the ff package also provides compatibility functions that facilitate writing code for ff and ram objects and support for batch processing on ff objects (e.g. as.ram, as.ff, ffapply). ff interfaces closely with functionality from package 'bit': chunked looping, fast bit operations and coercions between different objects that can store subscript information ('bit', 'bitwhich', ff 'boolean', ri range index, hi hybrid index). This allows to work interactively with selections of large datasets and quickly modify selection criteria. Further high-performance enhancements can be made available upon request.
How to cite:
Jens Oehlschlägel (2007). ff: Memory-Efficient Storage of Large Data on Disk and Fast Access Functions. R package version 4.5.2, https://cran.r-project.org/web/packages/ff. Accessed 10 Apr. 2025.
Previous versions and publish date:
1.0-1 (2007-11-03 19:34), 2.0.0 (2008-08-03 23:06), 2.0.1 (2009-04-16 08:57), 2.1-0 (2009-11-03 22:01), 2.1-1 (2009-11-05 13:34), 2.1-2 (2010-01-20 14:11), 2.2-0 (2010-09-30 07:42), 2.2-1 (2010-10-05 10:41), 2.2-2 (2011-04-24 21:28), 2.2-3 (2011-07-25 11:06), 2.2-4 (2012-01-15 18:01), 2.2-5 (2012-01-27 22:44), 2.2-6 (2012-03-31 07:20), 2.2-7 (2012-05-08 21:23), 2.2-9 (2012-10-24 14:23), 2.2-10 (2012-11-06 11:30), 2.2-11 (2013-03-12 07:59), 2.2-12 (2013-10-16 15:48), 2.2-13 (2014-04-09 09:54), 2.2-14.2 (2020-04-16 14:10), 2.2-14 (2018-05-15 23:19), 4.0.2 (2020-07-30 13:31), 4.0.4 (2020-10-13 20:20), 4.0.5 (2021-10-29 16:10), 4.0.7 (2022-05-06 18:10), 4.0.9 (2023-01-25 20:30), 4.0.12 (2024-01-12 16:10), 4.5.0 (2024-09-20 17:20)
Other packages that cited ff R package
View ff citation profile
Other R packages that ff depends, imports, suggests or enhances
Complete documentation for ff
Functions, R codes and Examples using the ff R package
Some associated functions:
Some associated R codes: CFUN.R . array.R . as.ff.R . bigsample.R . ff.R . ffapply.R . ffbit.R . ffcsv.R . ffdf.R . fffactor.R . ffreturn.R . ffsave.R . fileresize.R . fileutil.R . generics.R . getpagesize.R . hi.R . ordermerge.R . util.R . vmode.R . vt.R . zzz.R .  Full ff package functions and examples
Downloads during the last 30 days
03/1103/1203/1303/1403/1503/1603/1703/1803/1903/2003/2103/2203/2303/2403/2503/2603/2703/2803/2903/3003/3104/0104/0204/0304/0404/0504/0604/0704/0804/09Downloads for ff10001100120013001400150016001700TrendBars

Today's Hot Picks in Authors and Packages

RODBC  
ODBC Database Access
webglobe  
3D Interactive Globes
Displays geospatial data on an interactive 3D globe in the web browser. ...
Download / Learn more Package Citations See dependency  
xtermStyle  
Terminal Text Formatting Using Escape Sequences
Can be used for coloring output in terminals. It was developed for the standard Ubuntu terminal but ...
Download / Learn more Package Citations See dependency  
r2resize  
In-Text Resize for Images, Tables and Fancy Resize Containers in 'shiny', 'rmarkdown' and 'quarto' Documents
Automatic resizing toolbar for containers, images and tables. Various resizable or expandable contai ...
Download / Learn more Package Citations See dependency  
GRNNs  
General Regression Neural Networks Package
This General Regression Neural Networks Package uses various distance functions. It was motivated b ...
Download / Learn more Package Citations See dependency  
txtq  
A Small Message Queue for Parallel Processes
This queue is a data structure that lets parallel processes send and receive messages, and it can he ...
Download / Learn more Package Citations See dependency  

24,012

R Packages

207,311

Dependencies

64,867

Author Associations

24,013

Publication Badges

© Copyright since 2022. All right reserved, rpkg.net.  Based in Cambridge, Massachusetts, USA