R package citation, R package reverse dependencies, R package scholars, install an r package from GitHub hy is package acceptance pending why is package undeliverable amazon why is package on hold dhl tour packages why in r package r and r package full form why is r free why r is bad which r package to install which r package has which r package which r package version which r package readxl which r package ggplot which r package fread which r package license where is package.json where is package-lock.json where is package.swift where is package explorer in eclipse where is package where is package manager unity where is package installer android where is package manager console in visual studio who r package which r package to install which r package version who is package who is package deal who is package design r and r package full form r and r package meaning what r package has what package r what is package in java what is package what is package-lock.json what is package in python what is package.json what is package installer do r package can't install r packages r can't find package r can't load package can't load xlsx package r can't install psych package r can't install sf package r Write if else in NONMEM pk pd
sentencepiece
View on CRAN: Click
here
Download and install sentencepiece package within the R console
Install from CRAN:
install.packages("sentencepiece")
Install from Github:
library("remotes")
install_github("cran/sentencepiece")
Install by package version:
library("remotes")
install_version("sentencepiece", "0.2.3")
Attach the package and use:
library("sentencepiece")
Maintained by
Jan Wijffels
[Scholar Profile | Author Map]
[Scholar Profile | Author Map]
All associated links for this package
https://github.com/google/sentencepiece . 10.32614/CRAN.package.sentencepiece . https://github.com/bnosac/sentencepiece . sentencepiece results . sentencepiece.pdf . sentencepiece_0.2.3.tar.gz . sentencepiece_0.2.3.zip . sentencepiece_0.2.3.zip . sentencepiece_0.2.3.zip . sentencepiece_0.2.3.tgz . sentencepiece_0.2.3.tgz . sentencepiece_0.2.3.tgz . sentencepiece_0.2.3.tgz . sentencepiece archive . https://CRAN.R-project.org/package=sentencepiece .
First Published: 2020-06-04
Latest Update: 2022-11-13
Description:
Unsupervised text tokenizer allowing to perform byte pair encoding and unigram modelling.
Wraps the 'sentencepiece' library which provides a language independent tokenizer to split text in words and smaller subword units.
The techniques are explained in the paper "SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing" by Taku Kudo and John Richardson (2018) .
Provides as well straightforward access to pretrained byte pair encoding models and subword embeddings trained on Wikipedia using 'word2vec',
as described in "BPEmb: Tokenization-free Pre-trained Subword Embeddings in 275 Languages" by Benjamin Heinzerling and Michael Strube (2018) .
How to cite:
Jan Wijffels (2020). sentencepiece: Text Tokenization using Byte Pair Encoding and Unigram Modelling. R package version 0.2.3, https://cran.r-project.org/web/packages/sentencepiece. Accessed 12 May. 2025.
Previous versions and publish date:
Other packages that cited sentencepiece R package
View sentencepiece citation profile
Other R packages that sentencepiece depends,
imports, suggests or enhances
Complete documentation for sentencepiece
Functions, R codes and Examples using
the sentencepiece R package
Some associated functions: BPEembed . BPEembedder . predict.BPEembed . read_word2vec . sentencepiece . sentencepiece_decode . sentencepiece_download_model . sentencepiece_encode . sentencepiece_load_model . txt_remove_ . wordpiece_encode .
Some associated R codes: AAA.R . RcppExports.R . bpemb.R . pkg.R . sentencepiece.R . utils.R . word2vec.R . wordpiece.R . Full sentencepiece package functions and examples
Downloads during the last 30 days
Today's Hot Picks in Authors and Packages
pander
Contains some functions catching all messages, 'stdout' and other
useful information while evaluati ...
Download / Learn more Package Citations See dependency
Download / Learn more Package Citations See dependency
Maintainer: Gergely Daróczi (view profile)
epinet
A collection of epidemic/network-related tools. Simulates transmission of diseases through contact n ...
Download / Learn more Package Citations See dependency
Download / Learn more Package Citations See dependency
Maintainer: Chris Groendyke (view profile)
nextGenShinyApps
Nove responsive tools for designing and developing 'Shiny' dashboards and applications. The scripts ...
Download / Learn more Package Citations See dependency
Download / Learn more Package Citations See dependency
Maintainer: Obinna Obianom (view profile)
LatticeKrig
Methods for the interpolation of large spatial
datasets. This package follows a "fixed rank Kriging ...
Download / Learn more Package Citations See dependency
Download / Learn more Package Citations See dependency
Maintainer: Douglas Nychka (view profile)
errorist
Provides environment hooks that obtain errors and warnings which
occur during the execution of code ...
Download / Learn more Package Citations See dependency
Download / Learn more Package Citations See dependency
Maintainer: James Balamuta (view profile)
sGMRFmix
An implementation of sparse Gaussian Markov random field mixtures
presented by Ide et al. (2016) ...
Download / Learn more Package Citations See dependency
Download / Learn more Package Citations See dependency
Maintainer: Koji Makiyama (view profile)
24,223
R Packages
207,311
Dependencies
65,402
Author Associations
24,224
Publication Badges
