The class of latent Markov models, including hidden Markov models, hidden semi-Markov models, state space models, and point processes, is a very popular and powerful framework for inference of time series driven by latent processes. Furthermore, all these models can be fitted using direct numerical maximum likelihood estimation using the so-called forward algorithm as discussed in Zucchini et al. (2016) <doi:10.1201/b20790>. However, due to their great flexibility, researchers using these models in applied work often need to build highly customized models for which standard software implementation is lacking, or the construction of such models in said software is as complicated as writing fully tailored 'R' code. While providing greater flexibility and control, the latter suffers from slow estimation speeds that make custom solutions inconvenient. We address the above issues in two ways. First, standard blocks of code, common to all these model classes, are implemented as simple-to-use functions that can be added like Lego blocks to an otherwise fully custom likelihood function, making writing custom code much easier. Second, under the hood, these functions are written in 'C++', allowing for 10-20 times faster evaluation time, and thus drastically speeding up model estimation. To aid in building fully custom likelihood functions, several vignettes are included that show how to simulate data from and estimate all the above model classes.