information: Information Theoretic Tools in R

The information package contains a suite of important information theoretic functions. Its includes entropy, relative entropy, mutual information, and other quantities. All functions are simple, lightweight, and rely on few dependencies. As such, they should be robust and easy to put into practice when applying information theoretic insights in R.

Installation

For now, information is available only on GitHub. To install it, run the following code in R:

devtools::install_github("jackobailey/information")

Example

All of the functions in information take probability distributions as inputs. Here is a simple example showing how to compute the entropy of 100 fair coin flips in bits.

# Load package
library(information)

# Set seed
set.seed(01)

# Simulate 100 fair coin flips
coin_flips <- 
  sample(
    c("H", "T"),
    size = 100,
    replace = T
    ) |> 
  table() |> 
  prop.table()

# Compute entropy
entropy(coin_flips)
#> [1] 0.9997114