measure

Overview

measure extends tidymodels with preprocessing steps for analytical measurement data such as spectroscopy, chromatography, and other instrument-generated signals. It provides a recipes-style interface for common spectral preprocessing techniques.

measure helps you:

• Convert measurement data from wide or long formats into an internal representation
• Preprocess spectra using techniques like smoothing, derivatives, and normalization
• Transform data back to wide or long format for modeling or visualization
• Handle multi-dimensional data like LC-DAD, EEM fluorescence, and 2D NMR with native nD support
• Decompose complex signals using PARAFAC, Tucker, and MCR-ALS methods

Installation

You can install the development version of measure from GitHub:

# install.packages("pak")
pak::pak("JamesHWade/measure")

Usage

The measure workflow follows the familiar recipes pattern: define a recipe, add steps, prep, and bake.

library(measure)
library(recipes)
library(ggplot2)

# NIR spectroscopy data for predicting meat composition
data(meats_long)
head(meats_long)
#> # A tibble: 6 × 6
#>      id water   fat protein channel transmittance
#>   <int> <dbl> <dbl>   <dbl>   <int>         <dbl>
#> 1     1  60.5  22.5    16.7       1          2.62
#> 2     1  60.5  22.5    16.7       2          2.62
#> 3     1  60.5  22.5    16.7       3          2.62
#> 4     1  60.5  22.5    16.7       4          2.62
#> 5     1  60.5  22.5    16.7       5          2.62
#> 6     1  60.5  22.5    16.7       6          2.62

Building a preprocessing recipe

rec <- recipe(water + fat + protein ~ ., data = meats_long) |>
  # Assign sample ID role (not used as predictor)
  update_role(id, new_role = "id") |>
  # Convert long-format measurements to internal representation
  step_measure_input_long(transmittance, location = vars(channel)) |>
  # Apply Savitzky-Golay smoothing with first derivative
  step_measure_savitzky_golay(window_side = 5, differentiation_order = 1) |>
  # Standard Normal Variate normalization
  step_measure_snv() |>
  # Convert back to wide format for modeling
  step_measure_output_wide(prefix = "nir_")

Preparing and applying the recipe

# Prep learns any parameters from training data
prepped <- prep(rec)
#> Dropping 1 list column for wide output: channel

# Bake applies the transformations
processed <- bake(prepped, new_data = NULL)

# Result is ready for modeling
processed[1:5, 1:8]
#> # A tibble: 5 × 8
#>      id water   fat protein  nir_01  nir_02  nir_03  nir_04
#>   <int> <dbl> <dbl>   <dbl>   <dbl>   <dbl>   <dbl>   <dbl>
#> 1     1  60.5  22.5    16.7 -0.126  -0.110  -0.0928 -0.0745
#> 2     2  46    40.1    13.5  0.0184  0.0381  0.0601  0.0841
#> 3     3  71     8.4    20.5  0.105   0.114   0.125   0.136 
#> 4     4  72.8   5.9    20.7  0.0716  0.0786  0.0871  0.0974
#> 5     5  58.3  25.5    15.5 -0.132  -0.118  -0.101  -0.0817

Visualizing the preprocessing

# Get data at intermediate step (before output conversion)
rec_for_viz <- recipe(water + fat + protein ~ ., data = meats_long) |>
update_role(id, new_role = "id") |>
  step_measure_input_long(transmittance, location = vars(channel)) |>
  step_measure_savitzky_golay(window_side = 5, differentiation_order = 1) |>
  step_measure_snv()

processed_long <- bake(prep(rec_for_viz), new_data = NULL)

# Extract and plot a few spectra
library(tidyr)
library(dplyr)

plot_data <- processed_long |>
  slice(1:10) |>
  mutate(sample_id = row_number()) |>
  unnest(.measures)

ggplot(plot_data, aes(x = location, y = value, group = sample_id, color = factor(sample_id))) +
  geom_line(alpha = 0.7) +
  labs(
    x = "Channel",
    y = "Preprocessed Signal",
    title = "NIR Spectra After Preprocessing",
    subtitle = "Savitzky-Golay first derivative + SNV normalization",
    color = "Sample"
  ) +
  theme_minimal() +
  theme(legend.position = "none")

Available Steps

Input/Output Steps

Step	Description
step_measure_input_wide()	Convert wide format (measurements in columns) to internal format
step_measure_input_long()	Convert long format (measurements in rows) to internal format
step_measure_output_wide()	Convert back to wide format for modeling
step_measure_output_long()	Convert back to long format

Spectral Math

Step	Description
step_measure_absorbance()	Convert transmittance to absorbance
step_measure_transmittance()	Convert absorbance to transmittance
step_measure_log()	Log transformation with configurable base/offset
step_measure_kubelka_munk()	Kubelka-Munk transformation for reflectance
step_measure_derivative()	Simple finite difference derivatives
step_measure_derivative_gap()	Gap (Norris-Williams) derivatives

Filtering & Scatter Correction

Step	Description
step_measure_savitzky_golay()	Smoothing and/or differentiation
step_measure_snv()	Standard Normal Variate normalization
step_measure_msc()	Multiplicative Scatter Correction
step_measure_emsc()	Extended MSC with wavelength-dependent correction
step_measure_osc()	Orthogonal Signal Correction

Smoothing & Noise Reduction

Step	Description
step_measure_smooth_ma()	Moving average smoothing
step_measure_smooth_median()	Median filter (robust to spikes)
step_measure_smooth_gaussian()	Gaussian kernel smoothing
step_measure_smooth_wavelet()	Wavelet denoising
step_measure_filter_fourier()	Fourier low-pass/high-pass filtering
step_measure_despike()	Spike/outlier detection and removal

Sample-wise Normalization

Step	Description
step_measure_normalize_sum()	Divide by sum (total intensity)
step_measure_normalize_max()	Divide by maximum value
step_measure_normalize_range()	Scale to 0-1 range
step_measure_normalize_vector()	L2/Euclidean normalization
step_measure_normalize_auc()	Divide by area under curve
step_measure_normalize_peak()	Normalize by peak region (tunable)

Variable-wise Scaling

Step	Description
step_measure_center()	Mean centering
step_measure_scale_auto()	Auto-scaling (z-score)
step_measure_scale_pareto()	Pareto scaling
step_measure_scale_range()	Range scaling
step_measure_scale_vast()	VAST scaling

Baseline Correction

Step	Description
step_measure_baseline_als()	Asymmetric least squares
step_measure_baseline_poly()	Polynomial baseline fitting
step_measure_baseline_rf()	Rolling ball/LOESS baseline
step_measure_baseline_rolling()	Rolling ball algorithm
step_measure_baseline_airpls()	Adaptive Iteratively Reweighted PLS
step_measure_baseline_arpls()	Asymmetrically Reweighted PLS
step_measure_baseline_snip()	SNIP (Statistics-sensitive Non-linear Iterative Peak-clipping)
step_measure_baseline_tophat()	Top-hat morphological filter
step_measure_baseline_morph()	Iterative morphological correction
step_measure_baseline_minima()	Local minima interpolation
step_measure_baseline_auto()	Automatic method selection
step_measure_detrend()	Polynomial detrending

Reference Corrections

Step	Description
step_measure_subtract_blank()	Blank/background subtraction
step_measure_subtract_reference()	Reference spectrum subtraction
step_measure_ratio_reference()	Reference ratio with optional blank

Region Operations

Step	Description
step_measure_trim()	Keep measurements within specified x-range
step_measure_exclude()	Remove measurements within specified range(s)
step_measure_resample()	Interpolate to new regular grid

Alignment & Registration

Step	Description
step_measure_align_shift()	Cross-correlation shift alignment
step_measure_align_reference()	Align to external reference spectrum
step_measure_align_dtw()	Dynamic Time Warping alignment
step_measure_align_ptw()	Parametric Time Warping
step_measure_align_cow()	Correlation Optimized Warping (tunable)

Quality Control

Step	Description
step_measure_qc_snr()	Calculate signal-to-noise ratio
step_measure_qc_saturated()	Detect saturated measurements
step_measure_qc_outlier()	Detect outlier samples
step_measure_impute()	Interpolate missing values

Peak Operations

Step	Description
step_measure_peaks_detect()	Detect peaks using prominence or derivative methods
step_measure_peaks_integrate()	Calculate peak areas
step_measure_peaks_filter()	Filter peaks by height, area, or count
step_measure_peaks_deconvolve()	Deconvolve overlapping peaks
step_measure_peaks_to_table()	Convert peaks to wide format for modeling

SEC/GPC Analysis

Step	Description
step_measure_mw_averages()	Calculate Mn, Mw, Mz, Mp, and dispersity
step_measure_mw_distribution()	Generate molecular weight distribution curve
step_measure_mw_fractions()	Calculate molecular weight fractions

Feature Engineering

Step	Description
step_measure_integrals()	Calculate integrated areas for specified regions
step_measure_ratios()	Calculate ratios between integrated regions
step_measure_moments()	Calculate statistical moments from spectra
step_measure_bin()	Reduce spectrum to fewer points via binning

Data Augmentation

Step	Description
step_measure_augment_noise()	Add random noise for training augmentation
step_measure_augment_shift()	Random x-axis shifts for shift invariance
step_measure_augment_scale()	Random intensity scaling

Drift & Batch Correction

Step/Function	Description
step_measure_drift_qc_loess()	QC-RLSC drift correction using LOESS
step_measure_drift_linear()	Linear drift correction
step_measure_drift_spline()	Spline-based drift correction
step_measure_qc_bracket()	QC bracketing interpolation
step_measure_batch_reference()	Reference-based batch correction
measure_detect_drift()	Detect significant drift in QC samples

Analytical Validation Functions

measure provides a comprehensive suite of functions for analytical method validation, designed for compatibility with ICH Q2(R2), ISO 17025, and similar regulatory frameworks.

Calibration & Quantitation

Function	Description
measure_calibration_fit()	Fit weighted calibration curves (linear/quadratic)
measure_calibration_predict()	Predict concentrations with uncertainty
measure_calibration_verify()	Continuing calibration verification
measure_lod() / measure_loq()	Detection and quantitation limits

Precision & Accuracy

Function	Description
measure_repeatability()	Within-run precision
measure_intermediate_precision()	Between-run precision with variance components
measure_reproducibility()	Between-lab precision
measure_gage_rr()	Gage R&R / Measurement System Analysis
measure_accuracy()	Bias, recovery, and accuracy assessment
measure_linearity()	Linearity with lack-of-fit testing
measure_carryover()	Carryover evaluation

Method Comparison

Function	Description
measure_bland_altman()	Bland-Altman analysis with limits of agreement
measure_deming_regression()	Deming regression for method comparison
measure_passing_bablok()	Passing-Bablok non-parametric regression
measure_proficiency_score()	z-scores, En scores, zeta scores for PT

Matrix Effects & Sample Prep QC

Function/Step	Description
measure_matrix_effect()	Quantify ion suppression/enhancement
step_measure_standard_addition()	Standard addition correction
step_measure_dilution_correct()	Back-calculate diluted concentrations
step_measure_surrogate_recovery()	Surrogate/internal standard recovery

Uncertainty & Quality Control

Function	Description
measure_uncertainty_budget()	ISO GUM uncertainty budgets
measure_uncertainty()	Combined and expanded uncertainty
measure_control_limits()	Shewhart, EWMA, or CUSUM limits
measure_control_chart()	Westgard multi-rule control charts
measure_system_suitability()	System suitability testing

Criteria System

Function	Description
measure_criteria()	Define acceptance criteria
measure_assess()	Evaluate data against criteria
criteria_bioanalytical()	FDA/EMA bioanalytical presets
criteria_ich_q2()	ICH Q2 validation presets

Learning more

• Getting Started - A comprehensive introduction to measure
• Preprocessing Techniques - Deep dive into available preprocessing methods
• Analytical Validation - Calibration, uncertainty, and method validation

Datasets

Included Datasets

The package includes datasets for examples and testing:

Dataset	Technique	Samples	Description
meats_long	NIR	215	NIR transmittance spectra of meat samples (from modeldata)
bioreactors_small	Raman	210	Raman spectra from 15 small-scale bioreactors
bioreactors_large	Raman	42	Raman spectra from 3 large-scale bioreactors
hplc_chromatograms	HPLC-UV	20	Simulated HPLC chromatograms with 5 compounds
sec_chromatograms	SEC/GPC	10	Simulated SEC chromatograms (5 standards + 5 polymers)
sec_calibration	SEC/GPC	5	Calibration standards for molecular weight curves
maldi_spectra	MALDI-TOF	16	Simulated mass spectra (4 groups × 4 replicates)

# Load datasets
data(meats_long)
data(glucose_bioreactors)  # loads bioreactors_small and bioreactors_large
data(hplc_chromatograms)
data(sec_chromatograms)
data(sec_calibration)
data(maldi_spectra)

External Data Sources

For additional test data beyond what’s included with measure, these sources provide publicly available analytical measurement data:

R Packages with Spectral Data:

Package	Dataset	Technique	Description
modeldata	meats	NIR	Meat composition (wide format version)
prospectr	NIRsoil	NIR	Soil analysis with 825 samples
ChemoSpec	Various	IR, NMR	Multiple spectroscopy datasets
hyperSpec	Various	Raman, IR	Hyperspectral data examples

# Example: Load NIRsoil from prospectr
# install.packages("prospectr")
data(NIRsoil, package = "prospectr")

Online Repositories:

• Mendeley Data - Search “spectroscopy”, “chromatography”, or “mass spectrometry”
• Zenodo - Open science data repository
• Kaggle Datasets - Community-contributed datasets
• NIST Chemistry WebBook - Reference spectra (IR, MS, UV-Vis)
• SDBS - Spectral Database for Organic Compounds (NMR, IR, MS)

Domain-Specific Databases:

Database	Data Type	URL
MassBank	Mass spectra	https://massbank.eu/MassBank/
HMDB	NMR, MS metabolomics	https://hmdb.ca/
NMRShiftDB	NMR spectra	https://nmrshiftdb.nmr.uni-koeln.de/
Crystallography Open Database	XRD patterns	https://www.crystallography.net/cod/

Related packages

measure builds on the tidymodels ecosystem:

• recipes - The foundation for preprocessing pipelines
• parsnip - Unified modeling interface
• workflows - Bundle preprocessing and modeling
• tune - Hyperparameter tuning (works with measure’s tunable steps!)

For spectral analysis in R, you might also find these packages useful:

• prospectr - Spectral preprocessing functions
• ChemoSpec - Exploratory chemometrics
• mdatools - Multivariate data analysis

Contributing

This package is under active development. Contributions are welcome! Please see the contributing guidelines.

Code of Conduct

Please note that the measure project is released with a Contributor Code of Conduct. By contributing to this project, you agree to abide by its terms.