From 9c127e5a8165d18949f5d1963ef4d2569ce54be1 Mon Sep 17 00:00:00 2001 From: "Documenter.jl" Date: Tue, 31 Oct 2023 17:51:11 +0000 Subject: [PATCH] build based on 5abf0d6 --- dev/.documenter-siteinfo.json | 2 +- dev/alt_features/index.html | 2 +- dev/api/index.html | 6 +++--- dev/background/index.html | 2 +- dev/debugging/index.html | 2 +- dev/formulas/index.html | 2 +- dev/index.html | 2 +- dev/roadmap/index.html | 2 +- dev/tuto_builtin/index.html | 2 +- dev/tuto_custom/index.html | 6 +++--- 10 files changed, 14 insertions(+), 14 deletions(-) diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json index f835dd54..c73f81f3 100644 --- a/dev/.documenter-siteinfo.json +++ b/dev/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.9.3","generation_timestamp":"2023-10-30T15:19:23","documenter_version":"1.1.2"}} \ No newline at end of file +{"documenter":{"julia_version":"1.9.3","generation_timestamp":"2023-10-31T17:51:06","documenter_version":"1.1.2"}} \ No newline at end of file diff --git a/dev/alt_features/index.html b/dev/alt_features/index.html index 5fefdc99..562fe3d0 100644 --- a/dev/alt_features/index.html +++ b/dev/alt_features/index.html @@ -1,2 +1,2 @@ -Features · HiddenMarkovModels.jl
GitHub

Alternatives - features

We compare features among the following Julia packages:

We discard MarkovModels.jl because its focus is GPU computation. There are also more generic packages for probabilistic programming, which are able to perform MCMC or variational inference (eg. Turing.jl) but we leave those aside.

HMMs.jlHMMBase.jlHMMGradients.jl
AlgorithmsSim, FB, Vit, BWSim, FB, Vit, BWFB
Observation typesanythingNumber / Vectoranything
Observation distributionsDensityInterface.jlDistributions.jlmanual
Number typesanythingFloat64AbstractFloat
Priors / structurespossiblenopossible
Automatic differentiationyesnoyes
Multiple sequencesyesnoyes
Linear algebrayesyesno
Logarithmic probabilitieshalfwayhalfwayyes

Sim = Simulation, FB = Forward-Backward, Vit = Viterbi, BW = Baum-Welch

+Features · HiddenMarkovModels.jl
GitHub

Alternatives - features

We compare features among the following Julia packages:

We discard MarkovModels.jl because its focus is GPU computation. There are also more generic packages for probabilistic programming, which are able to perform MCMC or variational inference (eg. Turing.jl) but we leave those aside.

HMMs.jlHMMBase.jlHMMGradients.jl
AlgorithmsSim, FB, Vit, BWSim, FB, Vit, BWFB
Observation typesanythingNumber / Vectoranything
Observation distributionsDensityInterface.jlDistributions.jlmanual
Number typesanythingFloat64AbstractFloat
Priors / structurespossiblenopossible
Automatic differentiationyesnoyes
Multiple sequencesyesnoyes
Linear algebrayesyesno
Logarithmic probabilitieshalfwayhalfwayyes

Sim = Simulation, FB = Forward-Backward, Vit = Viterbi, BW = Baum-Welch

diff --git a/dev/api/index.html b/dev/api/index.html index 1d89b324..147ce73c 100644 --- a/dev/api/index.html +++ b/dev/api/index.html @@ -1,8 +1,8 @@ -API reference · HiddenMarkovModels.jl
GitHub

API reference

HiddenMarkovModels — Module
HiddenMarkovModels

A Julia package for HMM modeling, simulation, inference and learning.

The alias HMMs is exported for the package name.

source

Types

Markov chains

HiddenMarkovModels.AbstractMarkovChain — Type
AbstractMarkovChain

Abstract supertype for a Markov chain amenable to simulation, inference and learning.

Required interface

  • initial_distribution(mc)
  • transition_matrix(mc)
  • fit!(mc, init_count, trans_count) (optional)

Applicable methods

  • rand([rng,] mc, T)
  • logdensityof(mc, state_seq)
  • fit(mc, state_seq_or_seqs) (if fit! is implemented)
source
HiddenMarkovModels.MarkovChain — Type
MarkovChain <: AbstractMarkovChain

Basic implementation of a discrete-state Markov chain.

Fields

  • init::AbstractVector: initial state probabilities
  • trans::AbstractMatrix: state transition matrix
source

Hidden Markov Models

HiddenMarkovModels.AbstractHiddenMarkovModel — Type
AbstractHiddenMarkovModel <: AbstractMarkovChain

Abstract supertype for an HMM amenable to simulation, inference and learning.

Required interface

  • initial_distribution(hmm)
  • transition_matrix(hmm)
  • obs_distribution(hmm, i)
  • fit!(hmm, init_count, trans_count, obs_seq, state_marginals) (optional)

Applicable methods

  • rand([rng,] hmm, T)
  • logdensityof(hmm, obs_seq) / logdensityof(hmm, obs_seqs, nb_seqs)
  • forward(hmm, obs_seq) / forward(hmm, obs_seqs, nb_seqs)
  • viterbi(hmm, obs_seq) / viterbi(hmm, obs_seqs, nb_seqs)
  • forward_backward(hmm, obs_seq) / forward_backward(hmm, obs_seqs, nb_seqs)
  • baum_welch(hmm, obs_seq) / baum_welch(hmm, obs_seqs, nb_seqs) if fit! is implemented
source
HiddenMarkovModels.HiddenMarkovModel — Type
HiddenMarkovModel{D} <: AbstractHiddenMarkovModel

Basic implementation of an HMM.

Fields

  • init::AbstractVector: initial state probabilities
  • trans::AbstractMatrix: state transition matrix
  • dists::AbstractVector{D}: observation distributions
source
HiddenMarkovModels.PermutedHMM — Type
PermutedHMM{H<:AbstractHMM}

Wrapper around an AbstractHMM that permutes its states.

This is computationally inefficient and mostly useful for evaluation.

Fields

  • hmm:H: the old HMM
  • perm::Vector{Int}: a permutation such that state i in the new HMM corresponds to state perm[i] in the old.
source

Basics

Base.rand — Function
rand([rng=default_rng(),] mc::AbstractMarkovChain, T)

Simulate mc for T time steps with a specified rng.

source
Base.length — Function
length(mc::AbstractMarkovChain)

Return the number of states of model.

source
HiddenMarkovModels.obs_distribution — Function
obs_distribution(hmm::AbstractHMM, i)

Return the observation distribution of hmm associated with state i.

The returned object dist must implement

  • rand(rng, dist)
  • DensityInterface.logdensityof(dist, x)
source

Inference

DensityInterface.logdensityof — Function
logdensityof(mc, state_seq)

Compute the loglikelihood of a single state sequence for a Markov chain.

source
logdensityof(hmm, obs_seq)

Apply the forward algorithm to compute the loglikelihood of a single observation sequence for an HMM.

Return a number.

source
logdensityof(hmm, obs_seqs, nb_seqs)

Apply the forward algorithm to compute the total loglikelihood of multiple observation sequences for an HMM.

Return a number.

Multithreading

This function is parallelized across sequences.

source
HiddenMarkovModels.forward — Function
forward(hmm, obs_seq)

Apply the forward algorithm to an HMM.

Return a tuple (α, logL) where

  • logL is the loglikelihood of the sequence
  • α[i] is the posterior probability of state i at the end of the sequence.
source
forward(hmm, obs_seqs, nb_seqs)

Apply the forward algorithm to an HMM, based on multiple observation sequences.

Return a vector of tuples (αₖ, logLₖ), where

  • logLₖ is the loglikelihood of sequence k
  • αₖ[i] is the posterior probability of state i at the end of sequence k
Multithreading

This function is parallelized across sequences.

source
HiddenMarkovModels.viterbi — Function
viterbi(hmm, obs_seq)

Apply the Viterbi algorithm to compute the most likely state sequence of an HMM.

Return a vector of integers.

source
viterbi(hmm, obs_seqs, nb_seqs)

Apply the Viterbi algorithm to compute the most likely state sequences of an HMM, based on multiple observation sequences.

Return a vector of vectors of integers.

Multithreading

This function is parallelized across sequences.

source
HiddenMarkovModels.forward_backward — Function
forward_backward(hmm, obs_seq)

Apply the forward-backward algorithm to estimate the posterior state marginals of an HMM.

Return a ForwardBackwardStorage.

source
forward_backward(hmm, obs_seqs, nb_seqs)

Apply the forward-backward algorithm to estimate the posterior state marginals of an HMM, based on multiple observation sequences.

Return a vector of ForwardBackwardStorage objects.

Multithreading

This function is parallelized across sequences.

source

Learning

StatsAPI.fit! — Function
fit!(mc::MC, init_count, trans_count)

Update mc in-place based on information generated from a state sequence.

source
fit!(hmm::HMM, init_count, trans_count, obs_seq, state_marginals)

Update hmm in-place based on information generated during forward-backward.

source
StatsAPI.fit — Function
fit(mc, state_seq_or_seqs)

Fit a Markov chain of the same type as mc to one or several state sequence(s).

Beware that mc must be an actual object of type MarkovChain, and not the type itself as is usually done eg. in Distributions.jl.

source
HiddenMarkovModels.baum_welch — Function
baum_welch(
+API reference · HiddenMarkovModels.jl
GitHub
API reference
HiddenMarkovModels — Module
HiddenMarkovModels
A Julia package for HMM modeling, simulation, inference and learning.
The alias HMMs is exported for the package name.
source
Types
Markov chains
HiddenMarkovModels.AbstractMarkovChain — Type
AbstractMarkovChain
Abstract supertype for a Markov chain amenable to simulation, inference and learning.
Required interface
  • initial_distribution(mc)
  • transition_matrix(mc)
  • fit!(mc, init_count, trans_count) (optional)
Applicable methods
  • rand([rng,] mc, T)
  • logdensityof(mc, state_seq)
  • fit(mc, state_seq_or_seqs) (if fit! is implemented)
source
HiddenMarkovModels.MarkovChain — Type
MarkovChain <: AbstractMarkovChain
Basic implementation of a discrete-state Markov chain.
Fields
  • init::AbstractVector: initial state probabilities
  • trans::AbstractMatrix: state transition matrix
source
Hidden Markov Models
HiddenMarkovModels.AbstractHiddenMarkovModel — Type
AbstractHiddenMarkovModel <: AbstractMarkovChain
Abstract supertype for an HMM amenable to simulation, inference and learning.
Required interface
  • initial_distribution(hmm)
  • transition_matrix(hmm)
  • obs_distribution(hmm, i)
  • fit!(hmm, init_count, trans_count, obs_seq, state_marginals) (optional)
Applicable methods
  • rand([rng,] hmm, T)
  • logdensityof(hmm, obs_seq) / logdensityof(hmm, obs_seqs, nb_seqs)
  • forward(hmm, obs_seq) / forward(hmm, obs_seqs, nb_seqs)
  • viterbi(hmm, obs_seq) / viterbi(hmm, obs_seqs, nb_seqs)
  • forward_backward(hmm, obs_seq) / forward_backward(hmm, obs_seqs, nb_seqs)
  • baum_welch(hmm, obs_seq) / baum_welch(hmm, obs_seqs, nb_seqs) if fit! is implemented
source
HiddenMarkovModels.HiddenMarkovModel — Type
HiddenMarkovModel{D} <: AbstractHiddenMarkovModel
Basic implementation of an HMM.
Fields
  • init::AbstractVector: initial state probabilities
  • trans::AbstractMatrix: state transition matrix
  • dists::AbstractVector{D}: observation distributions
source
HiddenMarkovModels.PermutedHMM — Type
PermutedHMM{H<:AbstractHMM}
Wrapper around an AbstractHMM that permutes its states.
This is computationally inefficient and mostly useful for evaluation.
Fields
  • hmm:H: the old HMM
  • perm::Vector{Int}: a permutation such that state i in the new HMM corresponds to state perm[i] in the old.
source
Basics
Base.rand — Function
rand([rng=default_rng(),] mc::AbstractMarkovChain, T)
Simulate mc for T time steps with a specified rng.
source
Base.length — Function
length(mc::AbstractMarkovChain)
Return the number of states of model.
source
HiddenMarkovModels.obs_distribution — Function
obs_distribution(hmm::AbstractHMM, i)
Return the observation distribution of hmm associated with state i.
The returned object dist must implement
  • rand(rng, dist)
  • DensityInterface.logdensityof(dist, x)
source
Inference
DensityInterface.logdensityof — Function
logdensityof(mc, state_seq)
Compute the loglikelihood of a single state sequence for a Markov chain.
source
logdensityof(hmm, obs_seq)
Apply the forward algorithm to compute the loglikelihood of a single observation sequence for an HMM.
Return a number.
source
logdensityof(hmm, obs_seqs, nb_seqs)
Apply the forward algorithm to compute the total loglikelihood of multiple observation sequences for an HMM.
Return a number.
Multithreading
This function is parallelized across sequences.
source
HiddenMarkovModels.forward — Function
forward(hmm, obs_seq)
Apply the forward algorithm to an HMM.
Return a tuple (α, logL) where
  • logL is the loglikelihood of the sequence
  • α[i] is the posterior probability of state i at the end of the sequence.
source
forward(hmm, obs_seqs, nb_seqs)
Apply the forward algorithm to an HMM, based on multiple observation sequences.
Return a vector of tuples (αₖ, logLₖ), where
  • logLₖ is the loglikelihood of sequence k
  • αₖ[i] is the posterior probability of state i at the end of sequence k
Multithreading
This function is parallelized across sequences.
source
HiddenMarkovModels.viterbi — Function
viterbi(hmm, obs_seq)
Apply the Viterbi algorithm to compute the most likely state sequence of an HMM.
Return a vector of integers.
source
viterbi(hmm, obs_seqs, nb_seqs)
Apply the Viterbi algorithm to compute the most likely state sequences of an HMM, based on multiple observation sequences.
Return a vector of vectors of integers.
Multithreading
This function is parallelized across sequences.
source
HiddenMarkovModels.forward_backward — Function
forward_backward(hmm, obs_seq)
Apply the forward-backward algorithm to estimate the posterior state marginals of an HMM.
Return a ForwardBackwardStorage.
source
forward_backward(hmm, obs_seqs, nb_seqs)
Apply the forward-backward algorithm to estimate the posterior state marginals of an HMM, based on multiple observation sequences.
Return a vector of ForwardBackwardStorage objects.
Multithreading
This function is parallelized across sequences.
source
Learning
StatsAPI.fit! — Function
fit!(mc::MC, init_count, trans_count)
Update mc in-place based on information generated from a state sequence.
source
fit!(hmm::HMM, init_count, trans_count, obs_seq, state_marginals)
Update hmm in-place based on information generated during forward-backward.
source
StatsAPI.fit — Function
fit(mc, state_seq_or_seqs)
Fit a Markov chain of the same type as mc to one or several state sequence(s).
Beware that mc must be an actual object of type MarkovChain, and not the type itself as is usually done eg. in Distributions.jl.
source
HiddenMarkovModels.baum_welch — Function
baum_welch(
     hmm_init, obs_seq;
     atol, max_iterations, check_loglikelihood_increasing
-)
Apply the Baum-Welch algorithm to estimate the parameters of an HMM starting from hmm_init.
Return a tuple (hmm_est, logL_evolution).
Keyword arguments
  • atol: Minimum loglikelihood increase at an iteration of the algorithm (otherwise the algorithm is deemed to have converged)
  • max_iterations: Maximum number of iterations of the algorithm
  • check_loglikelihood_increasing: Whether to throw an error if the loglikelihood decreases
source
baum_welch(
+)
Apply the Baum-Welch algorithm to estimate the parameters of an HMM starting from hmm_init.
Return a tuple (hmm_est, logL_evolution).
Keyword arguments
  • atol: Minimum loglikelihood increase at an iteration of the algorithm (otherwise the algorithm is deemed to have converged)
  • max_iterations: Maximum number of iterations of the algorithm
  • check_loglikelihood_increasing: Whether to throw an error if the loglikelihood decreases
source
baum_welch(
     hmm_init, obs_seqs, nb_seqs;
     atol, max_iterations, check_loglikelihood_increasing
-)
Apply the Baum-Welch algorithm to estimate the parameters of an HMM starting from hmm_init, based on nb_seqs observation sequences.
Return a tuple (hmm_est, logL_evolution).
Multithreading
This function is parallelized across sequences.
Keyword arguments
  • atol: Minimum loglikelihood increase at an iteration of the algorithm (otherwise the algorithm is deemed to have converged)
  • max_iterations: Maximum number of iterations of the algorithm
  • check_loglikelihood_increasing: Whether to throw an error if the loglikelihood decreases
source
Internals
HiddenMarkovModels.ForwardBackwardStorage — Type
ForwardBackwardStorage{R}
Store forward-backward quantities with element type R.
Fields
Let X denote the vector of hidden states and Y denote the vector of observations. The following fields are part of the API:
  • γ::Matrix{R}: posterior one-state marginals γ[i,t] = ℙ(X[t]=i | Y[1:T])
  • ξ::Array{R,3}: posterior two-state marginals ξ[i,j,t] = ℙ(X[t:t+1]=(i,j) | Y[1:T])
The following fields are internals and subject to change:
  • α::Matrix{R}: scaled forward variables α[i,t] proportional to ℙ(Y[1:t], X[t]=i) (up to a function of t)
  • β::Matrix{R}: scaled backward variables β[i,t] proportional to ℙ(Y[t+1:T] | X[t]=i) (up to a function of t)
  • c::Vector{R}: forward variable inverse normalizations c[t] = 1 / sum(α[:,t])
  • logm::Vector{R}: maximum of the observation loglikelihoods logm[t] = maximum(logB[:, t])
  • Bscaled::Matrix{R}: numerically stabilized observation likelihoods Bscaled[i,t] = exp.(logB[i,t] - logm[t])
  • Bβscaled::Matrix{R}: numerically stabilized product Bβscaled[i,t] = Bscaled[i,t] * β[i,t]
source
HiddenMarkovModels.fit_element_from_sequence! — Function
fit_element_from_sequence!(dists, i, x, w)
Modify the i-th element of dists by fitting it to an observation sequence x with associated weight sequence w.
The default behavior is a fallback on StatsAPI.fit!, which users are encouraged to implement if their observation distributions are mutable. If this is not possible, please override fit_element_from_sequence! directly.
source
HiddenMarkovModels.LightDiagNormal — Type
LightDiagNormal
An HMMs-compatible implementation of a multivariate normal distribution with diagonal covariance, enabling allocation-free estimation.
This is not part of the public API and is expected to change.
source
Notations
Integers
  • N: number of states
  • D: dimension of the observations
  • T: trajectory length
  • K: number of trajectories
Models and simulations
  • p or init: initial_distribution (vector of state probabilities)
  • A or trans: transition_matrix (matrix of transition probabilities)
  • dists: observation distribution (vector of rand-able and logdensityof-able objects)
  • state_seq: a sequence of states (vector of integers)
  • obs_seq: a sequence of observations (vector of individual observations)
  • obs_seqs: several sequences of observations
Forward backward
  • (log)b: vector of observation (log)likelihoods by state for an individual observation
  • (log)B: matrix of observation (log)likelihoods by state for a sequence of observations
  • α: scaled forward variables
  • β: scaled backward variables
  • γ: one-state marginals
  • ξ: two-state marginals
  • logL: loglikelihood of a sequence of observations
Index

+)

Apply the Baum-Welch algorithm to estimate the parameters of an HMM starting from hmm_init, based on nb_seqs observation sequences.

Return a tuple (hmm_est, logL_evolution).

Multithreading

This function is parallelized across sequences.

Keyword arguments

  • atol: Minimum loglikelihood increase at an iteration of the algorithm (otherwise the algorithm is deemed to have converged)
  • max_iterations: Maximum number of iterations of the algorithm
  • check_loglikelihood_increasing: Whether to throw an error if the loglikelihood decreases
source

Internals

HiddenMarkovModels.ForwardBackwardStorage — Type
ForwardBackwardStorage{R}

Store forward-backward quantities with element type R.

Fields

Let X denote the vector of hidden states and Y denote the vector of observations. The following fields are part of the API:

  • γ::Matrix{R}: posterior one-state marginals γ[i,t] = ℙ(X[t]=i | Y[1:T])
  • ξ::Array{R,3}: posterior two-state marginals ξ[i,j,t] = ℙ(X[t:t+1]=(i,j) | Y[1:T])

The following fields are internals and subject to change:

  • α::Matrix{R}: scaled forward variables α[i,t] proportional to ℙ(Y[1:t], X[t]=i) (up to a function of t)
  • β::Matrix{R}: scaled backward variables β[i,t] proportional to ℙ(Y[t+1:T] | X[t]=i) (up to a function of t)
  • c::Vector{R}: forward variable inverse normalizations c[t] = 1 / sum(α[:,t])
  • logm::Vector{R}: maximum of the observation loglikelihoods logm[t] = maximum(logB[:, t])
  • Bscaled::Matrix{R}: numerically stabilized observation likelihoods Bscaled[i,t] = exp.(logB[i,t] - logm[t])
  • Bβscaled::Matrix{R}: numerically stabilized product Bβscaled[i,t] = Bscaled[i,t] * β[i,t]
source
HiddenMarkovModels.fit_element_from_sequence! — Function
fit_element_from_sequence!(dists, i, x, w)

Modify the i-th element of dists by fitting it to an observation sequence x with associated weight sequence w.

The default behavior is a fallback on StatsAPI.fit!, which users are encouraged to implement if their observation distributions are mutable. If this is not possible, please override fit_element_from_sequence! directly.

source
HiddenMarkovModels.LightDiagNormal — Type
LightDiagNormal

An HMMs-compatible implementation of a multivariate normal distribution with diagonal covariance, enabling allocation-free estimation.

This is not part of the public API and is expected to change.

source

Notations

Integers

  • N: number of states
  • D: dimension of the observations
  • T: trajectory length
  • K: number of trajectories

Models and simulations

  • p or init: initial_distribution (vector of state probabilities)
  • A or trans: transition_matrix (matrix of transition probabilities)
  • dists: observation distribution (vector of rand-able and logdensityof-able objects)
  • state_seq: a sequence of states (vector of integers)
  • obs_seq: a sequence of observations (vector of individual observations)
  • obs_seqs: several sequences of observations

Forward backward

  • (log)b: vector of observation (log)likelihoods by state for an individual observation
  • (log)B: matrix of observation (log)likelihoods by state for a sequence of observations
  • α: scaled forward variables
  • β: scaled backward variables
  • γ: one-state marginals
  • ξ: two-state marginals
  • logL: loglikelihood of a sequence of observations

Index

diff --git a/dev/background/index.html b/dev/background/index.html index 14a79955..da8c89f2 100644 --- a/dev/background/index.html +++ b/dev/background/index.html @@ -4,4 +4,4 @@
    • - + diff --git a/dev/debugging/index.html b/dev/debugging/index.html index 28691dd9..4192426d 100644 --- a/dev/debugging/index.html +++ b/dev/debugging/index.html @@ -1,2 +1,2 @@ -Debugging · HiddenMarkovModels.jl
    GitHub

    Debugging

    Numerical overflow

    The most frequent error you will encounter is an OverflowError during forward-backward, telling you that "some values are infinite / NaN". This can happen for a variety of reasons, so here are a few leads worth investigating:

    • Increase the duration of the sequence / the number of sequences (to get more data)
    • Reduce the number of states (to make every one of them useful)
    • Add a prior to your transition matrix / observation distributions (to avoid degenerate behavior like zero variance in a Gaussian)
    • Pick a better initialization (to start closer to the supposed ground truth)
    • Use LogarithmicNumbers.jl in strategic places (to guarantee numerical stability). Note that these numbers don't play nicely with Distributions.jl, so you may have to roll out your own observation distribution.
    +Debugging · HiddenMarkovModels.jl
    GitHub

    Debugging

    Numerical overflow

    The most frequent error you will encounter is an OverflowError during forward-backward, telling you that "some values are infinite / NaN". This can happen for a variety of reasons, so here are a few leads worth investigating:

    • Increase the duration of the sequence / the number of sequences (to get more data)
    • Reduce the number of states (to make every one of them useful)
    • Add a prior to your transition matrix / observation distributions (to avoid degenerate behavior like zero variance in a Gaussian)
    • Pick a better initialization (to start closer to the supposed ground truth)
    • Use LogarithmicNumbers.jl in strategic places (to guarantee numerical stability). Note that these numbers don't play nicely with Distributions.jl, so you may have to roll out your own observation distribution.
    diff --git a/dev/formulas/index.html b/dev/formulas/index.html index 31d5951f..a4db12d1 100644 --- a/dev/formulas/index.html +++ b/dev/formulas/index.html @@ -56,4 +56,4 @@ \frac{\partial \log \mathcal{L}}{\partial a_{i,j}} &= \sum_{t=1}^{T-1} \bar{\alpha}_{i,t} \frac{b_{j,t+1}}{m_{t+1}} \bar{\beta}_{j,t+1} \\ \frac{\partial \log \mathcal{L}}{\partial \log b_{j,1}} &= \pi_j \frac{b_{j,1}}{m_1} \bar{\beta}_{j,1} = \frac{\bar{\alpha}_{j,1} \bar{\beta}_{j,1}}{c_1} = \gamma_{j,1} \\ \frac{\partial \log \mathcal{L}}{\partial \log b_{j,t}} &= \sum_{i=1}^N \bar{\alpha}_{i,t-1} a_{i,j} \frac{b_{j,t}}{m_t} \bar{\beta}_{j,t} = \frac{\bar{\alpha}_{j,t} \bar{\beta}_{j,t}}{c_t} = \gamma_{j,t} -\end{align*}\]

    +\end{align*}\]

    diff --git a/dev/index.html b/dev/index.html index f6ceed74..132c6bcd 100644 --- a/dev/index.html +++ b/dev/index.html @@ -3,4 +3,4 @@ init = [0.2, 0.8] trans = [0.1 0.9; 0.7 0.3] dists = [Normal(-1), Normal(1)] -hmm = HMM(init, trans, dists)

    Take a look at the documentation to know what to do next!

    Main features

    This package is generic. Observations can be arbitrary Julia objects, not just scalars or arrays. Their distributions only need to implement rand(rng, dist) and logdensityof(dist, x) from DensityInterface.jl. Number types are not restricted to floating point, and automatic differentiation is supported in forward mode (ForwardDiff.jl) and reverse mode (ChainRules.jl).

    This package is fast. All the inference functions have allocation-free versions, which leverage efficient linear algebra subroutines. Multithreading is used to parallelize computations across sequences, and compatibility with various array types is ensured. We include extensive benchmarks against Julia and Python competitors thanks to BenchmarkTools.jl and PythonCall.jl.

    This package is reliable. It gives the same results as the previous reference package HMMBase.jl up to numerical accuracy. The test suite incorporates quality checks with Aqua.jl, as well as linting and type stability checks with JET.jl. A detailed documentation will help you find the functions you need.

    But this package is limited in scope. It is designed for HMMs with a small number of states, because memory and runtime scale quadratically (even if the transitions are sparse). It is also meant to perform best on a CPU, and not tested at all on GPUs.

    Contributing

    If you spot a bug or want to ask about a new feature, please open an issue on the GitHub repository. Once the issue receives positive feedback, feel free to try and fix it with a pull request that follows the BlueStyle guidelines.

    Acknowledgements

    A big thank you to Maxime Mouchet and Jacob Schreiber, the respective lead devs of HMMBase.jl and pomegranate, for their help and advice. Logo by Clément Mantoux based on a portrait of Andrey Markov.

    +hmm = HMM(init, trans, dists)

    Take a look at the documentation to know what to do next!

    Main features

    This package is generic. Observations can be arbitrary Julia objects, not just scalars or arrays. Their distributions only need to implement rand(rng, dist) and logdensityof(dist, x) from DensityInterface.jl. Number types are not restricted to floating point, and automatic differentiation is supported in forward mode (ForwardDiff.jl) and reverse mode (ChainRules.jl).

    This package is fast. All the inference functions have allocation-free versions, which leverage efficient linear algebra subroutines. Multithreading is used to parallelize computations across sequences, and compatibility with various array types is ensured. We include extensive benchmarks against Julia and Python competitors thanks to BenchmarkTools.jl and PythonCall.jl.

    This package is reliable. It gives the same results as the previous reference package HMMBase.jl up to numerical accuracy. The test suite incorporates quality checks with Aqua.jl, as well as linting and type stability checks with JET.jl. A detailed documentation will help you find the functions you need.

    But this package is limited in scope. It is designed for HMMs with a small number of states, because memory and runtime scale quadratically (even if the transitions are sparse). It is also meant to perform best on a CPU, and not tested at all on GPUs.

    Contributing

    If you spot a bug or want to ask about a new feature, please open an issue on the GitHub repository. Once the issue receives positive feedback, feel free to try and fix it with a pull request that follows the BlueStyle guidelines.

    Acknowledgements

    A big thank you to Maxime Mouchet and Jacob Schreiber, the respective lead devs of HMMBase.jl and pomegranate, for their help and advice. Logo by Clément Mantoux based on a portrait of Andrey Markov.

    diff --git a/dev/roadmap/index.html b/dev/roadmap/index.html index cef15a47..747af2d3 100644 --- a/dev/roadmap/index.html +++ b/dev/roadmap/index.html @@ -1,2 +1,2 @@ -Roadmap · HiddenMarkovModels.jl
    GitHub
    +Roadmap · HiddenMarkovModels.jl
    GitHub
    diff --git a/dev/tuto_builtin/index.html b/dev/tuto_builtin/index.html index 81c50025..5175f83a 100644 --- a/dev/tuto_builtin/index.html +++ b/dev/tuto_builtin/index.html @@ -38,4 +38,4 @@ [:, :, 2] = 0.323233 0.512336 0.164431 0.337336 0.0936302 0.569033 - 0.474491 0.211947 0.313561 + 0.474491 0.211947 0.313561 diff --git a/dev/tuto_custom/index.html b/dev/tuto_custom/index.html index 983fb410..ecf4e902 100644 --- a/dev/tuto_custom/index.html +++ b/dev/tuto_custom/index.html @@ -9,8 +9,8 @@ RequiredInterfaces.check_interface_implemented(AbstractHMM, EmptyHMM)
    4-element Vector{Tuple{Any, Tuple}}:
  (HiddenMarkovModels.initial_distribution, (Main.EmptyHMM,))
  (HiddenMarkovModels.transition_matrix, (Main.EmptyHMM,))
- (HiddenMarkovModels.obs_distribution, (Main.EmptyHMM, Integer))
- (length, (Main.EmptyHMM,))

    Note that this test does not check the StatsAPI.fit! method. Since it is only used in the Baum-Welch algorithm, it is an optional part of the AbstractHMM interface.

    Example

    We show how to implement an HMM whose initial distribution is always the equilibrium distribution of the underlying Markov chain. The code that follows is not efficient (it leads to a lot of allocations), but it would be fairly easy to optimize if needed.

    The equilibrium distribution of a Markov chain is the (only) left eigenvector associated with the left eigenvalue $1$.

    function markov_equilibrium(A)
+ (length, (Main.EmptyHMM,))
+ (HiddenMarkovModels.obs_distribution, (Main.EmptyHMM, Integer))

    Note that this test does not check the StatsAPI.fit! method. Since it is only used in the Baum-Welch algorithm, it is an optional part of the AbstractHMM interface.

    Example

    We show how to implement an HMM whose initial distribution is always the equilibrium distribution of the underlying Markov chain. The code that follows is not efficient (it leads to a lot of allocations), but it would be fairly easy to optimize if needed.

    The equilibrium distribution of a Markov chain is the (only) left eigenvector associated with the left eigenvalue $1$.

    function markov_equilibrium(A)
     p = real.(eigvecs(A')[:, end])
     return p ./ sum(p)
 end
    markov_equilibrium (generic function with 1 method)

    We now define our custom HMM by taking inspiration from src/types/hmm.jl and making a few modifications:

    struct EquilibriumHMM{R,D} <: AbstractHMM
@@ -61,4 +61,4 @@
 [:, :, 2] =
  0.323233  0.512336   0.164431
  0.337336  0.0936302  0.569033
- 0.474491  0.211947   0.313561
    + 0.474491 0.211947 0.313561