Training a Pairwise Ising-type Model with Maximum Entropy Principle
The ising model energy is defined as:
$$
E(\sigma) = -\sum_{ij}J_{ij}\sigma_i\sigma_j - \sum_{i} H_i\sigma_i
$$
In which, $\sigma_i = {0, 1}$ in this work.
Let $\mathcal S$ is the set of all possible configuration. $|\mathcal S| = 2^n$,
where $n$ is the number of sites, and is equal to the length of $\sigma$.
In maximum entropy principle, we expect to maximize the entropy $S(p)=-\sum_\sigma p(\sigma)logp(\sigma)$, in the constraints that
$\langle \sigma_i \sigma_j\rangle^{emp} = \langle \sigma_i \sigma_j\rangle$, $\langle \sigma_i\rangle^{emp}=\langle \sigma_i\rangle$ and $\sum_\sigma p(\sigma) = 1$.
Combine to the Lagrange function:
$\mathcal L(p;J;H)=S(p) -\lambda((\sum_\sigma p(\sigma))-1)-\sum_{ij}J_{ij}(\langle \sigma_i \sigma_j\rangle -\langle \sigma_i \sigma_j\rangle^{emp})-\sum_{j}H_{i}(\langle \sigma_i \rangle -\langle \sigma_i\rangle^{emp})$