How can I use HMM for handwriting recognition?

Question

The problem is slightly different from traditional handwriting recognition. I have a dataset that are thousands of the following. For one drawn character, I have several consecutive coordinates (x, y) where the pen was pressed. So this is a temporary (temporary) problem.

I want to be able to classify handwritten characters based on this data and would like to implement HMM for training. But is this the right approach? How can they be used for this?

Answer 1

I think that HMM can be used in both issues mentioned by @jens. I also work on online handwriting, and HMM is used in many articles. The easiest way:

• Select a function.
• If the selected function is continuous, convert it to a discrete one.
• Select HMM options: topology and # states.
• Draw character models using HMM. one model for each class.
• Testing using a test suite.

for each item:

• the simplest feature is the angle of the vector, which connects consecutive points. You can use more complex functions, such as angles of vectors obtained by the Douglas and Picker algorithm.
• The easiest way to sample is to use Freeman codes, but clustering algorithms such as k-tools and GMM can also be used.
• HMM topologies: Ergodic, Left-Right, Bakis, and Linear. # states can be obtained by trial and error. HMM parameters can be variable for each model. The number of observations is determined by discretization. observation samples may have a variable length.
• I recommend the Kevin Murphy HMM toolbox.
• Good luck.

Answer 2

This problem is a combination of two problems:

• recognition of one character from your data
• word recognition from a (noisy) character sequence

A HMM is used to find the most likely sequence of a finite number of discrete states from noisy measurements. This is precisely Problem 2, since the sequence follows the measured measurements of discrete states az, 0-9.

For problem 1, HMM is useless because you are not interested in the basic sequence. You want to increase your handwritten number with information about how you wrote it.

Personally, I would start by introducing the usual modern handwriting recognition, which is already very good (with convolutional neural networks or deep learning). After that, you can add information about how it was written, for example, clockwise / counterclockwise.