SVM^hmm: Using Support Vector Machines to Train Hidden Markov Models

Abstract

Support vector machines (SVMs) map each input instance (in the form of a vector) to one of two classes represented by the integers +1 and -1. Prof. Thorsten Joachims and others have introduced support vector machines for learning more complicated forms of output with structure to them, such as trees and sequences. I've extended SVM^struct, Prof. Joachims' structural-SVM package written in C, to learn hidden Markov models (HMMs). So far we have tested SVM^hmm on part-of-speech tagging of the Wall Street Journal corpus from the Penn Treebank project. It outperforms three popular non-SVM taggers even after the non-SVM packages are tweaked for the test set.

Prof. Joachims has also recently improved the underlying algorithm, so that the training runtime of our SVM-based tagger is now within an order of magnitude of those of the rule-based system and the decision tree generator.

Resources

• Get much more detail, or download SVM^hmm, at the SVM^hmm site.
• Prof. Joachims' site, prominently featuring SVM^struct, which is also downloadable.
• I'm happy to answer questions via e-mail.