Boosting != Boosting Cascade

In machine learning literature, boosting and support vector machine (SVM) are the most popular algorithms out of box.

The emergence of boosting, which originated from almost an engineering-based perspective (multiple weak classifiers combine together to yield a strong one), is earlier, and less mathematically deliberate than SVM.

Upon the introduction of rapid face detection by Viola and Jones, the use of boosting, at least in the CV context, almost bears the same meaning as fast and simple. This is, however, not quite the exactly true.

First of all, the 2001 paper by Viola and Jones uses Haar feature, a feature that can be computed very fast from integral image, which is also easily obtained from a gray image.

Secondly, they employ a modified boosting algorithm that uses 1 stage at a time. As a reminder, suppose that the boosting score is expressed as a weighted voting of individual classifiers

h_m(\mathbf x) = \alpha_1 h_1(\mathbf x) + \alpha_2 h_2(\mathbf x) + \cdots + \alpha_m h_m(\mathbf x)

This expression is by no means simpler than the SVM computation. It does not even differ from any monolithic classifier.

The two reasons above bankrupt the idea that boosting == fast.

The boosting cascade introduced by VJ is trained in a way such that the classification is divided into multiple stages. Each time only one test h_i(\mathbf x) is considered. If the test passes, go to the next test; otherwise reject. Only the candidate that passes all m rounds is regarded as positive. This is effectively a degenerated decision tree, which boosts the classification speed by rejecting many candidates that fail at early stages.

Therefore, boosting cascade == fast.


Deadline pushes you to work hard

According to the Oxford English Dictionary, the original meaning of deadline was a line that does not move. The use of “dead” probably conveys a message that preemptively warns the consequence of crossing the line.

Although we never have the chance of being physically penalized for crossing a line, we are, indeed, striving in a world constrained by deadlines.

Assignments, exams, applications, conference/journal papers, thesis… anything you come up with in your college life. You are totally screwed if you ever dare cross any one of them. For a fair competition, no exemption is allowed.

Yet deadline has another function. Most of us never start to worry about it until there is one week left. A stopwatch is automatically set up. Deadline is so pushy that you cancelled any non-urgent schedule.

Even if the work is finished days before the deadline, one always tends to continue to work on it until the deadline.

Yet another magic power of deadline is that you always hand in a complete work, regardless of how well it is done.


Bonus chatter: One of my labmates started his own stopwatch 60 days before a conference deadline. Indeed, the stopwatch was virtually visible to everyone as he put it on his MSN status and kept updating it every day. This definitely nullified the need of setting up our own stopwatches.