The right and the wrong way to do incentive pay.

The right way to think about "incentive pay" is that pay should give the long run incentives that attract good employees and the incentives to get them to stay with you long run.  Incentive pay does not usually work well for motivating workers on a day-to-day basis except in certain kinds of easily measurable work like piece-work manufacturing or some kinds of sales that have few externalities for the firm.  Even in these cases, it is not clear whether the incentive pay really motivates employees on a day to day basis or whether it works by screening the pool of workers to make a biased selection for people who want to work hard. 
The following study shows that incentive pay does not work for motivating good work.  However, high pay can bring good teachers to the profession and keep good teachers from quitting and working elsewhere.  This kind of long run incentive pay has never been studied to my knowledge and the optimal incentives would be different.  It would probably be best to pay teachers a salary that is adjusted according to their best year of results in the past four years.  That way if a teacher gets a bad class or has a rough year or two, they can make up for it another year.  This is better than paying teachers according to their yearly results because there would be much less variation and teachers are currently a risk-averse lot.  They are used to high job stability and it would be best to continue to give them fairly stable pay.  It might also be good to have a small component of their pay vary according to how their school is doing so that they have some team spirit in making their school function well.  We want teachers to work together well and we want them to encourage effective management. 

Matthew Yglesias:

Linda Pearlstein summarizes a pretty good new controlled study from Vanderbilt University that tested the idea that offering teachers bonuses of up to $15,000 could improve student outcomes. The results—nope...

The right way to think about teacher compensation, I think, is this. You could have a system in which all teachers are paid the same amount. But we don’t have that system. Instead we have a system where veteran teachers are paid much more than novice teachers, and teachers with master’s degrees are paid more than teachers without master’s degrees. We could switch this to a system where teachers whose kids do much worse than average on value-added measures get fired, and teachers whose kids to much better than average get paid more than average teachers. The idea here wouldn’t so much be to create an “incentive” as simply to ensure that the best teachers aren’t tempted to leave the profession while the worst teachers are encouraged to do so. If you want to do something through a bonus/incentive mechanism, what would make sense is to offer teachers extra money to take on challenging assignments in high poverty schools.

The point is that an absolutely flat salary structure makes no sense. Instead, we prefer to rely on proxies for quality. Currently, we use length of service and possession of a master’s degree as our proxies. But the evidence suggests that these are bad proxies and that value-added metrics, despite their flaws, are better.

Another, unrelated issue with incentive pay is that regression to the mean would tend to make negative incentives look more effective than positive incentives.  After a punishment for exceptionally bad performance, it will usually look like there is improvement due to random regression to the mean and after a reward for unusually good performance, there will usually be worse performance due to random chance.  A controlled experiment like the above can eliminate this problem, but it is something that we need to be careful about.