I've tracked it down to the following problem: If the field, that gets the 'tabledrag-handle' prepended has a colspan, then the hidden column is not calculated correctly. The column that is one to the right of the column that should be hidden is hidden.
The markup of the table is fine, I've checked that without the tabledrag enabled. Also, if I replace the one field (the first one in the row) that has the colspan (2), with 2 regular fields each with colspan 1, everything works. That's what's lead me to believe this is actually a bug.
I can provide more info, of course, if that's needed.