This worked normally again when I disabled the admin module.
I have no clue what caused this. I have a copy of this website in production and it has the same issue. I also can't remember if I ever enabled the admin module before without js compression, because I know I installed the admin module quite recently.
Any help and tips to debug this problem is higly welcome. I will try to reproduce this issue on another drupal setup as well.