This is a meta-issue for collecting the main tasks needed for implementing a working version of the Search API. Everyone familiar with the Search API, especially (but not only) development-wise, is encouraged to chime in, discuss the changes we should make and also propose others.
The upgrade can be divided into two categories: necessary changes, and optional changes. While the former are absolutely necessary to make the Search API compatible with D8, the latter are nice-to-have code changes which we might want to make now that we have the change to easily re-define our API.
For the optional changes, see the child issues.
Especially people familiar with Drupal 8 and its changes are needed, but in general all developers (even new to D8) are more to welcome to help, there are tasks for all knowledge levels!
Regarding the inevitable question of whether the API/architecture changes will be backported to a new 7.x-2.x branch: I don't know. While having these advantages available in D7, too, would of course be a good thing, dealing with different, incompatible module branches and APIs is just a big pain for all extension modules. Probably most or all of them would have to create their own 2.x branch to stay compatible.
Also, kind of a killer argument, at the moment I have to be glad if I've even got enough time to complete the D8 upgrade in time. There's little chance I'll be able to simultaneously backport all API changes to D7.
So, let's see about that once the D8 version is working and in more or less good shape.