Session warming remains disabled by default
A person involved says it makes clients appear more active to inference providers, removing a signal they can use to optimize resources.
TLDR
A person involved in adding session warming says it was added a while ago but remains disabled by default. They say it effectively makes a client appear more active than it is, removing a signal inference providers can use to optimize resources. They wondered how providers would feel about it.
Combined views
6K
1 Source, first seen ago
