How AI self-improvement could level off

One post considers strong recursive self-improvement—AI improving itself—as a hypothetical, arguing that practical limits would turn its progress into a “hidden S curve.” It questions how a system could reliably judge its own improvement, especially for tasks without clear verification, such as writing a poem or choosing a gift. Other proposed constraints include physical trial and error, the time needed to assess long-term drug side effects, computing resources and reliable memory. The post also asks whether a system able to adjust its own internal weights could know whether those changes made it better or worse.