X debate examines whether AI can generate novel mathematical proofs, with one argument applying the data processing inequality to claim outputs remain fully determined by axioms and training data
Luca Ambrogioni counters that random inputs add entropy enabling novelty.
@LucaAmb regardless of the process it combined known ideas in a known way. you could theoretically run it exhaustively without randomness, it would just take much longer
Luca Ambrogioni@LucaAmb@yoavgo That does not work because there are random numbers coming it. There is genuinely new entropy involved. At most you can say there is no information concerning the original training dataset, but that's besides the point
@andrewgwils @EmilevanKrieken yes
@DimitrisPapail exactly!
Dimitris Papailiopoulos@DimitrisPapail@yoavgo Thereโs no new information anyways in anything we do, because of the data processing inequality ๐
@yoavgo Thereโs no new information anyways in anything we do, because of the data processing inequality ๐
(((ู()(ู() 'yoav))))๐พ@yoavgowhere are the information theory people who will tell us there is no new information in the proof because of the data processing inequality?
@andrewgwils @EmilevanKrieken @yoavgo Ha I was reading your paper just yesterday!
@yoavgo That does not work because there are random numbers coming it. There is genuinely new entropy involved.
At most you can say there is no information concerning the original training dataset, but that's besides the point
(((ู()(ู() 'yoav))))๐พ@yoavgowhere are the information theory people who will tell us there is no new information in the proof because of the data processing inequality?
That's the library of babel, is equivalent to random sampling, you are just immagining all the samples to exist at once, but it is just a layer of immagination
The entropy is in the space of initial configuration, not it is just expressed as Kolmogorov instead of Shannon entropy, but it is a matter of perspective
(((ู()(ู() 'yoav))))๐พ@yoavgo@LucaAmb regardless of the process it combined known ideas in a known way. you could theoretically run it exhaustively without randomness, it would just take much longer