@pwang You have the right to inspect what is running on your computer. There are absolutely no legal mechanisms for preventing you from doing that and there are also no technical mechanisms introduced by the people who developed the model to try to make it harder for you to do that.
One of the foundational principles for the original open source movement was the right to inspect what’s actually running on your computer. The 1970s way to manifest this was the require the availability of the source code (precursor for executable binaries).
For an LLM, that transparency and legibility has to manifest as interpretability. Indeed, no model is there yet, but that’s what the original spirit of “open source” would require.

