Techmeme says the access is part of Anthropic's pledge to partner with evaluators assessing its models and safety practices.
Techmeme relays Anthropic's statement that it is giving METR employee-like access as part of a pledge to partner with evaluators assessing its models and safety practices.
14.8K
3 Sources, first seen ago
@TechmemeAnthropic says it is giving employee-like access to METR as part of its pledge to partner with evaluators for assessment of its models and safety practices (Anthropic) (Visit Techmeme dot com for the link and full context!)
@ndtvAnthropic Grants Outside Evaluator METR Employee-Level Access In AI Safety Push https://www.ndtv.com/artificial-intelligence/anthropic-grants-outside-evaluator-metr-employee-level-access-in-ai-safety-push-12193541
@kevinnbassBUT WHO AUDITS THE AUDITORS? Two weeks ago, on 29 September, Google, Anthropic, Meta, OpenAI, xAI and Nvidia signed the White House Accord on Super Intelligence. The Accord declared that every SI company should partner with an "independent external auditor or evaluator" to ensure safety. There's just one problem: The evaluator ecosystem itself is deeply entangled financially, ideologically, and in terms of personnel with the SI companies themselves. Many independent evaluator organizations are not independent at all -- but a front organization for carrying out compliance theater on behalf of their affiliated SI labs. The Accord, for all it good to defuse a volatile situation, inadvertently rubberstamped a captured evaluator system through which some of the major SI labs are laundering credibility. In this thread and accompanying tables, documents, and public repo, I show this entanglement systematically, organization by organization, relationship by relationship, every publicly available detail. 43 organizations 22,650 data points 2,635 relationships Who watches the watchmen? Who audits the auditors? I did. In earlier posts, we saw that what I call the Anthropic Network is supported by a large stock investment donated by Dustin Mostovitz to the Good Ventures Foundation. Money flows out from GVF to infuse the Anthropic Network. The Good Ventures Foundation is the source of funding for SI Doom outfits like the Tarbell Fellowship, several SI Doom research organizations, and, indirectly, the safety evaluators themselves, like METR. As Anthropic stocks increase in value, the associated non-profit funding grows correspondingly. Political lobbying by Anthropic and by lobbying groups that Anthropic has hired for SI regulation legislation is also currently rising dramatically as Anthropic achieves great economic success. The ongoing health of these nonprofits is thus intrinsically linked to the success of Anthropic itself. These independent organizations are not independent at all -- they depend on Anthropic's economic success. Additionally, these organizations are closely tied together by shared a shared founding milieu, shared personnel, and a shared ideology -- one that increasingly promotes the idea of LLMs having personhood and being an existential threat to humanity. Hence, the Anthropic Network. I argued that this creates a feedback loop that I have called the Doom Loop Machine. As Anthropic becomes more successful, the capabilities and disruptiveness of the models increase. Meanwhile, the cash flow SI Doom also increases -- because this cash flow is directly impacted by Anthropic's success -- making SI Doom increasingly louder and better funded. As Anthropic achieves greater economic success, the power to shape public perceptions by Doomer ideology increases proportionally. This produces not only potential hostility to SI among the general public in America -- which is fed SI-hostile narratives by actors closely entangled with the SI developers themselves! -- but it may impair American competitiveness with China as such attitudes are actively, propagated through the Anthropic Network's highly coordinated and capable SI Doomerism. This benefits the SI companies themselves, who may be able to push through regulation during periods of unrestrained SI hysteria that it helps to shape both over time and at pivotal moments of uncertainty. This could lead to regulatory capture and a monopoly, eliminating competitors, helping the leading SI companies, but slowing the pace of American innovation. I argue that this is the game plan -- not legible in any document (that I am aware of), but clearly legible in the financial structure and ideological structure that the public record maps. In other words, the interests of our currently leading SI companies may be at odds with the national interests of America -- and perversely, aligned with the national interests of China -- who are far behind America in model capabilities. We saw the effects of this two weeks ago: many prominent politicians, ostensibly unconnected to Anthropic, such as Barack Obama and Hillary Clinton, were pushing the idea of banning open source models -- something that, if you watched them speak, they clearly knew nothing about apart from the talking points they were apparently fed. This is not a conspiracy theory. To be explicit, it is a structural fact legible in thousands of publicly available documents that I am presenting here. I am presenting these data, exhaustively, without censoring or selecting any -- Claude artifact and Github attached to the end of this thread. Many current observations are best explained in this way. Just to name a few examples: 1. Why are SI safety orgs so weak in cybersecurity expertise? If the ideological orientation is toward SI doom, and if the financial incentives go in that same direction, then cybersecurity experts are ideologically and financially anathema to SI safety organizations. That would explain why, despite dramatically increasing, they are so bad at cybersecurity. 2. Why do such extraordinarily intelligent people at companies like Anthropic publish such bad papers about open models, some of which verge on pseudoscience? Their latest about open models infecting devices and becoming a giant SI botnet is riddled with holes and fallacies that many intelligent readers with familiarity with the field will immediately identify -- many have, such as David Sacks. Why did such brilliant people publish such a terrible paper? Why have Anthropic researchers been publishing so many negative articles about open source models in general. Incentives. The more Anthropic can generate fear about open models, the more leverage they will have to demand regulations during critical moments over the course of the SI rollout. This is, I argue, a long-term play. 3. Finally, why are attitudes in China and America toward SI so divergent? The Chinese are very optimistic about SI. The Americans are very fearful. I would argue a part of that is that Chinese frontier SI is open source, while American frontier SI is closed source. Closed source SI promotes SI Doom because the long-term financial play incentivizes attempts to achieve regulatory capture, and open source SI promotes optimism toward SI because that's how open source SI gets more public support. If this is right, frontier SI labs may be creating friction that is already harming American SI efforts; the harm is not in the future; it is already here. The White House Accord did not fully address this structural problem. We should be grateful for the White House's efforts. How liability is structured plays a major role in limiting the damage that SI Doomers/regulatory capture enthusiasts can do, by neutralizing claims that some entity other than the frontier lab itself (i.e., the government) must take responsibility for what the frontier lab creates. But these networks and incentives still exist after the Accord, just as before it. And significant incidents that Doomers could exploit could unravel the current detente. Arguably, the Accord handed a win to the SI Doomers. When the Accord insisted on "independent" evaluators but did not define what these are, it allowed frontier SI labs to claim affiliated organizations as "independent" to launder credibility, in the name of adhering to the letter of the Accord. But these frontier SI labs do not have the interests of American national security first in mind, but their own. Thus, we are back to business as usual. METR is now touted by Anthropic as an independent evaluator, which it most assuredly is not. And so on. But what about when we are not back to business as usual? As I alluded to, incidents, small and large, are inevitable. Institutional capacity is being built through the SI Safety ecosystem, driven by actors whose goal is regulatory capture. This capacity will allow these organizations to respond effectively to incidents when they occur -- and shape public perception and legislation to advance the financial interests of the frontier labs. In short, evaluators, because they are not really independent, are in fact merely part of the companies' PR and lobbying apparatus, shaping public narratives around incidents to benefit the companies that they are "independently" evaluating. This comes at the detriment of the public interest in the United States. We therefore need to be clear-eyed about what these conflicts are. Attached to this lead post is the "mega-figure" showing all of the entanglements of the organizations in the SI and safety ecosystems. This thread will work through that in different slices and representations, and some of their policy implications. The Github repo and http://claude.ai artifact at the end of this thread will be a valuable reference for journalists and interested laypeople. Download the raw data or point your favorite LLM at it and ask questions.@TechmemeAnthropic says it is giving employee-like access to METR as part of its pledge to partner with evaluators for assessment of its models and safety practices (Anthropic) (Visit Techmeme dot com for the link and full context!)@ndtvAnthropic Grants Outside Evaluator METR Employee-Level Access In AI Safety Push https://www.ndtv.com/artificial-intelligence/anthropic-grants-outside-evaluator-metr-employee-level-access-in-ai-safety-push-12193541@kevinnbassBUT WHO AUDITS THE AUDITORS? Two weeks ago, on 29 September, Google, Anthropic, Meta, OpenAI, xAI and Nvidia signed the White House Accord on Super Intelligence. The Accord declared that every SI company should partner with an "independent external auditor or evaluator" to ensure safety. There's just one problem: The evaluator ecosystem itself is deeply entangled financially, ideologically, and in terms of personnel with the SI companies themselves. Many independent evaluator organizations are not independent at all -- but a front organization for carrying out compliance theater on behalf of their affiliated SI labs. The Accord, for all it good to defuse a volatile situation, inadvertently rubberstamped a captured evaluator system through which some of the major SI labs are laundering credibility. In this thread and accompanying tables, documents, and public repo, I show this entanglement systematically, organization by organization, relationship by relationship, every publicly available detail. 43 organizations 22,650 data points 2,635 relationships Who watches the watchmen? Who audits the auditors? I did. In earlier posts, we saw that what I call the Anthropic Network is supported by a large stock investment donated by Dustin Mostovitz to the Good Ventures Foundation. Money flows out from GVF to infuse the Anthropic Network. The Good Ventures Foundation is the source of funding for SI Doom outfits like the Tarbell Fellowship, several SI Doom research organizations, and, indirectly, the safety evaluators themselves, like METR. As Anthropic stocks increase in value, the associated non-profit funding grows correspondingly. Political lobbying by Anthropic and by lobbying groups that Anthropic has hired for SI regulation legislation is also currently rising dramatically as Anthropic achieves great economic success. The ongoing health of these nonprofits is thus intrinsically linked to the success of Anthropic itself. These independent organizations are not independent at all -- they depend on Anthropic's economic success. Additionally, these organizations are closely tied together by shared a shared founding milieu, shared personnel, and a shared ideology -- one that increasingly promotes the idea of LLMs having personhood and being an existential threat to humanity. Hence, the Anthropic Network. I argued that this creates a feedback loop that I have called the Doom Loop Machine. As Anthropic becomes more successful, the capabilities and disruptiveness of the models increase. Meanwhile, the cash flow SI Doom also increases -- because this cash flow is directly impacted by Anthropic's success -- making SI Doom increasingly louder and better funded. As Anthropic achieves greater economic success, the power to shape public perceptions by Doomer ideology increases proportionally. This produces not only potential hostility to SI among the general public in America -- which is fed SI-hostile narratives by actors closely entangled with the SI developers themselves! -- but it may impair American competitiveness with China as such attitudes are actively, propagated through the Anthropic Network's highly coordinated and capable SI Doomerism. This benefits the SI companies themselves, who may be able to push through regulation during periods of unrestrained SI hysteria that it helps to shape both over time and at pivotal moments of uncertainty. This could lead to regulatory capture and a monopoly, eliminating competitors, helping the leading SI companies, but slowing the pace of American innovation. I argue that this is the game plan -- not legible in any document (that I am aware of), but clearly legible in the financial structure and ideological structure that the public record maps. In other words, the interests of our currently leading SI companies may be at odds with the national interests of America -- and perversely, aligned with the national interests of China -- who are far behind America in model capabilities. We saw the effects of this two weeks ago: many prominent politicians, ostensibly unconnected to Anthropic, such as Barack Obama and Hillary Clinton, were pushing the idea of banning open source models -- something that, if you watched them speak, they clearly knew nothing about apart from the talking points they were apparently fed. This is not a conspiracy theory. To be explicit, it is a structural fact legible in thousands of publicly available documents that I am presenting here. I am presenting these data, exhaustively, without censoring or selecting any -- Claude artifact and Github attached to the end of this thread. Many current observations are best explained in this way. Just to name a few examples: 1. Why are SI safety orgs so weak in cybersecurity expertise? If the ideological orientation is toward SI doom, and if the financial incentives go in that same direction, then cybersecurity experts are ideologically and financially anathema to SI safety organizations. That would explain why, despite dramatically increasing, they are so bad at cybersecurity. 2. Why do such extraordinarily intelligent people at companies like Anthropic publish such bad papers about open models, some of which verge on pseudoscience? Their latest about open models infecting devices and becoming a giant SI botnet is riddled with holes and fallacies that many intelligent readers with familiarity with the field will immediately identify -- many have, such as David Sacks. Why did such brilliant people publish such a terrible paper? Why have Anthropic researchers been publishing so many negative articles about open source models in general. Incentives. The more Anthropic can generate fear about open models, the more leverage they will have to demand regulations during critical moments over the course of the SI rollout. This is, I argue, a long-term play. 3. Finally, why are attitudes in China and America toward SI so divergent? The Chinese are very optimistic about SI. The Americans are very fearful. I would argue a part of that is that Chinese frontier SI is open source, while American frontier SI is closed source. Closed source SI promotes SI Doom because the long-term financial play incentivizes attempts to achieve regulatory capture, and open source SI promotes optimism toward SI because that's how open source SI gets more public support. If this is right, frontier SI labs may be creating friction that is already harming American SI efforts; the harm is not in the future; it is already here. The White House Accord did not fully address this structural problem. We should be grateful for the White House's efforts. How liability is structured plays a major role in limiting the damage that SI Doomers/regulatory capture enthusiasts can do, by neutralizing claims that some entity other than the frontier lab itself (i.e., the government) must take responsibility for what the frontier lab creates. But these networks and incentives still exist after the Accord, just as before it. And significant incidents that Doomers could exploit could unravel the current detente. Arguably, the Accord handed a win to the SI Doomers. When the Accord insisted on "independent" evaluators but did not define what these are, it allowed frontier SI labs to claim affiliated organizations as "independent" to launder credibility, in the name of adhering to the letter of the Accord. But these frontier SI labs do not have the interests of American national security first in mind, but their own. Thus, we are back to business as usual. METR is now touted by Anthropic as an independent evaluator, which it most assuredly is not. And so on. But what about when we are not back to business as usual? As I alluded to, incidents, small and large, are inevitable. Institutional capacity is being built through the SI Safety ecosystem, driven by actors whose goal is regulatory capture. This capacity will allow these organizations to respond effectively to incidents when they occur -- and shape public perception and legislation to advance the financial interests of the frontier labs. In short, evaluators, because they are not really independent, are in fact merely part of the companies' PR and lobbying apparatus, shaping public narratives around incidents to benefit the companies that they are "independently" evaluating. This comes at the detriment of the public interest in the United States. We therefore need to be clear-eyed about what these conflicts are. Attached to this lead post is the "mega-figure" showing all of the entanglements of the organizations in the SI and safety ecosystems. This thread will work through that in different slices and representations, and some of their policy implications. The Github repo and http://claude.ai artifact at the end of this thread will be a valuable reference for journalists and interested laypeople. Download the raw data or point your favorite LLM at it and ask questions.@TechmemeAnthropic says it is giving employee-like access to METR as part of its pledge to partner with evaluators for assessment of its models and safety practices (Anthropic) (Visit Techmeme dot com for the link and full context!)@ndtvAnthropic Grants Outside Evaluator METR Employee-Level Access In AI Safety Push https://www.ndtv.com/artificial-intelligence/anthropic-grants-outside-evaluator-metr-employee-level-access-in-ai-safety-push-12193541@kevinnbassBUT WHO AUDITS THE AUDITORS? Two weeks ago, on 29 September, Google, Anthropic, Meta, OpenAI, xAI and Nvidia signed the White House Accord on Super Intelligence. The Accord declared that every SI company should partner with an "independent external auditor or evaluator" to ensure safety. There's just one problem: The evaluator ecosystem itself is deeply entangled financially, ideologically, and in terms of personnel with the SI companies themselves. Many independent evaluator organizations are not independent at all -- but a front organization for carrying out compliance theater on behalf of their affiliated SI labs. The Accord, for all it good to defuse a volatile situation, inadvertently rubberstamped a captured evaluator system through which some of the major SI labs are laundering credibility. In this thread and accompanying tables, documents, and public repo, I show this entanglement systematically, organization by organization, relationship by relationship, every publicly available detail. 43 organizations 22,650 data points 2,635 relationships Who watches the watchmen? Who audits the auditors? I did. In earlier posts, we saw that what I call the Anthropic Network is supported by a large stock investment donated by Dustin Mostovitz to the Good Ventures Foundation. Money flows out from GVF to infuse the Anthropic Network. The Good Ventures Foundation is the source of funding for SI Doom outfits like the Tarbell Fellowship, several SI Doom research organizations, and, indirectly, the safety evaluators themselves, like METR. As Anthropic stocks increase in value, the associated non-profit funding grows correspondingly. Political lobbying by Anthropic and by lobbying groups that Anthropic has hired for SI regulation legislation is also currently rising dramatically as Anthropic achieves great economic success. The ongoing health of these nonprofits is thus intrinsically linked to the success of Anthropic itself. These independent organizations are not independent at all -- they depend on Anthropic's economic success. Additionally, these organizations are closely tied together by shared a shared founding milieu, shared personnel, and a shared ideology -- one that increasingly promotes the idea of LLMs having personhood and being an existential threat to humanity. Hence, the Anthropic Network. I argued that this creates a feedback loop that I have called the Doom Loop Machine. As Anthropic becomes more successful, the capabilities and disruptiveness of the models increase. Meanwhile, the cash flow SI Doom also increases -- because this cash flow is directly impacted by Anthropic's success -- making SI Doom increasingly louder and better funded. As Anthropic achieves greater economic success, the power to shape public perceptions by Doomer ideology increases proportionally. This produces not only potential hostility to SI among the general public in America -- which is fed SI-hostile narratives by actors closely entangled with the SI developers themselves! -- but it may impair American competitiveness with China as such attitudes are actively, propagated through the Anthropic Network's highly coordinated and capable SI Doomerism. This benefits the SI companies themselves, who may be able to push through regulation during periods of unrestrained SI hysteria that it helps to shape both over time and at pivotal moments of uncertainty. This could lead to regulatory capture and a monopoly, eliminating competitors, helping the leading SI companies, but slowing the pace of American innovation. I argue that this is the game plan -- not legible in any document (that I am aware of), but clearly legible in the financial structure and ideological structure that the public record maps. In other words, the interests of our currently leading SI companies may be at odds with the national interests of America -- and perversely, aligned with the national interests of China -- who are far behind America in model capabilities. We saw the effects of this two weeks ago: many prominent politicians, ostensibly unconnected to Anthropic, such as Barack Obama and Hillary Clinton, were pushing the idea of banning open source models -- something that, if you watched them speak, they clearly knew nothing about apart from the talking points they were apparently fed. This is not a conspiracy theory. To be explicit, it is a structural fact legible in thousands of publicly available documents that I am presenting here. I am presenting these data, exhaustively, without censoring or selecting any -- Claude artifact and Github attached to the end of this thread. Many current observations are best explained in this way. Just to name a few examples: 1. Why are SI safety orgs so weak in cybersecurity expertise? If the ideological orientation is toward SI doom, and if the financial incentives go in that same direction, then cybersecurity experts are ideologically and financially anathema to SI safety organizations. That would explain why, despite dramatically increasing, they are so bad at cybersecurity. 2. Why do such extraordinarily intelligent people at companies like Anthropic publish such bad papers about open models, some of which verge on pseudoscience? Their latest about open models infecting devices and becoming a giant SI botnet is riddled with holes and fallacies that many intelligent readers with familiarity with the field will immediately identify -- many have, such as David Sacks. Why did such brilliant people publish such a terrible paper? Why have Anthropic researchers been publishing so many negative articles about open source models in general. Incentives. The more Anthropic can generate fear about open models, the more leverage they will have to demand regulations during critical moments over the course of the SI rollout. This is, I argue, a long-term play. 3. Finally, why are attitudes in China and America toward SI so divergent? The Chinese are very optimistic about SI. The Americans are very fearful. I would argue a part of that is that Chinese frontier SI is open source, while American frontier SI is closed source. Closed source SI promotes SI Doom because the long-term financial play incentivizes attempts to achieve regulatory capture, and open source SI promotes optimism toward SI because that's how open source SI gets more public support. If this is right, frontier SI labs may be creating friction that is already harming American SI efforts; the harm is not in the future; it is already here. The White House Accord did not fully address this structural problem. We should be grateful for the White House's efforts. How liability is structured plays a major role in limiting the damage that SI Doomers/regulatory capture enthusiasts can do, by neutralizing claims that some entity other than the frontier lab itself (i.e., the government) must take responsibility for what the frontier lab creates. But these networks and incentives still exist after the Accord, just as before it. And significant incidents that Doomers could exploit could unravel the current detente. Arguably, the Accord handed a win to the SI Doomers. When the Accord insisted on "independent" evaluators but did not define what these are, it allowed frontier SI labs to claim affiliated organizations as "independent" to launder credibility, in the name of adhering to the letter of the Accord. But these frontier SI labs do not have the interests of American national security first in mind, but their own. Thus, we are back to business as usual. METR is now touted by Anthropic as an independent evaluator, which it most assuredly is not. And so on. But what about when we are not back to business as usual? As I alluded to, incidents, small and large, are inevitable. Institutional capacity is being built through the SI Safety ecosystem, driven by actors whose goal is regulatory capture. This capacity will allow these organizations to respond effectively to incidents when they occur -- and shape public perception and legislation to advance the financial interests of the frontier labs. In short, evaluators, because they are not really independent, are in fact merely part of the companies' PR and lobbying apparatus, shaping public narratives around incidents to benefit the companies that they are "independently" evaluating. This comes at the detriment of the public interest in the United States. We therefore need to be clear-eyed about what these conflicts are. Attached to this lead post is the "mega-figure" showing all of the entanglements of the organizations in the SI and safety ecosystems. This thread will work through that in different slices and representations, and some of their policy implications. The Github repo and http://claude.ai artifact at the end of this thread will be a valuable reference for journalists and interested laypeople. Download the raw data or point your favorite LLM at it and ask questions.@TechmemeAnthropic says it is giving employee-like access to METR as part of its pledge to partner with evaluators for assessment of its models and safety practices (Anthropic) (Visit Techmeme dot com for the link and full context!)@ndtvAnthropic Grants Outside Evaluator METR Employee-Level Access In AI Safety Push https://www.ndtv.com/artificial-intelligence/anthropic-grants-outside-evaluator-metr-employee-level-access-in-ai-safety-push-12193541@kevinnbassBUT WHO AUDITS THE AUDITORS? Two weeks ago, on 29 September, Google, Anthropic, Meta, OpenAI, xAI and Nvidia signed the White House Accord on Super Intelligence. The Accord declared that every SI company should partner with an "independent external auditor or evaluator" to ensure safety. There's just one problem: The evaluator ecosystem itself is deeply entangled financially, ideologically, and in terms of personnel with the SI companies themselves. Many independent evaluator organizations are not independent at all -- but a front organization for carrying out compliance theater on behalf of their affiliated SI labs. The Accord, for all it good to defuse a volatile situation, inadvertently rubberstamped a captured evaluator system through which some of the major SI labs are laundering credibility. In this thread and accompanying tables, documents, and public repo, I show this entanglement systematically, organization by organization, relationship by relationship, every publicly available detail. 43 organizations 22,650 data points 2,635 relationships Who watches the watchmen? Who audits the auditors? I did. In earlier posts, we saw that what I call the Anthropic Network is supported by a large stock investment donated by Dustin Mostovitz to the Good Ventures Foundation. Money flows out from GVF to infuse the Anthropic Network. The Good Ventures Foundation is the source of funding for SI Doom outfits like the Tarbell Fellowship, several SI Doom research organizations, and, indirectly, the safety evaluators themselves, like METR. As Anthropic stocks increase in value, the associated non-profit funding grows correspondingly. Political lobbying by Anthropic and by lobbying groups that Anthropic has hired for SI regulation legislation is also currently rising dramatically as Anthropic achieves great economic success. The ongoing health of these nonprofits is thus intrinsically linked to the success of Anthropic itself. These independent organizations are not independent at all -- they depend on Anthropic's economic success. Additionally, these organizations are closely tied together by shared a shared founding milieu, shared personnel, and a shared ideology -- one that increasingly promotes the idea of LLMs having personhood and being an existential threat to humanity. Hence, the Anthropic Network. I argued that this creates a feedback loop that I have called the Doom Loop Machine. As Anthropic becomes more successful, the capabilities and disruptiveness of the models increase. Meanwhile, the cash flow SI Doom also increases -- because this cash flow is directly impacted by Anthropic's success -- making SI Doom increasingly louder and better funded. As Anthropic achieves greater economic success, the power to shape public perceptions by Doomer ideology increases proportionally. This produces not only potential hostility to SI among the general public in America -- which is fed SI-hostile narratives by actors closely entangled with the SI developers themselves! -- but it may impair American competitiveness with China as such attitudes are actively, propagated through the Anthropic Network's highly coordinated and capable SI Doomerism. This benefits the SI companies themselves, who may be able to push through regulation during periods of unrestrained SI hysteria that it helps to shape both over time and at pivotal moments of uncertainty. This could lead to regulatory capture and a monopoly, eliminating competitors, helping the leading SI companies, but slowing the pace of American innovation. I argue that this is the game plan -- not legible in any document (that I am aware of), but clearly legible in the financial structure and ideological structure that the public record maps. In other words, the interests of our currently leading SI companies may be at odds with the national interests of America -- and perversely, aligned with the national interests of China -- who are far behind America in model capabilities. We saw the effects of this two weeks ago: many prominent politicians, ostensibly unconnected to Anthropic, such as Barack Obama and Hillary Clinton, were pushing the idea of banning open source models -- something that, if you watched them speak, they clearly knew nothing about apart from the talking points they were apparently fed. This is not a conspiracy theory. To be explicit, it is a structural fact legible in thousands of publicly available documents that I am presenting here. I am presenting these data, exhaustively, without censoring or selecting any -- Claude artifact and Github attached to the end of this thread. Many current observations are best explained in this way. Just to name a few examples: 1. Why are SI safety orgs so weak in cybersecurity expertise? If the ideological orientation is toward SI doom, and if the financial incentives go in that same direction, then cybersecurity experts are ideologically and financially anathema to SI safety organizations. That would explain why, despite dramatically increasing, they are so bad at cybersecurity. 2. Why do such extraordinarily intelligent people at companies like Anthropic publish such bad papers about open models, some of which verge on pseudoscience? Their latest about open models infecting devices and becoming a giant SI botnet is riddled with holes and fallacies that many intelligent readers with familiarity with the field will immediately identify -- many have, such as David Sacks. Why did such brilliant people publish such a terrible paper? Why have Anthropic researchers been publishing so many negative articles about open source models in general. Incentives. The more Anthropic can generate fear about open models, the more leverage they will have to demand regulations during critical moments over the course of the SI rollout. This is, I argue, a long-term play. 3. Finally, why are attitudes in China and America toward SI so divergent? The Chinese are very optimistic about SI. The Americans are very fearful. I would argue a part of that is that Chinese frontier SI is open source, while American frontier SI is closed source. Closed source SI promotes SI Doom because the long-term financial play incentivizes attempts to achieve regulatory capture, and open source SI promotes optimism toward SI because that's how open source SI gets more public support. If this is right, frontier SI labs may be creating friction that is already harming American SI efforts; the harm is not in the future; it is already here. The White House Accord did not fully address this structural problem. We should be grateful for the White House's efforts. How liability is structured plays a major role in limiting the damage that SI Doomers/regulatory capture enthusiasts can do, by neutralizing claims that some entity other than the frontier lab itself (i.e., the government) must take responsibility for what the frontier lab creates. But these networks and incentives still exist after the Accord, just as before it. And significant incidents that Doomers could exploit could unravel the current detente. Arguably, the Accord handed a win to the SI Doomers. When the Accord insisted on "independent" evaluators but did not define what these are, it allowed frontier SI labs to claim affiliated organizations as "independent" to launder credibility, in the name of adhering to the letter of the Accord. But these frontier SI labs do not have the interests of American national security first in mind, but their own. Thus, we are back to business as usual. METR is now touted by Anthropic as an independent evaluator, which it most assuredly is not. And so on. But what about when we are not back to business as usual? As I alluded to, incidents, small and large, are inevitable. Institutional capacity is being built through the SI Safety ecosystem, driven by actors whose goal is regulatory capture. This capacity will allow these organizations to respond effectively to incidents when they occur -- and shape public perception and legislation to advance the financial interests of the frontier labs. In short, evaluators, because they are not really independent, are in fact merely part of the companies' PR and lobbying apparatus, shaping public narratives around incidents to benefit the companies that they are "independently" evaluating. This comes at the detriment of the public interest in the United States. We therefore need to be clear-eyed about what these conflicts are. Attached to this lead post is the "mega-figure" showing all of the entanglements of the organizations in the SI and safety ecosystems. This thread will work through that in different slices and representations, and some of their policy implications. The Github repo and http://claude.ai artifact at the end of this thread will be a valuable reference for journalists and interested laypeople. Download the raw data or point your favorite LLM at it and ask questions.—
Not ranked yet
—
Not ranked yet
