• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
  • HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    AI

    A satirical guide to “faking” robotics papers calls out cherry-picked results

    One post lists hidden failures, carefully controlled demos and weak comparison baselines as tactics to avoid—not advice to follow.

    CP
    2 Sources, 19d ago, first seen 19d ago

    TLDR

    A post framed as a guide to “how to fake a robotics paper” criticizes restricting access to models, tailoring demo conditions and cherry-picking benchmarks. It argues that outside testing can expose a model’s limitations and that strong robotics results can show failures with confidence. The author explicitly rejects the listed tactics: “obviously don't do these things please.”

    Combined views

    10.3K

    2 Sources, first seen 19d ago

    Combined views

    10.3K

    2 Sources, first seen 19d ago

    122 likes
    122 likes
    9 comments
    69 saves
    19 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    9 comments
    69 saves
    19 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet

    2 Sources

    @chris_j_paxtonWith the ICRA deadline only a few days away, it's a great time to look over my guide for how to fake a robotics paper: - Don’t let other people run comparisons. If other people can try your model out, they’ll quickly uncover its limitations. If you can keep your model secret, that’s best. - Control the environment of your demo carefully. Lighting, objects, initial robot configuration, and so on. This lets you overfit the demo scene and get really nice, smooth, high quality motions. - Never show your failures. This one might seem obvious — why would I show failures? — but if people can see where your model fails, they can start to see the limits of what you can do. Only strong robotics papers and results can show failures with confidence. - If you have to let other people run comparisons, choose the people carefully. Make sure they’re only happening in the right circumstances, ones your model supports as roughly in distribution. Absolutely don’t do what Physical Intelligence or NVIDIA do and open source your model so anyone can benchmark it. - When working on the results section of your research paper or blog post, you may be tempted to include some baselines. This is a good idea; just be careful to choose weak baselines so you look good. - On the same note: cherry-pick your benchmarks. There are a ton of robotics benchmarks out there, and they all test subtly different things. Importantly, these differences (obviously don't do these things please) https://itcanthink.substack.com/p/how-to-fake-a-robotics-result

    2 Sources

    @chris_j_paxtonWith the ICRA deadline only a few days away, it's a great time to look over my guide for how to fake a robotics paper: - Don’t let other people run comparisons. If other people can try your model out, they’ll quickly uncover its limitations. If you can keep your model secret, that’s best. - Control the environment of your demo carefully. Lighting, objects, initial robot configuration, and so on. This lets you overfit the demo scene and get really nice, smooth, high quality motions. - Never show your failures. This one might seem obvious — why would I show failures? — but if people can see where your model fails, they can start to see the limits of what you can do. Only strong robotics papers and results can show failures with confidence. - If you have to let other people run comparisons, choose the people carefully. Make sure they’re only happening in the right circumstances, ones your model supports as roughly in distribution. Absolutely don’t do what Physical Intelligence or NVIDIA do and open source your model so anyone can benchmark it. - When working on the results section of your research paper or blog post, you may be tempted to include some baselines. This is a good idea; just be careful to choose weak baselines so you look good. - On the same note: cherry-pick your benchmarks. There are a ton of robotics benchmarks out there, and they all test subtly different things. Importantly, these differences (obviously don't do these things please) https://itcanthink.substack.com/p/how-to-fake-a-robotics-result