Positive users welcomed the idea that AI policies could select their own action wait times because actions are often unnecessary anyway, while one reply dismissed the discussion as buzzwords.
No Digg Deeper questions have been answered for this story yet.
Most Activity
my goal is mostly to reduce the amount of time my computer spends simulating things
I wonder if anyone has ever let a policy decide how much time it wants to wait before making its next action
@Control_wiz i guess you could do environment based wakeups
@yacineMTB This would be an issue for open loop unstable systems or if you have fast disturbances/uncertainties.
I'd think about free energy principle.
If your model has work to do and it has to make a series of actions, what is the most efficient way to do the work and preserve the energy? Should it prepare the body/hands before next action? How can we encode "anticipating" the next action?
I know the free energy principle is about the brain / markov chains and not training the robots but this question makes me think that this is optimization problem. Deciding whether or not to move before taking next action.
@yacineMTB I have but I usually keep it as part of my AGENTS.md. I haven't spent too much time polishing it since most of my prompts are bite sized so it doesn't run off the rails
@yacineMTB yeah but my network was running on cpu so it sucked. but just a delta T output passed to the system delay.
@kushalk_ Yes, my goal is to reduce env proc time
@yacineMTB Problem is that unpredictable events can happen while waiting. Seems like it would be better to train a small net for basic balance and free-space action, that also learns to recognize imminent contacts or perturbations it can't deal with, signaling the full-size net to run
@yacineMTB this can be analogized to perseverance which is a heuristic used in modern chess solvers: it decides how deep it needs to explore based on observations
@yacineMTB Ooh that’s a good idea. A lot of the time you probably don’t need to do anything at all.
@yacineMTB this is where my GCODE like idea could work, for this problem I was thinking the robot actions as music symbols/tokens where each symbol as a magnitude and duration
@yacineMTB I have thought about it in the past but this idea of variable control Hz has best ROI if the sim speeds up when the policy is waiting. Are you working on that?
@yacineMTB RL quietly becoming phenomenology
@yacineMTB Waiting is an action
@yacineMTB damn policies acting like they got free will while we watch charts do their own thing
@yacineMTB bunch of buzzwords as always
@yacineMTB LLMs are still not the best at writing software, so I try to reduce the probability of it running wild
@yacineMTB I got you thanks for the idea I’ll see if it works as a test function would fit in to the pipeline reading it now the import function in the autonomous line in that module would actually make it easier since it’s being routed to a single mapped out context hmmm 🤔
@yacineMTB or do you mean like you want a lazy policy, in the strictly technical haskell sense
@yacineMTB I'm not an ML researcher though. And I know nothing about robotics. Just your average web dev with too much time on his hands.
So take my advice with a grain of salt.