RL Foundation Models Are Coming!

This is a really high quality video, on par with 2 minute papers but with a more detail oriented approach. Also you have a lovable vibe king, keep it up

edan bro makes my dopamine policy gradients high everytime. fingers crossed we get open rl foundation models.

Just give this environment to speed runners, watch the true potential of what humans can do with games. Thanks for the video!

Another way to frame the problem of neural network representations becoming “too specific” to learn new tasks at 25:59 is to consider exactly how the gradient of weights is computed. It’s the matrix multiplication between the directional error after a layer and the directional values before the layer. When the values become totally orthogonal to the error (they contain no information relative to the error), then it’s impossible to reduce the error by changing the weights in that layer. The reason weight randomization helps with this problem is it introduces new values after the layer that was randomized. However a much more efficient way to do this is to instead reduce the existing weights in a layer with linear regression over a representative sample of data to “pack” the good information into fewer existing neurons. Then you’re free to randomly initialize the remaining neurons, or even better to initialize weights that produce values already aligned with the directional error! I’ve got some ongoing research in this area if anyone is interested in collaborating. 🤓

amazing breakdown, thank you for making this paper accessible to me!

At 7:10, the first pronounciation of Muesli is right. German Müsli, Muesli may be the Swiss-German spelling.

Really love it !

Been thinking about this for some time

sounds like RL is progressing? maybe I should jump back in !

since wandb doesn’t work for me i will actually try clearml thanks to you

Thanks for your videos, but at 7:44, efficient zero and mu zero do not reconstruct the raw observation/image, mu zero learns it’s latent representation based on value equivalence only while efficient zero also cares about temporal consistency, so they take next observation to supervise the representation and dynamics part of the model in an unsupervised manner(simsiam)

Good stuff

Wow x)

Coffee is culture too!

I wonder if there is any benefit to be had at all from, like, across multiple full training iterations, distill a large model into a smaller one and then distill the small one back into a larger one (vs. just repeatedly distilling a large model into a model of the same size)

My 2nd petition on this matter. Please make a video of how you read and implement papers. Thank you *kiss*

Do you think the approaches here could be applied to Dreamer V3?

If you're ever interested in collaborations, let me know. I'd love to have you on my newsletter to cover some of your most interesting ideas.

I really liked vscode theme on the clear ml section. Can you share it?

7:10 Myu-slee. It's a quick, easy and tasty breakfast so that you too, can be reinforced!