I've been using a similar setup, but based on Dagger. I don't have it as a proper dev container (yet), but I get a lot of the vscode benefits by presenting it as a virtual filesystem and terminal. I can make fully custom envs that myself or my agents can use, we can even share the same ones and see each others changes.