It might be too early to tell, but I disabled all the "preferred
activity" settings. I had video conferencing enabled. So far it's been
more stable for a few days at least in my case.
I have been experiencing this issue as well, I noticed it a few weeks
ago. Device download speed drops significantly and other devices have
trouble maintaining their connection. I restart the network through the
Google Home app and its back to normal...