Google Nest Community

In my GoogleTV chromecast there is a setting under system - cast I found this. Not sure if it may solve your problem but I would turn this off on each chromecast/googleTV and see if it changes anything? I could be completely wrong but might be worth ...

Do you not have to push cast to start casting something? Therefore you have control on what is being casted and where and when?