The problem is the chromecast only has the one output, hdmi. So you need a devic that can receive the audio from the hdmi and then outputs the sound. The only 2 common devices to do that are a TV & a stereo receiver with an hdmi input. But neither of those would make sense to use if trying to play on a google speaker.
And, as youve probably seen, when you try to cast a video stream you cant select an audio only device.
So, Im not sure this is possible, wirelessly. If you speaker has an input (headphone jack type) you could use your phone (if it has a headphone jack) to connect the two devices with a male-to-male audio wire.
@MrMillyMike, thanks for the help.
@Ole3105, thanks for posting. I totally understand that this is a good feature to use with your Chromecast and Nest speaker, but this isn’t available. I'd suggest you submit feedback about it, here's how.
Let us know if you have other questions and concerns otherwise, we will be locking this thread after 24 hours.