Have you adjusted the speaker's bass/treble settings in the Google Home app? Setting either too high can cause this (speaking generally, not specific to the Nest Speaker). Basically, turning up anything on an EQ is just raising the volume of certain frequencies of the audio. Most music is already at the max volume it can be at before it clips (the distortion you're describing). So raising the EQ values (bass/treble), you're raising volume for certain sounds to the territory of clipping. You can test this out by lowering your bass and treble.
Clipping is when the audio waveform gets cut off (think like taking scissors and cutting off the peaks and valleys of the waveform). The cut off parts don't have data for what the audio there should sound like, so it gets distorted.
Without knowing your setup or hearing it for myself, I can only offer this as a suggestion. This could not be it at all. But something to consider.
Another possibility is there's damage to the diaphragm of the speaker. This would be like the cone or material that pushes the air to create sound. Loud volume causes this part of a speaker (again, this is general, not specific to this speaker) to move harder back and forth and cause more wear and tear. A lot of moisture can cause damage, depending on the material, and too dry can too depending on the material. Unfortunately, you won't know if this is the issue without taking the whole thing apart and looking for damage.
It's also possible its however the audio is being processed, which you'll probably never know.
