Next time it occurs try running an internet speed test from the Google
Home App. Then try trying a test from a client again.There seems to be a
bug in the recent firmware and running the speed test "resets" whatever
slowdown there is. It does however...
Next time it gets slow run an internet speed test from the wifi device
in the Google Home App. Then test from our device again. May be similar