It's probably the Home/Away detection. Look at the settings in the Nest app (or the Google Home app); these are general to the account, not in the thermostat settings themselves. There's a "how does Nest determine if you're home" setting, and in there, you'll have the phones being away from home (geoloc), and the Nest thermostat not seeing you move around. You can switch either/both off.
I wanted to stop by and see how I could help.
Thanks for the input, @Zouf.
@busdriver01, in addition to what was mentioned by Zouf. You might have had a schedule temperature; try clearing it all or editing if needed. If you have the learning thermostats (Nest Learning Thermostat 3rd Generation and Nest Thermostat E), make sure to disable the auto-learning schedule. You can find these settings in the Nest app or in the actual thermostat itself. Also, turning off the Home and Away Assist on your thermostat settings might help.
Please let me know if that helps or if the issue still persists.