This is unfortunately just how these devices are right now. With huge LLMs like ChatGPT and even Google's own Gemini (bard) LLM not only making headlines but also being available for public use for free, it is a huge letdown that Google specifically have yet to push this technology to one of the places they would be the most useful, your smart speakers.
How the assistant works right now is quite lame in the current world of large language models that can do far more than you could ever imagine, but on the other hand it is important to realize that the google assistant is a service that has actually existed for a long time before the incredibly smart and useful LLMs we know now have come around. It was quite advanced for its time, allowing you to be fairly conversational with it, while understanding what you are saying consistently and replying quickly in a fairly realistic voice. That being said, it is really showing its age Google, the old system of a few set keywords and your heavily locked down system are looking more and more disappointing with every new advancement in AI.
If that was a confusing response, let me word it this way: Yes, google did not do a very good job with the "quick question" feature, (I just made up that name), they clearly should have made it far more likely for the "smart" assistant to decide that if it doesn't hear one of it's key words like "turn on the ___" or "what's the weather," then it should then realize that what you are asking for should fall under the google search function, where it runs off a quick search and reads off the first result it finds. If you want to try this function for yourself, there are some ways of wording your question so that it does understand and realizes to use this feature, like how if you say "Who is Elon Musk," it will read off his Wikipedia description.
That being said, looking at what Google has been working on, like combining Google Bard and it's other AI products into one Google Gemini, Google's new Gemini 1.5 model with a ludicrously large pool of context tokens, and their research into multi-modality, (the ability for a large language model to interact with other mediums of information like audio and visual input and output,) it is very clear that Google is getting ready to become a world leader in AI technology, and since Google already has a smart assistant and a huge range of integrated products (like google search, gmail, calendar, etc,) it is therefore also quite likely that the google assistant will be receiving a revolutionary overhaul over the next few years to create a truly scifi "artificially intelligent personal assistant."
TLDR: Yeah it sucks, but the devices you have can still be quite useful if you learn to use them well and you don't have too high expectations, me and my family mostly use them to play music, check the time, weather, and temperature, and I have filled my bedroom with smart home devices that integrate well giving me really awesome momentary, granular, and even automated control over my bedroom's lighting, heating, and airflow. Sell/return them if you want, but me personally, I wouldn't.
