I'm having the same problem as Andybrad, who posted a few weeks ago. Sometimes, people come to the door and there is no alert of any kind. The Nest cam looks out the window right above the door and most of the time I get nearly instant alerts of a person in my red zone (the alert zone covers the whole front walk) and an email sent to me, exactly as I have Nest configured, but occasionally a person walks right up the front door and Nest is silent. After extensive communication with Nest support without any resolution I have come here.
I can review the video and these people are all perfectly obvious (to the most casual observer), yet no detection occurs. Support has suggested things that really aren't relevant concerning the two cell phones connected to the account. These phones could be turned off and the email of the detection should still be sent. There is some basic flaw in the detection software.
I have several captured video clips to show examples of how detection is failing.