It's not annoyed. It's for safety. It's only take a few mins to stop at a safe location and input your destination into the GPS. You could always preset a location on memory which you can navigate or use voice to activate the navigation. Bypass is not a good idea.
Prestigious society sells a product to do this. I had one in my 2011 Sequoia and just put one in my 2018. Works great. Allows a passenger to access all the buttons while moving. In some areas where I drive (NYC, parts of NF, DC, Baltimore) stopping is not an option.