svykseler Technically, both of you have valid points, but there's a much cleaner middle ground. @RosalindEverly-n4h is right that changing the boot order is "proper," but you are entirely correct about the file system state being safe to unplug.
However, the absolute best practice to avoid this entirely: Leave your internal drive as #1 in the BIOS permanently, and use the Motherboard's One-Time Boot Menu (usually hitting F12, F8, or F11 on startup) to select the USB just once. That way, when the PC reboots, it naturally goes straight to the hard drive without any physical pulling or BIOS rearranging.