Even with a direct connection to the commercial cameras I sell, install and maintain I have direct access over a private network and No Authentication.
It will still take 2-10 seconds to make a connection and display live video, this is due to a multitude of reasons such as the connection speed, pingtime, number of hops to server, etc. I have when remotely accessing the camera. These are professional cameras such as the WTI Sidewinder or Viper IP cameras, Or CoHu IP cameras.
Accessing a camera via internet (IP) is not like a Closed Circuit TV system with cameras video over coax, the video is encoded and compressed and then sent over a network where it is decompressed to be viewable. So even high-end cameras do not provide an instantaneous connection and at best maybe 0.7 seconds would be the fastest under ideal conditions, and this is without authentication,
1-6 seconds is very reasonable for any authenticated camera video in my opinion