The data is for device heartbeat. We need to keep the device alive for future connections. All three servers were verified to be our P2P provider’s servers. We have some articles about using ThroughTek as our P2P service provider. They are at https://support.wyzecam.com/hc/en-us/articles/360027279571-ThroughTek-User-Privacy. From your data, the average packet size is 585 bytes. There is not too much we can collect with 585 bytes (including all the packet overhead).
We are working really hard to protect customer’s security and privacy. It is the whole company’s goal since we formed the company. I will respect your decision if you want to keep or return your camera. No matter what we will keep working hard to deliver our promise to the customers. Thank you and have a great day!
@WyzeTao,
Thanks for chiming on on this thread. I appreciate the link you posted to the ThroughTek FAQ as it has some useful commentary on the system architecture.
I’ve looked at the firewall data provided in post #16 above and find the numbers rather surprising. According to @gelly035-happy, the line stats are for a single V2 camera that had no media flow (“… no one was connected to my camera and object/event detection OFF”). The screenshot shows nearly 75 MB of data transferred over 11.5 hours. Here’s how this breaks down:
on average, a constant data flow of around 14 kbps (kilobits per second).
the packet rate is about 3 pps (packets per second).
average packet size is 585 bytes.
You indicated that this data flow is required for “device heartbeat”.
I fully understand the need for some sort of heartbeat in the Wyze network architecture. The primary reason is to maintain the NAT mapping in the Customer’s router so a P2P connection can be established when the owner wants to view remotely. But sending 3 packets per second to maintain the NAT mapping seems extreme. Typical timeout for a TCP mapping is 10 to 15 minutes, sometimes even longer. If it’s a UDP flow, the typical timeout for consumer routers is 30 seconds. I can’t imagine any legitimate reason to send a ‘heartbeat’ packet every 0.33 seconds.
The size of these heartbeats also seems excessive. It does not require a 585 byte packet for a keep-alive. The NAT mapping can be maintained with single minimum-size packet, transmitted much less often than 3pps. You indicate that “there is not too much we can collect with 585 bytes”. Perhaps not with a single 585 byte packet. But according to @gelly035-happy, the V2 camera is sending three such packets every second. That’s over 6.5 MB per hour. That’s a lot of customer data that could be getting exfiltrated, one 585 byte packet at a time.
To be clear, I have no reason to believe that Wyzecams are exfiltrating Customer data. But the mere fact that these large and seemingly unnecessary flows exist is a concern to many.
I hope you can explain (a) the reason for such a high packet rate for the ‘heartbeat’, and (b) the reason the packets are so large.
On a practical note, assuming gelly’s firewall stats are correct, the V2 camera is sending around 150 MB per day, or 4.5 GB per month. That’s an awful lot of data for an administrative heartbeat, and may result in internet overage charges for Customers who don’t have the luxury of an unlimited internet plan. And I shudder to think of the the traffic flows hitting 23.83.132.78 from hundreds of thousands of Wyzecams sending these heartbeats at 3pps. Deployment of 100K cameras would hammer that server with over 1Gbps just of heartbeat traffic. That would overload a GigE ethernet link. As a system engineer, this seems to me to be a sub-optimal network design that may not scale gracefully…
I wonder if there’s an anomaly (a.k.a. bug) in gelly’s Wyzecam such that it was constantly transmitting media of some sort even though all such features were apparently turned OFF. That might account for the 75MB of data reported over 11.5 hours.
@WyzeTao can you explain what “future connections” mean? having that open effectively gives you access to the camera remotely and on a potential scenario where your servers get hacked, pretty much the hackers have access to everyones LAN through the wyzecams.
I am paranoid yes, understand that you guys strive for security, then please give us an option to use the cameras within our LANs and in private networks. It is an option that will bring this product to the next level in terms of privacy.
You do this already with RTSP, it can be accessed without internet. However 10FPS is unusable, need at least 20FPS. Maybe you will need to do a better port of the RTSP GNU/Open Source implementation which is likely what was done.
In summary:
Significant data is being transferred silently from your network to the internet
Unknown/Unpredictable “future connections” from Wyze servers to your network exposing your LAN to who knows where/who
Assumptions that you will never be hacked…
No option to give the user privacy or use in private networks or firewall environment
The day Wyze servers experience a glitch everyone will be down
My two wyzecams are boxed, ready to go back to Amazon. I will stay tuned, in case you come with a LAN only option or better FPS with RTSP that allows me to do that, then i will go back and buy the cameras. I am willing to pay up to $40 for a version of these cameras with those features. The app is good, the quality of the video is good and it is a small camera that fits everywhere. But the requirement to be connected to the cloud and no option to use within a firewall environment is a killer for me.
On top of that there is the reliability problem. Servers go down, and all your camera users will go down with it. Google, Amazon and Netflix servers experienced these problems and they “own” the Internet let’s say. The assumption that it won’t happen to you and leave all your users in the dark is a major one in terms of this future. Raise the prices a little and deliver a more holistic product, people will still buy and you’ll get glowing reviews.
I’ll stay tuned to these forums to see if anything changes
The cameras are intended to be accessed by your phone, which could be your local network or anywhere your phone goes. After a camera is setup the camera will periodically tell our servers and say ‘I am alive’. When an app tries to connect to the camera, it talks to the server first to find the device (only the server knows where the device is and create a connection from internet to the camera). That is what I meant by ‘future connections’.
There is one exception when you try to connect from home. Our app will broadcast in local network and search for the device as well. Once your camera responds a connection can be made without our server.
You have a good point. I only did the math for the size of the packets but not the # of packets. This needs some digging. I will get back on this. Unfortunately I won’t get camera log since the device is taken down. Otherwise it could be easier to figure it out.
@gelly035-happy Can you confirm you didn’t perform any streaming tasks since the very beginning? Thanks!
the age of the connection is 11 hours. So it remained open for that amoung.
when the firewall rule is on, you can’t do this. THe camera somehow needs to conenct to the internet to validate LAN usage. If this was the case i wouldn’t be here discussing this.
it’s 10 FPS, at least based on the last RTSP version on your site. You confirmed that is 10 FPS on the other thread due to “night mode” so 15 or 10? i see 10
@WyzeTao,
I have observed this behavior - it’s a clever design, as no internet bandwidth need be consumed for viewing the camera if the client (the app) is local. However, your search for the device doesn’t work if there is no internet connection. Even if the app (smartphone) and camera are on the same WiLAN and the same subnet, the camera does not respond to the search if there’s no internet connectivity. So the app cannot connect to the camera. No Live View. No viewing of Events recorded on the SDcard. No functionality of any kind. Is that a bug or by design?
It’s not so much the # of packets, but the packet rate that is of concern. The 3 packets per second over 11 hours reported by gelly is tremendously excessive (wasteful, expensive) for any kind of a ‘heartbeat’. To keep the NAT mapping in the router alive, a packet every few minutes should suffice.
More math for you to ponder: the 3pps rate is for the connection he reported to 23.83.132.78. Take a look at the flows to the other two servers (91.132.137.155 and 66.23.205.130). The firewall log shows a packet every 3.5 seconds or so over those 11.5 hour connections. That’s an order of magnitude less that what’s hitting 23.83.132.78, but an order of magnitude more than what should be required for a keep-alive.
How long did you run after interruption of the Internet, @Loki ? Recent tests I did indicated the Internet was required for viewing the live stream point-to-point inside your house. I found that if the Internet wasn’t connected you would lose contact with the cameras in a little over 3 minutes. Once you lost authorization, you had reconnect to the Internet to regain it.
It was obvious the stream wasn’t going external, as I got to watch all cams for 3 minutes. But it appeared there was an authorization heartbeat that goes out every 3-4 minutes that would stop the steams if I didn’t get reauthorized.
I did not run it that long, although I had not connected to the cameras for several hours prior to this test. I guess I’ll have to test again and let it run 5 or 10 minutes.
My recent tests had the same result as that reported by Newshound (and not what was reported by Loki). Absent a connection from my WiFi router to the internet, my app (iOS version) can’t connect to the camera, can’t view Live Stream, can’t configure the camera to do local recording to SDcard, etc with direct on-LAN connection. However, my iPhone can ping the camera. The camera is on my WiLAN with an IP address assigned by the router, and has IP connectivity with the iPhone. But the lack of internet connection to WyzeHQ is preventing the camera from interacting with the app.
I’ve done another similar experiment to further investigate what if anything works without internet. With normal internet connectivity, I configured a V2 cam to do Event recording to SDcard, (but no recording or notifications to the cloud).
Powered down camera. Disconnected internet from my WiFi router. Then booted up camera. (this use case replicates taking a camera that’s been configured for local recording to an off-net location). It happily associated with WiFi and obtained an IP address. I left it for about 30 minutes, and provided lots of motion-action in front of it to trigger event detection. After restoring normal environment (with internet), I found that nothing had been recorded to the SDcard.
I’ve concluded the lack of internet connection to WyzeHQ inhibits it from doing any local recording to SDcard (as well as preventing Live viewing from within the LAN). I’ve pretty much given up on Wyze for any kind of offline (no internet) applications.
Let me modify that a bit. SD card is the only thing that still worked in my tests, but it was because the camera continued to be powered after I removed the Internet. So the SD card routines still had what they considered as authorization to continue (not like the live stream that needs authorization every 3-4 minutes).
The difference between my test and kyphos is kyphos powers down the cam, moves it to a location without Internet, and powers it back up again. In that case the SD card routines can not get the initial authorization to proceed, and no recording to the SD card will take place.
kyphos needs it to power up in a remote location without Internet for their use of time lapse to work. Most of the rest of us are blessed with cell phone hotspot coverage where we travel, so we can get that initial authorization.
Newshound: Point taken. Your summary is bang on. I’ll try to be more precise with my comments.
“I’ve concluded that the lack of internet connection to WyzeHQ inhibits it from starting any local recording process to SDcard”.
However, a design that requires continuous power-up operation to maintain functionality is what a system engineer would call “brittle”. A cam can lose power for any number of reasons. A wobble of the microUSB cable. A momentary glitch in the mains power. Etc. So even if one can obtain the initial ‘authorization’ via cellphone hotspot, any ongoing local recording will fall over and die if there’s any loss of power. That’s brittle.
I’ve noticed one of my cams rebooting spontaneously on its own from time to time (I hear it clicking as it starts up), and I’m sure that would also cause it to lose what ever ‘authorization’ is required for local SDcard recording to continue.
Frankly, I don’t understand why any kind of real-time authorization from WyzeHQ is required for local SDcard recording to commence. There’s no security risk to having the camera do what its owner has configured it to do locally with the SDcard. The authorization imperative arises for remote features (Live View from afar), and transmitting 12-second clips and notifications to the cloud.
I just re-did my (flawed) test. Here’s what I did and found:
I unplugged my router, modem and camera. Restarted the router but not modem. Then re-powered the camera. Could not connect to the camera.
Plugged modem back in. Once internet was available, camera connected.
Unplugged the modem. Camera continued streaming for 3-4 minutes at which time its light started flashing blue and the stream froze. But after a minute or so, the stream resumed and ran for over 15 minutes. This is with the app continuously connected to the camera.
I then went back to the home screen in the app and tried to reconnect to the camera. That failed.
So I have to agree, local streaming is not possible except in cases where the stream was started in the presence of internet accessibility and the stream is not interrupted if the internet is not accessible.
As you know, my objective has been recording to SDcard in the absence of internet connectivity, but I certainly have used Live viewing when on location (i.e., off-net) to make sure the camera was pointed correctly, was level, etc. That was part of my setup prior to starting local recording to the SDcard (whether event driven, continuous, or time lapse). This all used to work quite well, and made the Wyzecan a very useful device for field recording. But no more.
Your latest test adds some useful pieces to the jigsaw puzzle of what works (and not) when internet access is not available, or ceases to be available:-)
Hopefully @WyzeTao and friends will be able to reassemble the pieces and restore off-line functionality to what it used to be.
I have a Pan Cam with the RTSP FW installed, and can confirm the same behavior that the others have experienced. If the cam is on a LAN w/o internet access the ONLY way to view the cam is via RTSP. Attempting to view or connect to the camera via the app only works if an active Internet connection exists.
I hate that everything is so awesome, and then it’s hobbled by the lack of a locally stored token in the APP to authenticate the APP: camera handshake when trying to connect and use the PTZ functionality of the APP on a LAN w/o internet connectivity.
If this was fixed I could get my company to deploy a bunch of these at work, And that’s even if the price tripled.
Just want to update that I didn’t forget this thread. I am waiting for an answer from ThroughTek since the heartbeat is a proprietary protocol. I will update when I get their reply.