Verizon Broadband router with voice BRV - Novatel T1114 problems

I live in a fairly rural area, so Internet connectivity with my primary ISP is occasionally spotty. Because I work at home most of the time, I thought I'd improve the quality of my Internet connection by getting a backup link. I already have a Cisco ASA 5510 in my house, and it supports dual-WAN connections very nicely.  So I procured the Verizon 4G LTE Broadband Router with Voice, the BRV which is actually made by Novatel, their model T1114. I already am a Verizon Wireless customer and have a family plan, so adding the device was a no-brainer at a $0 upfront fee and $20 per month to share my existing data.

I should inform you at this point that I confirmed 4G was good at my house. I use the Wifi hotspot feature on my DROID MAXX all the time to broadcast wireless so I can connect to the Internet with my laptop during primary Internet outages. It works quite well indeed, so a router like this should only be better than my little phone.

I received it just a couple days after ordering and was pretty excited to get going. I placed it on my desk and plugged it in. It quickly connected to Verizon and within a few minutes I could use the handy buttons and display right on the device to see what the WIFI SSID and password were. I saw the SSID on my laptop, connected and boom. I'm online. It couldn't be any easier. Seriously!

The next step involved a little testing. I went to my favorite site, speedtest.net and ran some tests. Impressive! I easily received 20Mbps up and down, right here in the countryside. Who could be disappointed with that? And even though the ping time was in the high 80ms to google's 8.8.8.8 DNS server where my other ISP is 20ms, it's better than my primary Internet connection for speed. Yeah, yeah, I know Verizon charges a rather hefty per GB fee, but that's okay. I have 50GB on my plan which I barely touch, and since I'm only going to route through the device when my primary Internet is down, I should be just fine.

I typed verizonbrv into my browser and was greeted with a login page for the device. I entered the password I received from the device's screen and voila, I was in.  I was quite pleased to see that just about every configuration option is now at my fingertips, so I began to click around and easily changed the LAN settings to a more desirable IP range and specified a DMZ IP which I subsequently attached to my Cisco ASA.

I know Cisco ASAs fairly well, so I connected a CAT6 cable from one of the ports on the back of the Broadband Router into another port on the ASA and configured it as a backup route. I set a ping check on my primary to Google DNS and when it fails, to update the route to the other. I won't go into all the gorey details of setting up the ASA in this post. Since I don't have any published NAT connections since there is nothing at home to reach from the outside (sorry folks, no web or mail servers here for you to hack into :) I was done.  I pulled the cord on my primary link and within 10 seconds I was running on my backup.

I use VOIP here at home, so I made a few phone calls using my Vonage connection and couldn't be more pleased with the audio quality through the Verizon 4G network. I really wasn't sure how it would handle VOIP, so this was a very pleasant surprise. In fact, just to be sure it was solid, I had a good 15 minute conversation with my friend and never did it gripe. Crystal clear the whole time. Brilliant!

So what could be the "problem" you're alluding to in the title, CJ? Well, after about a week my primary Internet died, which usually happens every week or two. Everything switched over nicely to my backup, but I noticed that as I surfed the web I would get frequent blank pages. So I ran a constant ping test using my favorite tool, Pingplotter Pro, to a dozen locations. What I discovered is that the pings all timeout at the same time, for about 3 or 4 tests.

So I logged back into the BRV to see what was going on and what appeared to be happening is that the IP address was disappearing, and then a new one came back precisely every minute or two.  I enabled the logging and it confirmed that it was losing WAN connectivity every couple of minutes. Here is what I saw:

[L_WAN] QmiEvtConnStateHandler: IDLE
[L_WAN] QmiEvtDevStateHandler: WAN_STATE_IDLE
[L_WAN] ModemEventCallback call dropped
[L_WAN] Device state: WAN_STATE_IDLE
[L_WAN] Performing auto-connect base on service
[L_WAN] validate_roaming_for_call: current roaming pref = 0xff
[L_WAN] validate_roaming_for_call: current roam status: ccm=0x00
[L_WAN] validate_roaming_for_call: Roaming check is okay
[L_WAN] Modem_StartCall: Modem passed mininum service time
[L_WAN] Starting Call: prof=3, apn=, user=, pwd=, auth_pref=0, tech_pref=0, ip_family=0
[L_WAN] Data call started
[L_WAN] start_network_interface_callback: call_handle = 0x43d86850
[L_WAN] start_network_interface_callback: end_reason = 0x00000000
[L_WAN] start_network_interface_callback: verbose_end_reason = 0x00000000
[L_WAN] QmiEvtConnStateHandler: CONNECTED
[L_WAN] QmiEvtDevStateHandler: WAN_STATE_CONNECTED
[L_WAN] ModemEventCallback call up
[L_WAN] QmiEvtConnDormHandler: NOT DORMANT
[L_WAN] Device state: WAN_STATE_CONNECTED
[L_WAN] Dormancy: 0

Well... interesting stuff. And if I wait a couple minutes, the entire stream is there again and again. So why WAN_STATE_IDLE? It wasn't idle? I was using the Internet, and my pings were going through too.

So I took a look the diagnostics tab and saw a signal strength of -91db with an SNR of 7. Perhaps that was the problem... but why now and not before? So I Took a look on verizonwireless.com and found the Proband Repeater Antenna. After looking at it and reading the reviews, however, this did not seem to be a good fit. Many users complained that it did nothing to improve things, and in my opinion, it is an indoor antenna and I need a big outdoor one to get the biggest boost possible.

So I went to my favorite store (amazon.com) and ordered a very nice Wilson Yagi Antenna, ensuring that it operated at 700Mhz, the Verizon 4G LTE frequency. This is important, because I almost ordered another Yagi antenna that was 850Mhz and up, which would have surely disappointed. I added to my cart some connectors, 50 feet of ultra low-loss Coax with N-connectors, a surge protector and waited patiently for prime shipping to deliver it within 2 days.

To my delight, Amazon delivered the coax cable the next day, so I began my install project right away. Being a complete geek, when I built my home, I ran conduit from my lower-level office's storage closet to the roof. And because it is a nice flat roof, it's a breeze to get everything in place. So the following day when the antenna arrived, I installed it on a pre-existing pole (I have a CDMA booster in my house which was already up there... and it works great, BTW), tightened up the connections and attached it to the back of the BRV down in my office. Incidentally, the BRV sits on the top of my ASA.

At first when I logged into the device I saw zero signal gain. What!?  But I had a hunch what the problem was. On the back of the BRV are two antenna connectors, and I connected to the top one, which obviously must have been for the voice (1x) network, which I haven't even subscribed to. So I relocated it to the bottom connection and voila. A 11 db gain. Now I was sitting at -80 with the occasional peak to -79. That's pretty darn good, right?

Back to testing! I pulled the connection on my primary link in order to throw everything onto the backup. I did the same speedtest.net tests and constant pings and while the speedtest continued to perform well, my constant pings continued their periodic timeouts, and surfing the web continued to be interrupted by the disconnects.  The BRV was still losing the IP address every few minutes... and to make matters more interesting, every time it regained an IP, it was a different one.

So I called up the support folks at Verizon for some assistance. I didn't have much time since I had a conference call looming in about 30 minutes, so I ran through some basic troubleshooting with the individual I was speaking to who was obviously reading from a guide for this particular router. Everything he suggested was stuff I already tried, but for posterity and to amuse him, I did it all again. He suggested boosting the signal with "any kind of antenna, it didn't have to be a Verizon one" and I said I'd give it a try and call back because by that time I only had a few minutes left.

The following day I called into the Verizon phone system again. Interestingly enough it detected that I had an issue the prior day and asked if I was calling about the same issue again. That's handy! So I pressed "1" for yes and was routed to a much more intelligent individual. Perhaps their phone system knows that if you're calling about the same issue again, they better up the ante, so they did. This fellow seemed to know a lot more about this device and I had an intelligent conversation with him.  We went through some additional troubleshooting including a full system reset and concluded that one of two things needs to happen. 1) We replace the device to see if that will solve the issue, or 2) He opens a ticket to have someone check our area to see if there are 4G issues and/or if something on the tower in my area is slightly off.

Because 4G tests via my phone work very nicely without ping loss, I opted for option 1 and requested a new device which would arrive in 3 to 5 days.

Well, 2 days later the replacement device arrives. Because it does not have a user swappable SIM card, I had to spend about 30 minutes on the phone with a lovely lady at Verizon to get it working. But she was very kind and patient, and so was I. So we got the job done.  Well... you want to know the results, don't you? After getting everything working and configured again, it still disconnects. This time it doesn't disconnect every couple of minutes, it lasts a little longer, but the same thing appears in the logs roughly every 5 minutes or less and continues to annoy me.

What gives? I guess I'm going to call Verizon again and go with option 2 of having someone check this area. I'll keep you updated! In the interim, I sure hope you don't have the same issues with this device.

C.J.

UPDATE: 05/31/2014
I spoke to Verizon this morning for quite some time. They insist that the "way I'm using it" is just abnormal and additionally, the disconnects are quite normal too. In a 24 hour period, they saw that the modem disconnected and reconnected 1558 times but only 8 of those times were "bad" disconnects. Therefore, based on the fact that my usage is abnormal and that the percentage of "bad" disconnects is significantly less than 2% means they cannot escalate this to anyone.

So you're curious what I do on the Internet now, right? Well, I ping things, I surf the web, go to Twitter, Facebook, News websites, remote desktop to different servers to work on them etc. Does that sound abnormal? Apparently Verizon thinks so.

So I asked if he thought 1558 modem disconnects in a 24 hour period was abnormal, because that only confirms how frequently it is cutting me off. He said that was quite normal because it is a wireless network and there is no guarantee of constant connectivity.  Sure, I'll give you that, but every few minutes a disconnect.  But this is the real kicker, in my opinion. When I use the mobile hotspot on my phone to share the 4G Internet with my laptop, I can run a constant ping to something like Google DNS servers all day long. I can also keep an open RDP session to a server all day. Via this device, that does not happen. Every 2 to 3 minutes the pings stop for 4 to 8 consecutive tests and my RDP session disconnects.

Your disconnects are abnormal, Verizon, not my activity.

More soon.
C.J.

UPDATE: 06/15/2015
I'm so sorry it has taken me a year to update this post. Thanks to a comment today that triggered an email to me (and apologies to the prior commenter, I guess the email from blogger got stuck in my spam folder)... I have an update, and it is not very exciting.

I was waiting for the Novatel folks to release a software update for the device to fix this apparent bug, but in over a year, no software update has been released. Trust me, I check often. This is somewhat typical and reminds me entirely of Samsung. Once a device goes to market, they release one or two updates to fix bugs, but never any updates to release desperately needed new features.  They would MUCH rather you purchase the next model instead. But that is the subject for another rant, I suppose : )

So to make a long story short, I have determined that the router detects the double NAT and restarts the 4G connection as a result. I have a hunch this is to stop people from getting around the 10 device limit. It is somewhat annoying that the ASA5510 doesn't appear as a single device to the broadband router. I'm not smart enough to know why that is the case, and I haven't been able to test other firewall devices since I'm married to Cisco (I know, I'm sick) primarily because I have a PTP VPN between my home and the office and it really is a pain to get a Cisco firewall (at the office) to talk to a non-Cisco firewall elsewhere (like my home). Trust me, I fought with many sonicwalls a number of years ago. Ugh.

So with that said, I researched other devices, and Verizon does have a solution for this. They call it their M2M or machine-to-machine router service. It is compatible with a bunch of very popular router manufacturers, including Cisco, but the issue is that you can't get just one of these devices. A business account (which we have) needs to get 5 to start. So that solution is not a solution at all for the guy who works at home and only needs one device. Plus, the bandwidth per GB seems to be a little more expensive too. Makes sense, that is their cash cow at the moment.

So, there you have it. Other than all of this, when I do use the Verizon 4G (which is all the time, I have my VoIP phone plugged into it permanently) it is great. The service is excellent, stable and reliable. I've had one bad call in a year, which is beyond impeccable.

C.J.