The Tale of Thor’s angry electrons

Background

I was spending Christmas with relatives on the western coast of Norway. A part of Norway where foul weather is no stranger and heavy rain is the norm. Where the Vikings learned to handle the waves of the Atlantic ocean next door.  Thus we are no strangers to voltage spikes. Power outages due to Thor’s angry electrons are quite common. It can be pretty, but such evening skies are usually a harbinger of bad weather.

2016122516-16-51-50

And surely it was. The next morning we were hammered by gale force winds. I would rate it as a medium storm, but the meteorologists gave it an official name (Urd, and old Viking female name) and called it extreme weather. The second night of the storm I was awaken by a loud crack from the direction of the intake breaker box located in the guest room, followed by thunder. The howling from the UPS in the server closet revealed a power outage. I waited for about 10 minutes, but all I could hear was the storm. The reason for waiting is this: If there is one place in the house you do not want to be during a lightning-strike, it is with your nose in the breaker box trying to get the power back on.

After a while the lack of electrical heating won over my concerns for further strikes, and I went to look at the main breaker box. I was expecting one of the breakers reduced to a pile of rubble, but everything looked OK.  If you wonder how I found my way through the darkness, let us just say that you do not grow up in this part of Norway without learning how to find one of your many flashlights in the dark. As all seemed OK in the intake box, on I went to investigate the distribution box in the next room. This is where the main distribution breaker, residual current device and surge protectors are located. Both the surge protectors and the residual current device were triggered, along with several circuit breakers. I primed the residual current device and switched it back on. Then I reset the circuit breakers, verified that the electric heater in my room was working and went back to sleep.

I was raised from my slumber by the users (i.e. my relatives) a couple of hours later. They complained about missing internet service and beseeched me to investigate. And investigate I did. For a normal residential house they have quite the advanced setup (I might be to blame for this), but it is made such to be resilient. After the installment of extra surge protectors some years back, the culprit is usually the ADSL modem. There are sadly no phone line surge protectors available that are powerful enough to resist the onslaught, so when the angry electrons enter the house through the phone lines they usually end up killing the modem. The ISDN phone connected to the same line has survived for more than a decade, but that is a German made ancient Siemens device unlike the chinesium crap the ISP calls an ADSL modem that is usually replaced twice per year.

A quick look at the modem revealed not a hint of status LEDs. Hoping for a quick fix in form of a power supply replacement I took it down from its mounting bracket, only to discover the unmistakable rattle of destroyed components from within. The VPN box was also dead.

image

I called the ISP and convinced them to send over a new modem. Due to this still being Christmas, it would take two days. Which isn’t bad, but usually we could get one the same day.

A quick summary of the components: The Wireless AP is there to provide a consistent WLAN. The modem has one built in, but each time it is replaced, the settings change. The VPN box is placed there by me to facilitate remote support. The DSL splitter is connected to the outside line and sends one signal to the ISDN NT1 and another signal to the modem. The NT1 is located in another room. The box on the lower right is supplied by the satellite TV supplier, and its function is unclear. It has some kind of wireless function, and I suspect it is a dedicated WLAN for the satellite decoder to call home.

Zyxel P8702N

But back to the modem. When you remove the top dark-out cover it looks like this:

2016122614-14-57-45

It identifies itself as a ZyXEL P8702N, which as far as I can tell is an ISP special, that is, only sold to ISPs. The hardware supports both an internal DSL modem and an external modem/adapter.

I was curious as to which components produced the rattle, so I removed the top cover. No screws, only fidgety plastic clips. Does not look like it is designed to be serviced. First glance revealed three separate confirmed problems.

image

1 – MNC G4804DG

This chip is a dual port gigabit ethernet line transformer. There are two of them, which correlates to the four LAN ports. There is also a WAN port connected to the G1806DG on the left. There are clear signs of carbon on the board, evident of a blue smoke leak. And as we all know, if the magic blue smoke gets out of the chip it stops working. This should have prompted me to investigate further at the other end of port 3, but more on that later.

2 – DSL line “protection”

image

The designers have tried to protect the modem from angry electrons by connecting the DSL line to a gas discharge tube and two in-line capacitors. As the picture clearly tells, this was not enough. As far as I could find out the capacitors are low quality chinesium, and I guess that goes for most of this box.

image

A close-up reveals further damage, even carbon on the connector itself. I would guess that the two capacitors were the source of the loud noise.

3 – Unknown chip

image

This could be the “modem” part, but it was to small and damaged to identify with the equipment I had available. The board shows a trail of destruction from the DSL-port down to this chip.

VPN Power

2016122721-21-25-44

image

All I could find is a broken 3-pin part, probably some kind of transistor. The power was luckily all that was broken, and a retrofit universal model from the local supplier brought it back to life. Local as in 50 clicks away, but I digress.

Network meltdown

I promised to return to case 1 from the modem. The one about carbon on the network interface transformer. After replacing the modem I quickly discovered that the server was no longer accessible. This is your typical small-business setup with one box running file, print, AD and accounting software connected to a couple of clients. There was sadly no time for pictures, but to sum it up, the angry electrons killed a HP Procurve switch and a network adapter in one of the computers.

Summary

All this from a single lightning strike far away. The angry electrons of Thor are not to be scoffed at.

Author: DizzyBadger

SQL Server DBA, Cluster expert, Principal Analyst

One thought on “The Tale of Thor’s angry electrons”

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.