New Zealand’s largest Telco and Internet Service Provider has suffered one of the worst internet outages ever experienced in this country. Reports started coming in between 9pm - 11pm on Friday night of internet connections outages, websites dropping offline and widespread email outages.
The impact on customers of Telecom New Zealand was swift with not only home and business broadband internet connections failing, but major brands such as Dick Smith Electronics and Tip Top losing their online presence as their websites became inaccessible. Numerous other internet based consumer and business services were also impacted.
Some 17 hours after the failures began issues were still being reported. Just after 4pm on Saturday Telecom's Managed Service Centre were advising of a number of outstanding issues (including email service and static IP addressing) with no estimate able to be provided on when they would be resolved.
One Telecom customer, the organisers of the popular Parachute Music Festival found their primary internet IP address had changed leading to email, remote access and Blackberry service being lost even after their Internet connection had been restored by Telecom. After adjusting settings to ensure the correct path for incoming/outgoing email it was found that Telecom was blocking email traffic.
It seems today at least, that Telecom's motto of 'Connecting New Zealanders' is a good idea but not quite a reality.
Fortunately for Telecom NZ stockholders, the outage was outside of usual business hours and and at an off-peak time for home users. Had it occurred 12 hours earlier it's impact would likely have had an immediate impact on the stock price of Telecom which was up 4.42% in trading Friday to NZ$2.49. The Friday prior it was trading as low as NZ$2.20.
Update, 8.38am, 30 Nov: Some ongoing issues continue and this is confirmed by Telecom's Managed Support team. (Faults have now lasted for up to 36 hours)
Update, 9.41am, 30 Nov: Most authentication issues and email issues now appear to be solved.
Update, 10am, 1 Dec: Mainstream media have given this outage very little coverage today. Most of their reporting was early afternoon Saturday saying that broadband Internet connections were restored between 4am - 5am Saturday - no detail on the issues that continued for another day or so. Major IT industry publications Computerworld and PC World don't have front page stories on their websites either - probably because their own websites were offline during the weekend (despite them not being hosted by Telecom).
Other related posts:
Live Event–Tue 10 March: NZ Tech Podcast with Dai Henwood, Michelle Dickinson (Nanogirl), Paul Spain
Sneak Peek: HP Envy X2
Want to start a career in IT Support?
Comment by cokemaster, on 29-Nov-2008 17:52
"One Telecom customer, the organisers of the popular Parachute Music Festival
found their primary internet IP address had changed leading to email,
remote access and Blackberry service being lost even after their
Internet connection had been restored by Telecom. After adjusting
settings to ensure the correct path for incoming/outgoing email it was
found that Telecom was blocking email traffic."
If they had a static IP, it shouldn't change when it came back online. If its a non static, it would have changed when the modem came back online. If you are running any type of remote service like remote access or email, then a static IP is a must.
For email, if you're running your own mail server, then opting out of port 25 filtering is a must (by default port 25 filtering is enabled for any connection unless its using an username that has opted out).
I found it annoying at the time but was able to find other things to do - I was going to pop online to play some BF2 but had some issues, was still able to browse geekzone until I restarted my modem :(
It was good to see the landlines and mobiles were still working... it would have driven me crazy otherwise.
Comment by Zippity, on 30-Nov-2008 22:26
pressf1.pcworld.co.nz is off line and has been for many hours.
Telecom should hang their head in shame - but of course the bastards will deny everything.
Add a comment
Please note: comments that are inappropriate or promotional in nature will be deleted.
E-mail addresses are not displayed, but you must enter a valid e-mail address to confirm your comments.