3DR downtime last night

[Joe Siegler]

Well, on Wed afternoon around 3ish or so, we lost power. Seems some folks on a construction site across the street cut our power line by accident. We were on emergency power, so we had time to properly turn everything off, which we did. Some guys played hackysack in the lobby, some tossed footballs around in the back yard.

The back yard guys had it better, because one of them realized he had a cooler of beer in his truck, so we broke that out and started having some fun.

After awhile when we realized it wasn't coming back on, everyone went home. I drove by 3DR around 9PM and saw the building was still black, so I called our office manager.

Apparently they fixed the original problem, but the power junction box where the lines terminate at our building had a problem too, so they were working on that, and then broke one of the connectors. Had to fix that. Then I'm told when they fixed the second problem and got ready to turn everything on again, there was an explosion on the power box, so they had to fix THAT TOO.

I don't know what time they actually got it working again, as I haven't talked to anyone yet this morning. I went to bed last night at 11PM, as I was really tired - just got up and see everything is going again, so obviously it happened while I was asleep.

That's why we were down yesterday/last night.

[crunchy superman]

Thanks - I was wondering.

[Deftones]

lol Something like that happened to the company I work for a few weeks ago except for the beer part. :(

Needless to say I stayed behind making sure all the battery back ups were plugged in and working on our key systems especially our main server. We called in a power generator company to get us partially up and running till we got the electrician out to fix it. Took only 2 days but it was interesting needless to say. ;/

Good to see you guys back up.

[Joe Siegler]

Just found out the servers came back up around 1:30AM today. That was one good outage because the construction guys cut our cable. :)

I said at the time I should go complain - not that they took us down, that they didn't do it right, and knock us out for a week. FREE VACATION! YEAH!

[ShadeEX]

Ahh so that's what happened.. Glad to see your back up again..

One December in Denmark we had a storm that knocked out our power for several days.. Police also ordered people to stay indoors..

So we didn't really have anything to do, other than play board games, cards or similar stuff.. Those days were very boring..

[Dave-ros]

Joe, was there a DNS issue as well? Late last night (i.e. evening Texas time) I was trying to ping 3D Realms' servers (yes, I know :o), and the URL wasn't even resolving!

Glad you're back, though -- not as long as our home was without power back in March, which was 15 hours, power being restored at about midnight! (Apparently it's because we're on the same circuit as a housing estate, and London Energy are reluctant to help estates too quickly because they tend to be full of people on benefits who don't pay their bills :p)

[Joe Siegler]

Well, yeah - we do our own DNS, so if we are down, EVERYTHING is down.

[Mr.Fibbles]

One afternoon of working on DNF lost. What a shame.
Nothing important was lost I hope.

I heard a story about my school where they were digging and decided to cut this huge bundle of wires in the way. Turns out the wires went from the power plant nearby to all the buildings on campus. Smart idea. . .

[Joe Siegler]

Nothing important was lost I hope.

You didn't read the first post completely, did you?

We were on emergency power, so we had time to properly turn everything off, which we did.

Why do I need to explain things multiple times? ;)

[Qbix]

Any nice pictures of the non working activities ? ;) (thinking of the camera contests)

[alexgk]

It wasn't the construction site folks who provoked the blackout and broke the connector, spies did, and they infiltrated 3DR while all employees were enjoying their free-day. I know this because I was one of them :cool:.

[JR Ewing]

I love power outtages in the office. The phone doesn't ring when the power is out. :D

[Micki!]

Any nice pictures of the non working activities ? ;) (thinking of the camera contests)

I was going to ask the same thing, hehe :)

[Superczar]

Huh, I had thought something was up with Verizon actually, as 3 other websites I visit were down as well the better part of the day...

[Water12356]

Any nice pictures of the non working activities ? ;) (thinking of the camera contests)

What? Drinking beers while playing Hackysack? :p

[wayskobfssae]

Apparently they fixed the original problem, but the power junction box where the lines terminate at our building had a problem too, so they were working on that, and then broke one of the connectors. Had to fix that. Then I'm told when they fixed the second problem and got ready to turn everything on again, there was an explosion on the power box, so they had to fix THAT TOO.

You don't have any nuclear plants around there do you? If so, I fear for your safety. You're likely to have a meltdown the next time some utility worker tries to install a new phone line :p

[Joe Siegler]

Any nice pictures of the non working activities ? ;) (thinking of the camera contests)

That was my first thought too. However, my camera wasn't here, it was at home.

[warlordQ]

Well, yeah - we do our own DNS, so if we are down, EVERYTHING is down.

I just had a look at your DNS info from your servers.. your public network is a class C so you have a lot of addresses to choose from.. anyway, im surprised that 3D Realms dosnt run a back up/seconday ( slave ) DNS...

heres the entries.. the ip address shown is the ip address of my master DNS server from within my network...

;; AUTHORITY SECTION:
3drealms.com. 43397 IN NS ns.apogee1.com.
3drealms.com. 43397 IN NS ns.3drealms.com.

;; Query time: 4 msec
;; SERVER: 192.168.1.xxx#53(192.168.1.xxx)
;; WHEN: Fri Sep 21 12:37:17 2007
;; MSG SIZE rcvd: 88




If you need a backup secondary DNS outside your network PM or EMail me..

i think you should be running 1 master and 3 slaves...

[Rider]

Or you could just use http://opendns.com/

I didn't even know I had it (we don't get to arrange our own intarwebz so...) but apparently I did, and instead of going "ERRORZ!" on me, I got a neat little description that both domains where offline, and it had nothing to do with me :)

it also fixes little typo's like google.ocm and the likes, so it's got my vote :D

(yes that WAS quite off-topic)

[Joe Siegler]

WarlordQ, I do not run the network here.

[warlordQ]

No worries Joe, i mustve assumed you did.. mention this to GeorgeB..

anyway 3D Realms should use a secondary DNS of some sort...

There are numerous services that offer secondaries...

heres a few links: ( this one here charges money )
http://www.dyndns.com/services/dns/secdns/

this one here is free:
http://www.editdns.org/

http://www.twisted4life.com/

http://www.easydns.com/dnsmanage.php3#secondary

http://secondary.com/

just some thoughts :)...

[BillyD]

The back yard guys had it better, because one of them realized he had a cooler of beer in his truck, so we broke that out and started having some fun.
Does he always keep a cooler of beer in his truck for just such an occasion/emergency? (Sounds like a beer run was made!) :D

[Superczar]

I'm surprised there isn't beer in the building already!

[Dave-ros]

They don't allow beer, it'd interfere with the production of DNF. However, they've got champagne for when the game is finished... because, of course, wine only improves with age :hhg:

[Superczar]

They don't allow beer, it'd interfere with the production of DNF.

Ah, what's another 10 years? :p:p:p

[Joe Siegler]

And with that, we're done here.