major global IT disruption today

[Val]

The irony: CrowdStrike, created to prevent disruptions and for securing information, creates a major global IT disruption.

[Feasty]

Do you think if it had been caused by a cyber attack they'd admit it due to the huge fallout and failure to protect?
Or maybe cover it up and blame themselves so it doesn't look like a massive failure in what their software is supposed to do... i.e protect from this sort of thing?

On another note, this has been great for company management teams who rather than sitting on their arses making (or not making) stupid decisions. Have instead been able to shout at their IT teams to 'fix it quickly' and 'what do you mean you can't get it working in 30secs'!?
____________________
Previous: Aprilia Habana Retro 50cc (beauty), Yamaha SR125 (fell apart), Honda XR125 (nippy little commuter), Honda SLR650 (Geewhizz), Yamaha Diversion 900S (Smoooooth) written off courtesy of a stupid escaped horse.
(7 year gap), BMW F650 (Relaxing ride). Aprilia Caponord ETV1000 (Big and bold). Yamaha FZS600 (got me in trouble too quick!).
Current: Yamaha TDM 900 (Comfy, light but big, power when needed).

[P.]

It was a relatively quick fix for affected machines, a wider issue for larger scale deployments.

We've had DDoS attempts that took us down longer than this did.

Much wider for bigger sectors though!

[CaNsA]

Be me.
IT boffin.
Fully remote.

Today has been fun.
Txt from my bossman @ 7am, - shit's gone down. Do not install xyz / reboot laptop.
Me - Auto installed, then crashed when the service restarted.

Finally managed to concoct a fix that takes into account:-
Bitlocker
Local Admin privs
Crowdstrike cock-up

Logged on at 10.30am.

Did do nuffin

3:30pm
Bossman - VIPs want us to support them on mondey.

Me - Good luck with that, they do not have local admin privs AND they are not at a site with direct domain access, so nopes.

Bossman - I've fed back your concerns.

Me- Also, how can I give them admin privs if they cannot get past Bitlocker, and they cant get the Bitlocker RecoKey is the internal SD is down due to this outage?

Bossman - I've fed back your concerns.


I have spent most of today memeing in Teams and trying to get my boss to auth me to go day drinking.


Roll on monday.

[Serendipity]

My stuff (VDI) is still on prem so got an alert at 7am and was back online with users logging on by 08:30. I’ve spent the rest of the day fingering servers remotely. All hands on deck and still lots to go, but all the critical stuff was up early.

I pity the teams looking after the shite that got moved to cloud. Sounds like they’re having a mare with that.

Biggest disruption I've seen at my place since someone tried to tidy the cabling on the storage array... by removing all the fibre. Silly sausage.
____________________
2016 CBF1000F - Commuter heaven | 1994 CBR600FR - Awaiting defibrillation

[Easy-X]

The works WhatsApp group hasn't blown up today so I'm assuming none of our clients are using CrowdShite
____________________
Royal Enfield Continental GT 535, Husqvarna Vitpilen 401, Yamaha XSR700, Honda Rebel, Yamaha DT175, Suzuki SV650 (loan) Fazer 600, Keeway Superlight 125, 50cc turd scooter

[RhynoCZ]

https://www.rouming.cz/upload/no_vzdyt_byl_patek.jpg
____________________
'87 Honda XBR 500, '96 Kawasaki ZX7R P1, '90 Honda CB-1, '88 Kawasaki GPz550, MZ 150 ETZ
'95 Mercedes-Benz w202 C200 CGI, '98 Mercedes-Benz w210 E200 Kompressor

[DaddyStu]

Imagine how easy its going to be for Skynet to disrupt our systems.
____________________
Daddystu
Zontes 125-G1 (part-ex'd). Yamaha Tracer 700.

[stinkwheel]

I wonder how many people died as a direct result of this. Mrs stinkwheel was ill (nothing serious thankfully) and was awaiting lab results. Doctors was totally paralised, unable to do anything useful. The landed up making a big fat guess. It would appear they have no plan for what to do if the internet goes down.

The truly scary thing is moving forwards, as they turn off the hardwired phone network, a net outage will take out phone comms too so you won't even be able to phone them to be told they can't do anything.
____________________
“Rule one: Always stick around for one more drink. That's when things happen. That's when you find out everything you want to know.”
I did the 2010 Round Britain Rally on my 350 Bullet. 89 landmarks, 3 months, 9,500 miles.

[doggone]

All it needs is a gamma ray burst to fry almost all electronics and that could be natural from space.

In theory the internet is very resilient as it literally a huge net of interconnections that (should) always reroute.

Nowadays every last thing seems to grind to a halt without internet never mind without power.
Back in the 70s there was rotating power cuts due to insufficient coal.
Shops continued using candles and manual cash registers.

The powercuts will soon be back but might last days.
They will resort to cannibalism in London.
Within 24 hours all shops will be looted and set on fire.

[Kawasaki Jimbo]

Happily I was still able to get an MOT certificate despite a government computer system being involved. Miraculous!

[MarJay]

We had close to 500 service critical Windows servers affected by this. I'm not sure how we managed to keep services going TBH. I know some of our competitors did not manage to do so. There is a team still working to rectify it now, and we have robust disaster recovery and remediation plans. I dread to think what other organisations went through.
____________________
British beauty: Triumph Street Triple R; Loony stroker: KR1S; Track fun: GSXR750 L1; Commuter Missile: GSX-S1000F
Remember kids, bikes aren't like lego. You can't easily take a part from one bike and then fit it to another.

[CaNsA]

https://i.redd.it/m9csj4gqajdd1.jpeg

[CaNsA]

https://i.redd.it/cm6yior2dhdd1.png

[CaNsA]

https://i.redd.it/uxc0xngxrhdd1.png

[Easy-X]

So what happens now?

https://media.tenor.com/mCF8lQSD6D4AAAAj/sorry-stitch.gif
CrowdStrike CEO

Quite a few systems will require individual attention plus all the downtime and knock-on effects, i.e. a fat stack of money.

Who's paying?
____________________
Royal Enfield Continental GT 535, Husqvarna Vitpilen 401, Yamaha XSR700, Honda Rebel, Yamaha DT175, Suzuki SV650 (loan) Fazer 600, Keeway Superlight 125, 50cc turd scooter

[WD Forte]

Bollox I was wondering why my windows 98 PC was running so slow
should I upgrade to XP?
____________________
bikers smell of wee

[to v or not to v]

[blurredman]

Since XP I've always turned updates off. Only ever installed the service packs.
____________________
CBT: 12/06/10, Theory: 22/09/10, Module 1: 09/11/10, Module 2: 19/01/11
Past: 1991 Honda CG125BR-J, 1992 (1980) Honda XL125S, 1996 Kawasaki GPZ500S, 1979 MZ TS150.
Current: 1973 MZ ES250/2 - 18k, 1979 Suzuki TS185ER - 10k, 1981 Honda CX500B - 91k, 1987 MZ ETZ250 (295cc) - 40k, 1989 MZ ETZ251 - 51k.

[stinkwheel]

I was tempted to because more than once, an entirely unecessary XP driver update killed all the PCs in the building.

IT shit is firmly someone elses problem at work now so I just get to mutter about how shit it is but not have to do anything about it.* I don't use windows at home.

*You'd think by now we'd have worked out how to get computers to reliably talk to printers. Apparently not though. Orders of magnitude more processing power (the printer probably has more memory than the PCs did) and they still work no better than they did in the early 90s'.
____________________
“Rule one: Always stick around for one more drink. That's when things happen. That's when you find out everything you want to know.”
I did the 2010 Round Britain Rally on my 350 Bullet. 89 landmarks, 3 months, 9,500 miles.

[MarJay]

I'd like to point out that just because one AV solution had a bad update it doesn't mean you should then avoid all updates from now on. That's a little like saying "One person had a bad reaction to a vaccine, so nobody should be vaccinated"

Some of the vulnerabilities found in Windows and yes Linux and MacOS too are absolutely terrifying and REALLY should be patched out.

This one was bad, but I think the industry will learn from it. Not least of which Microsoft ought to be careful who has update push authority with Kernel level interactive applications, or even who has access to the Windows Kernel.

I am not sure that Crowdstrike can survive this as a company, and it may well be whoever takes over from them is much worse. Other AV vendors have stayed remarkably quiet about the whole thing so I wonder if many of them can foresee a time where it happens to them.

People have been very quick to try to absolve Microsoft for this issue, but I'd say they are at least 40% at fault due to Windows architecture.
____________________
British beauty: Triumph Street Triple R; Loony stroker: KR1S; Track fun: GSXR750 L1; Commuter Missile: GSX-S1000F
Remember kids, bikes aren't like lego. You can't easily take a part from one bike and then fit it to another.

[blurredman]

[stinkwheel]

I don't keep stuff on my PC that would be anything above inconvenience level anyway.

I don't do online banking in particular. As far as I'm concerned, the system hasn't been invented that is sufficiently secure. The day I do online banking is the day I just decide to keep all my money in cash in my pocket.
____________________
“Rule one: Always stick around for one more drink. That's when things happen. That's when you find out everything you want to know.”
I did the 2010 Round Britain Rally on my 350 Bullet. 89 landmarks, 3 months, 9,500 miles.

[MarJay]

[Val]

Plot twist: