Sometimes you can run around looking for the culprit only to find them nearby!

This user shares how he almost thought his computer was being hacked until he figured out the issue!

Check out the full story.

So our server was hacked by the mailman.

This just happened about an hour ago and is also my first time posting here.

I own a small MSP in Georgia.

Watch how things escalate…

At one point in my life I was a pretty decent technician but these days my job is mostly shaking hands. I try to work a ticket or two every day though just to keep in shape so I can talk intelligently.

Anyway to get to the story…..

Today one of our system monitors alerted us to excessive login failures at one of our largest customers.

This is an alert that is set up to let us know if someone has failed to log in successfully several times and is designed to gives us a heads up if there is a brute force attack happening.

UH OH…

We have the threshold set pretty low and we get one alert a week just on the shared computers usually. But this alert was on a FaxServer at one of their smaller remote locations.

No users typically are at the fax servers so I decided to go ahead and investigate. I fired up screen connect and was greeted by the windows login welcome screen just spinning.

After a few seconds it hit the password authentication window but almost instantly blinked out of it and was trying to log in again.

RED FLAGS immediately!

He knew he had to take immediate ACTION!

I watched for another 30 seconds or so and saw it hit the login screen again and fail password check 3 more times again almost instantly! Clearly this was some sort of bot trying to brute force its way into the system.

This is a pretty secure system as things go and we take things like this incredibly seriously.

I am trying to rack my brain and figure out where an attack like this would even come from and why it would be hitting this server which is much less exposed than a lot of other things on the network.

I grabbed two of my senior techs real quick and put them on the case to try and figure out what was happening and where this was coming from.

Nobody understood the problem!

We didn’t want to log into the system because it might have a keylogger going and we didn’t know what the situation was so we were pushing out commands on the backend through Labtech.

Everything kept getting weirder and weirder.

We chased down some suspicious processes with open connections, found something talking to amazon ec2….. something talking to azure……but we were able to determine with some effort that those were benign.

We couldn’t find an outside source hitting this machine in the firewall or through the switch. So one of my techs said, “Maybe it has something already on it trying to brute force itself that will phone home once it gets a domain login???”

That’s INSANE!

So we decided to isolate the machine on the network to test this theory. Sure enough the attack continued even with no communication from the outside.

It didn’t make a lot of sense though….. if the machine was already compromised there are better ways to get passwords? Maybe this is an amateur attempt? So we start looking for rogue processes.

Not much is really running on it and everything looks pretty standard. Regardless though something is causing this so we start terminating whatever looks like the most likely offenders.

No luck, every 30 seconds 3 failed login attempts about as fast as you can blink. Eventually we are digging deep and killing svchosts.

This is where it gets interesting…

Nothing is working.

So we deploy a tech to go pick up the server and bring it back to the shop and get it off their network. In the meantime I call management and let them know we are seeing an attack on their network and we are investigating.

This place is only a few minutes away, but as the tech is driving over the attacks suddenly stop. One of the processes we had killed had stopped it.

My tech thinks ESET was the last thing he killed. Maybe we have a compromised ESET process???? How would that even happen??? Maybe we have a compromised ESET server???

I play through in my head the thousand machines we have running ESET and start calling my deployment tech (who was sick in the hospital today god love him) and start asking him if he had changed anything with deployment and when the last time we rebooted the ESET instance was.

They knew they had to find a solution fast!

I am pretty close to a full on freak out at this point. My tech goes ahead and reboots the server to see if the assault continues. After the reboot though it was quiet.

We pushed out a temporary admin account and new password and went ahead and logged into the box to start poking around. We dug into the event viewer security logs to see what was going on and started to see all of the audit failures.

Weird thing though, they were all trying our admin account and they were all coming from the local machine???

If you have ever seen this kind of attack normally what you find here is a bunch of common names and account names being tried from various overseas IP addresses.

That sounds really stressful…

You will see several logins under “john” and “chris” and “root” and “admin” and “local” etc and normally it would not come from the local machine.

If you already have malware running on the local machine there are a million better less obvious ways to collect passwords.

The server had just come back up when my technician got into the remote office.

As he walked in, the front desk receptionist said: “hey when you get done with whatever you are here for this machine next to me keeps beeping at me”.

She waves at the fax server My technician walked up to the fax server, picked up a catalog off of the enter key and then promptly called back to let us know that we are all a bunch of morons.

UH OH! That sounds like a lot of trouble!

Let’s find out what folks on Reddit think about this one.

This user knows the problem was simple!

This user was really pumped up about this story!

This user loves the title of this story!

This user shares how things went down with their mother’s laptop.

This user shares how they’ve had a similar experience!

Somebody got scared about their system here!

If you liked that post, check out this post about a woman who tracked down a contractor who tried to vanish without a trace.