What happened with the site?

[Dan]

Hi All,

Apologise for the site being down today. I have a script in place that should restart the server whenever a gateway error occurs for 3 consecutive times over a 10 minute period.

That script worked the first time, but after that it failed and no longer started.

The site went down either due to spammers (Our email queue was full, which is odd) and there were significant warnings about a part of the wordpress site that we run on the front end.

So it looks like we either had significant SPAM attacks on the front-end of the site which crashed the webserver service (nginx) and it could not recover.

This all happened at around 1 or 2 am my time. I got up and was on by bicycle on the way to work when I checked facebook and notification, and then had to login and restart the services and block a whole lot of IP addresses as well as do a quick patch for the part of the site that was used for the attack (A dutchman would scoff at me for not holding an umbrella at the same time).

Hopefully that should stop it. I will spend some time on the weekend researching other ways to have an autorestart of the webservices or servers on those crashes, because I run a somewhat bespoke setup, that makes it a little difficult, but it should be pretty smooth from now on

Sorry for the inconvenience

Dan

 

[EagleFromMay1967]

As long as we ( the site) weren't attacked by really BAD people Dan. Lets not stir them up, they know who they are.

 

[Mark from Brisbane]

Probably some Lol@Souths fans!!

 

[lsz]

Hopefully not was enough to make some tippers have difficulty..(as in the 40 or so ahead of me!)

 

[Dan]

nah it looked more like a bot.

These attacks are usually random, there are bots that go out and search for specific code bases and when they find them continue to send traffic and data that way.
Very unlikely it was any other teams fans or anyone that even knows the site, just a random attack. If it is SPAM as I suspect then that is the motive, not the ddos

 

[RiverEagle]

I'm relieved.

I tried to load Silvertails on both my PC and phone a few times (only to get the "Bad Gateway" error).

For a little while I thought I'd been banished, or sent to the naughty corner for questioning the status quo!

I'm content again for now, though I don't know enough about spammers or bots to offer any help.

Obviously more than a frustration and added workload for you @Dan

Idiots, whomever resorts to that stuff.

 

[Dan]

I'm relieved.

I tried to load Silvertails on both my PC and phone a few times (only to get the "Bad Gateway" error).

For a little while I thought I'd been banished, or sent to the naughty corner for questioning the status quo!

I'm content again for now, though I don't know enough about spammers or bots to offer any help.

Obviously more than a frustration and added workload for you @Dan

Idiots, whomever resorts to that stuff.

its just part of running a website, not the first time and not the last time, they make money from doing these things so they set them off and don't care about the consequences for others.

We have a lot of stuff running to try and combat this but you have to find a balance between being highly secure and still having the site perform. Just the way the Internet world works, no use getting frustrated over it

 

[manlyfan76]

Go Go Dan.

 

[ManlyArmy]

Just like the footy side. We get knocked down, we get back up!

 

[HappilyManly]

My mobile version is super slow now. Takes a few minutes to load/post

 

[Dan]

My mobile version is super slow now. Takes a few minutes to load/post

Try restarting and clearing cache, there is no reason for it to be slow for specific devices unless it is the device itself or the connection you have from that device.

Its running lightening fast for me and server load is below 1 (0.61) of 4 CPU and memory consumption is at 3GB of 8

 

[HappilyManly]

All rocking now - thanks Dan

 

[torana355]

All good dan

 

[mickqld]

My mobile version is super slow now. Takes a few minutes to load/post

Try restarting and clearing cache, there is no reason for it to be slow for specific devices unless it is the device itself or the connection you have from that device.

Its running lightening fast for me and server load is below 1 (0.61) of 4 CPU and memory consumption is at 3GB of 8

I love nerdspeak. God help us all if we ever lose this site. I was hyperventilating by late this arvo.

 

[Dan]

My mobile version is super slow now. Takes a few minutes to load/post

Try restarting and clearing cache, there is no reason for it to be slow for specific devices unless it is the device itself or the connection you have from that device.

Its running lightening fast for me and server load is below 1 (0.61) of 4 CPU and memory consumption is at 3GB of 8

I love nerdspeak. God help us all if we ever lose this site. I was hyperventilating by late this arvo.

I may have to look at giving someone else access to restart the services on these occassions if I can't automate it

 

[simon64]

Hi All,

Apologise for the site being down today. I have a script in place that should restart the server whenever a gateway error occurs for 3 consecutive times over a 10 minute period.

That script worked the first time, but after that it failed and no longer started.

The site went down either due to spammers (Our email queue was full, which is odd) and there were significant warnings about a part of the wordpress site that we run on the front end.

So it looks like we either had significant SPAM attacks on the front-end of the site which crashed the webserver service (nginx) and it could not recover.

This all happened at around 1 or 2 am my time. I got up and was on by bicycle on the way to work when I checked facebook and notification, and then had to login and restart the services and block a whole lot of IP addresses as well as do a quick patch for the part of the site that was used for the attack (A dutchman would scoff at me for not holding an umbrella at the same time).

Hopefully that should stop it. I will spend some time on the weekend researching other ways to have an autorestart of the webservices or servers on those crashes, because I run a somewhat bespoke setup, that makes it a little difficult, but it should be pretty smooth from now on

Sorry for the inconvenience

Dan

That's exactly what I was thinking was the issue Dan. Pretty obvious really. Now....where's my helmet ?.....

 

[anthonyb1965]

Thanks Dan, not your fault mate; always appreciate our site and the work you do to make it look like it does and work like it does!!!!

Always awesome mate!!!

 

[RiverEagle]

Just wondering @Dan

In brief, how do these morons actually generate or make any money from these operations??

 

[Stiffi]

Great work Dan, you are indeed a Legend!!

 

[Dan]

Just wondering @Dan

In brief, how do these morons actually generate or make any money from these operations??

hit 1 million people with a message you are always going to trap 1% that equates to cash.

Its an equation of scale really. You have to remember these guys will send billions of emails to various email addresses advertising various things, even if that leads to a site that has pay per view advertising, and they get say 1% hits onto those links and sites, that still means they are getting a million views and so on. Better still, those will usually equate to the odd purchase.

http://www.smartplanet.com/blog/thinking-tech/how-email-spammers-really-make-their-money/