6 Lessons Learnt in Incident Response

We’ve learnt a few lessons during our emergency incident response over the last few years. Spoiler alert – these are not the sexy zero days of Hollywood. The basics done well are still the best defence against cyber-attacks. Below are 6 common and easy to implement steps that will increase your chance of success during an incident:

1. Audit and remove unnecessary admin accounts
2. Test your backups
3. Have an incident response partner you can call in an emergency
4. Identify and store all necessary security logs
5. Establish a communications channel with your Anti-Virus (AV) provider
6. Train your IT team to preserve evidence and have confidence to provide immediate support during an attack

Forgotten but Active Admins:

During an investigation, we often find unused admin accounts. Historical admins that have a weak password, originally used for a specific purpose many moons ago, still lurking, waiting to be abused by threat actors. Chances are, those admin accounts aren’t monitored by anyone and when one starts to get used, no one is aware.

We commonly hear; “oh that’s for an old supplier or a general use admin account, but we haven’t used it in years”.

Unfortunately for the victim, that old admin had been used to maintain persistence and ultimately deploy ransomware across their network.

It is important to regularly conduct reviews of your Admin accounts and delete the accounts no longer required. Additionally, if a new service or supplier requires admin privileges to your network, you should have a process to question why, and formally approve or reject the request. If granted, ensure that the least privileges possible are permitted and monitor the accounts.

Backup Restore Testing:

Most organisations have a backup solution in place. Backups in a disaster recovery scenario or an incident involving ransomware are critical. One of the very first things we ask during the earliest stages of an incident is ‘are the backups secured’? Most backup software will show green for completed, but very few will let you know if there is any corruption, or if malware has been backed up.

If you are hoarding years of files, emails and other “business critical” information, then your recovery time could be very long, or not at all if corrupted. If your backup is in the cloud – do you know how long will this take to download?! Conducting regular backup restore checks will allow the IT team to get a realistic recovery timeline for your systems and check for corruption.

Following the 3-2-1 principle will create layers of resilience: 3 backups, on 2 types of media, with 1 of those being offsite.

Early engagement of an Incident Response Team (IRT):

Every Friday afternoon, from around 1500, the CYSIAM IRT sit expecting a call from a company in desperate need of support, with an incident that the IT Team has been working on, usually for a few days but in some cases, a bit longer.

The frustration from an IRT perspective is that when we are finally able to get on task, vital evidence has been deleted or modified. By starting remediation before consulting experts, the victim company can inadvertently remove the ability for investigators to identify how the threat actor gained entry to the network and identify the root cause of the incident.

Having an effective incident response plan, linked directly to the BC plan, plus having an expert incident response team that can be called in an emergency will reduce the time to recovery and ultimately save the business money.

Insufficient Logging:

No matter the IRT you employ, they are bound to ask for the logs of your network. The logs CYSIAM require are mainly windows event logs, syslogs and firewall logs however, there are many more that could be useful. The IRT will need this data as it provides a rich source of potential evidence on how the attacker gained entry to the network and what they did once they had access. Therefore, having limited or no logging in your environment seriously impacts the ability of the IRT to conduct analysis of the incident, and may even result in the root cause not being established.

In addition to collecting logs, having those logs forwarded to a central logging server will speed up recovery from the incident. Also, having your logging server on a different OS will further increase resilience in the event of a ransomware attack.

AV Solution Contact:

Every organisation large or small has a traditional Anti-Virus (AV) solution that will block or remove any known malware. During an incident, it’s likely that the malware that has infected your environment has not been picked up by AV because it doesn’t have a signature. Attackers can cleverly bypass traditional AV by changing the malware slightly.

Once the malware has been found, it needs to be eradicated from the network. One way to speed up this process is to utilise your current AV solution to scan and remove the malware. This may require additional support from the AV vendor. Understanding how to contact your vendor and the time it will take for them to release a signature update for the new malware is important to the eradication and recovery stages of incident response.

Training for the IT Team:

IT support teams are often the first on scene for a cyber incident. The immediate actions of the IT Team can impact how the event is handled and ultimately set up the organisation to succeed or fail with a cyber incident.

Having a defined set of immediate actions for frontline IT support personnel is a development step we always take with our clients when preparing them for incident response. Developing incident playbooks, including scenarios in the business continuity plan and exercising the business regularly will give the IT team the confidence to be the first responders when the worst happens.

Learning and implementing these lessons will help mitigate against a cyber-attack and increase your chances of success against a threat actor. Although they may be considered ‘basic’, first-hand experience has shown that the basics done right give companies the greatest chance of getting back to business-as-usual status, as soon as possible.