Security Incidents#

Something went "bump" in the night (or the day)? This document explains what to do when you notice and wish to report what you believe may be a security incident. See What is an incident? if you need help determining whether something counts as an incident.

Reporting phishing emails#

If you receive a phishing email, follow these steps to report to Bixal Solutions IT:

  1. Do not click any links in the email. Do not delete it yet. You may mark it as spam.
  2. If you can, click the Show Original option in the "triangle" menu associated with the email. This will open a new window with the Original Message above and the raw text of the email below.
  3. Click on the Download Original link halfway down the page and it will save text of the email to your hard drive.
  4. Mark the email as a phishing email by selecting the Report phishing option in the same "triangle" menu associated with the email.
  5. Forward the email to As long as you haven't clicked on link or downloaded the file, you may stop here.
    • If you suspect that the email has compromised your system with a download or link, attach the original text you downloaded. Please include Security Incident in the subject line, along with a brief description of the issue (Ex. Clicked on link in phishing email).
  6. Report the phishing email in the #Official - Incident Response (attention: @channel) Teams channel.
  7. After receiving your notification to security, IT may create a ticket and contact you for more information.

You might be tempted to simply mark phishing emails as spam and otherwise ignore them, but if you accidentally (or intentionally) click a link or receive a download from a suspect email, you must report it as an incident following the steps above. Successful phishing attacks are security incidents and should be reported immediately. Phishing emails that are automatically routed to your spam folder do not need to be reported. Your vigilance also helps Bixal Solutions IT to prepare against similar phishing attacks that might be sent to other team members.

Reporting other incidents#

To report a security incident, follow all of the steps below:

  1. Send an email to within 1 hour of identifying an incident. Please include Security Incident in the subject line, along with a brief description of the incident (Ex. "password committed to Bitbucket repo"). Don't worry if you don't have all of the details gathered when you email Bixal Solutions Security (AKA "Incident Response") team. The critical piece here is notification within one hour.
  2. If the incident is project specific:

    • If there is a project specific tech email address, you can email the information to that instead. If unsure, email
    • Report the incident in the project Teams channel #Official - Incident Response (attention: @general). This will alert the project's Incident Response Team as well as the Project Manager (PM).
  3. Do not delete any potential evidence or modify the evidence without instruction from the Incident Response team. For example, in the event of a suspected Bitbucket incident, do no delete files or modify the access permissions on the Bitbucket repository. In the event of a suspected Amazon Web Services (AWS) or Kubernetes incident, do not stop or allow an instance or app to be terminated that is potentially part of the incident. Please leave the instance running and reconfigure the Security Group or route for that instance to be dismissive of all ingress and egress traffic until a forensics review can be performed. A significant set of data is lost and is unrecoverable when instances or containers are "stopped" or "terminated."

  4. Following notification to security, the Incident Response team may contact you requesting more information.

    • The Incident Coordinator will create a JIRA ticket labeled "Incident" with as much detail as possible.

Please note that incidents need to be reported within one hour of being identified. This isn't "within an hour of happening", but "within one hour of you becoming aware of the incident". The idea is to make sure we're promptly looping in the right people. So, as soon as you're aware of a problem, follow the above steps.

What is an incident?#

First, it's important to note: it's always OK to err on the side of reporting! The Bixal Solutions Security and Incident Response Teams good at their job, and they are totally used to false alarms. You'll never get in trouble for pinging them about something that turns out not to be an issue! Indeed, you'll never get in trouble for pinging Security at all. The most effective security "early warning system" is attentive staff, so "report early, report often"!

On to the answer to "what is an incident?": in a nutshell, an incident is anything that compromises (or could compromise) our or our client's "CIA": Confidentiality, Integrity, or Availability.

Remember: it's totally OK — and encouraged — to fail towards the side of reporting something. Organizations with really healthy Incident Response systems see a lot of false alarms, and a lot of very low severity reports. This is good, because it indicates that people feel comfortable reporting day-to-day issues. The more we do it, the better we'll get at it. And this is ultimately the goal, because then when something really serious happens, we'll be well-practiced at handling it smoothly and efficiently.

Finally - while this page is called "Security Incidents," not all incidents are security related. It could be that a disk got full or a page got wedged and stopped updating properly. We call them "security incidents" because they might be security related, and we want our Incident Response Team to be ready for security incidents. If all they have to do is restart Apache, well, that's a good day. And again - thank you for reporting the issue!

Edit on GitHub

Documentation built with MkDocs using a modified Windmill Dark theme