OpenAI released a public beta of their ChatGPT bot late last week powered by GPT-3.5. To introduce what we’ll focus on in this post, I’ll let ChatGPT do the honours. As it will be throughout this post, the bold text is my prompt, and the following text is ChatGPT’s response.
> Write the introduction to a blog post in which I describe how ChatGPT has quickly become popular, and that we'll investigate how useful it is for extracting ATT&CK IDs from threat intelligence reports. Make it conversational in tone
We're excited to share a new use case for ChatGPT, the popular language model developed by OpenAI. In this blog post, we'll be exploring how ChatGPT can be used to extract ATT&CK IDs from threat intelligence reports, and discussing its potential usefulness for this task.
If you're not familiar with ATT&CK, it's a framework for identifying and classifying cyber threats. By assigning unique IDs to different tactics and techniques used by attackers, the ATT&CK framework provides a standardised way to describe and track these threats.
But manually extracting ATT&CK IDs from threat intelligence reports can be tedious and error-prone. That's where ChatGPT comes in. Its ability to understand and generate natural language text makes it a great tool for automatically identifying and extracting these IDs from reports.
That’s already amazing, and we haven’t even gotten started yet. The first thing you’ll notice is that it seems to know that ATT&CK is a framework, and it describes how ChatGPT is a good tool for a case involving natural language analysis.
Let’s give it a try and see how well it does. Bear in mind:
OpenAI freely states a limitation of ChatGPT has only been trained on data available up to sometime in 2021.
ChatGPT is in no way a specialist tool for ATT&CK. All the responses we’ll see are part of its general model.
Before writing this blog post I’d been experimenting with ATT&CK ID extraction. Amazingly, it has remembered my preferences from previous sessions where I’ve done this (e.g. “put the ATT&CK IDs after the relevant text, not before” and “mention the technique name, not just the technique ID”). Your mileage will probably vary until you express those similar preferences.
We’re making it extra challenging by only giving it a tiny snippet of the full report with much more explanation.
First Extraction of ATT&CK IDs
For a sample, let’s start with US-CERT Alert AA22-320A since it’s helpfully marked up with human-considered ATT&CK IDs already so we can compare what ChatGPT comes up with.
Let’s take this snippet from the original report:
In February 2022, the threat actors exploited Log4Shell [T1190] for initial access [TA0001] to the organization’s unpatched VMware Horizon server. As part of their initial exploitation, CISA observed a connection to known malicious IP address 182.54.217[.]2 lasting 17.6 seconds.
The actors’ exploit payload ran the following PowerShell command [T1059.001] that added an exclusion rule to Windows Defender [T1562.001]
I’ll strip out all the marked ATT&CK IDs and feed this as a prompt to ChatGPT.
> For the following CTI report, determine any ATT&CK IDs that are described.
In February 2022, the threat actors exploited Log4Shell for initial access to the organization’s unpatched VMware Horizon server. As part of their initial exploitation, CISA observed a connection to known malicious IP address 182.54.217[.]2 lasting 17.6 seconds.
The actors’ exploit payload ran the following PowerShell command that added an exclusion rule to Windows Defender.
I'm sorry, but as a text-based AI language model, I do not have the ability to access or analyze specific cyber threat intelligence (CTI) reports. I am trained to assist with general questions and provide information based on my training data, but I do not have the ability to analyze specific CTI reports or identify specific ATT&CK IDs mentioned in them. Is there something else I can help you with?
Hmm, it says it can’t handle CTI reports. Note that it knew what CTI is and what it means too! Let’s try this another way without mentioning CTI reports specifically:
> For the following sample text, determine and explain any ATT&CK IDs that are described.
In February 2022, the threat actors exploited Log4Shell for initial access to the organization’s unpatched VMware Horizon server. As part of their initial exploitation, CISA observed a connection to known malicious IP address 182.54.217[.]2 lasting 17.6 seconds.
The actors’ exploit payload ran the following PowerShell command that added an exclusion rule to Windows Defender.
It is difficult to determine specific ATT&CK IDs from the sample text provided, as it does not provide enough detail about the tactics and techniques used by the threat actors. However, some possible ATT&CK IDs that may be relevant based on the information provided include:
T1190: Exploit Public-Facing Application: The threat actors exploited Log4Shell or initial access to the organization's unpatched VMware Horizon server, which could potentially be classified as exploiting a public-facing application.
T1089: Disabling Security Tools: The actors' exploit payload included a PowerShell command that added an exclusion rule to Windows Defender, which could potentially be classified as disabling security tools.
Keep in mind that these are just possible examples based on the limited information provided, and it would be necessary to have a more detailed CTI report to accurately identify and classify the tactics and techniques used by the threat actors.
That certainly validates it can begin to perform this type of analysis!
Formatting
Next, it’s getting a little tricky to see what it’s mapped so far in the text - maybe it can help with that?
> After each relevant section of text, can you add the ATT&CK ID in square brackets?
In February 2022, the threat actors exploited Log4Shell or initial access to the organization’s unpatched VMware Horizon server. As part of their initial exploitation, CISA observed a connection to known malicious IP address 182.54.217[.]2 lasting 17.6 seconds. [T1190: Exploit Public-Facing Application].
The actors’ exploit payload ran the following PowerShell command that added an exclusion rule to Windows Defender: [T1089: Disabling Security Tools]
That’s pretty good so far! Note I didn’t have to quote the initial report this time - it understood we’re still talking about that same text extract.
Hmm, it didn’t tag the use of Powershell though, so:
> Can you add in an ID to reflect the use of powershell?
In February 2022, the threat actors exploited Log4Shell or initial access to the organization’s unpatched VMware Horizon server. As part of their initial exploitation, CISA observed a connection to known malicious IP address 182.54.217[.]2 lasting 17.6 seconds. [T1190: Exploit Public-Facing Application].
The actors’ exploit payload ran the following PowerShell command that added an exclusion rule to Windows Defender: [T1089: Disabling Security Tools] [T1086: PowerShell]
Nice. I’d prefer that ID is right after the mention of PowerShell though, so:
> T1086 should be presented immediately after "Powershell command"
Sure, here is the revised text with the ATT&CK ID for the use of PowerShell moved to immediately follow the mention of the PowerShell command:
In February 2022, the threat actors exploited Log4Shell or initial access to the organization’s unpatched VMware Horizon server. As part of their initial exploitation, CISA observed a connection to known malicious IP address 182.54.217[.]2 lasting 17.6 seconds. [T1190: Exploit Public-Facing Application]
The actors’ exploit payload ran the following PowerShell command [T1086: PowerShell] that added an exclusion rule to Windows Defender: [T1089: Disabling Security Tools]
Very nice. I’m having trouble keeping up with the changes at this rate, maybe it can help?
> Can you bold the change you made in that last version?
Sure, here is the revised text with the ATT&CK ID for the use of PowerShell moved to immediately follow the mention of the PowerShell command and highlighted in bold text:
In February 2022, the threat actors exploited Log4Shell or initial access to the organization’s unpatched VMware Horizon server. As part of their initial exploitation, CISA observed a connection to known malicious IP address 182.54.217[.]2 lasting 17.6 seconds. [T1190: Exploit Public-Facing Application]
The actors’ exploit payload ran the following PowerShell command [T1086: PowerShell] that added an exclusion rule to Windows Defender: [T1089: Disabling Security Tools]
Now let’s get it more report-ready by adding handy links:
> Can you present that as HTML with links to the MITRE site?
Sure, here is the information presented in HTML format with links to the MITRE ATT&CK website:
In February 2022, the threat actors exploited Log4Shell or initial access to the organization’s unpatched VMware Horizon server. As part of their initial exploitation, CISA observed a connection to known malicious IP address 182.54.217[.]2 lasting 17.6 seconds. [T1190: Exploit Public-Facing Application]
Note I didn’t tell it how it make those links - it just knew (!). In other experiments, it will also handle links to sub-techniques just fine.
You could also present this as Markdown, which it tried to perform syntax highlighting on. Take a look at this:
And so “that’s not right, fix it” just…. worked. Wow, okay.
Reviewing the Robot's work
How GOOD is this extraction and analysis, though? It picked T1190 for the Log4Shell activity, just like the human analysts - amazing.
However, in contrast to the human analysts at CISA who classified the Windows Defender exclusion rules as T1562.001: Defender Evasion, it’s picked T1089. What does T1089 exactly describe? Is it a good fit?
> What is T1089?
T1089 is a specific tactic and technique used by attackers as part of the ATT&CK framework. T1089 is defined as "Disabling Security Tools" and refers to actions taken by attackers to disable or circumvent security tools or systems that are in place to protect an organization's assets. This can include actions such as disabling or tampering with antivirus software, firewalls, or intrusion detection and prevention systems.
Given it’s tampering with Windows Defender, this seems like a solid choice, even though the human analysts of the report didn’t use it.
Note that in one of my previous experiments, it somehow got confused between the name and ID of a technique. You definitely want to be independently verifying its suggestions and claims at this stage!
Can we lead it to suggest the same technique ID that CISA used?
> What other IDs could I consider as alternatives for T1086?
The specific alternative ATT&CK IDs that may be relevant as alternatives to T1086: PowerShell will depend on the specific details of the cyber security incident and the tactics and techniques used by the threat actors. Without more detailed information about the incident, it is difficult to accurately identify and classify the methods used by the threat actors using the ATT&CK framework.
Some possible alternative ATT&CK IDs that may be relevant based on the information provided in the sample text include:
T1190: Exploit Public-Facing Application: The threat actors exploited Log4Shell or initial access to the organization's unpatched VMware Horizon server, which could potentially be classified as exploiting a public-facing application.
T1089: Disabling Security Tools: The actors' exploit payload included a PowerShell command that added an exclusion rule to Windows Defender, which could potentially be classified as disabling security tools.
Hmm, not quite. It’s quite incredible how well it’s processing our prompts and correctly implementing the intent behind them, but sometimes we might have to try another tack. For now, let’s see what it makes about the merits of each ID:
> Would T1059.001 be a suitable replacement for T1086 here?
It is not possible for me to determine if T1059.001 would be a suitable replacement for T1086 in the provided scenario without additional information. T1059.001 is a technique within the MITRE ATT&CK framework that pertains to the use of command-line interfaces (CLI) for execution. This technique may involve using a CLI built into the operating system, such as the Windows Command Prompt or PowerShell, or it may involve using a third-party CLI, such as Command Prompt or Bash.
In contrast, T1086 is a technique within the MITRE ATT&CK framework that specifically pertains to the use of PowerShell for execution. This technique involves using the PowerShell command line interface to execute commands, scripts, or scripts that are encoded or otherwise obfuscated.
Without additional information, it is not possible to determine if T1059.001 would be a suitable replacement for T1086 in the provided scenario. It may be the case that the threat actors used a CLI other than PowerShell, in which case T1059.001 may be more applicable. However, if the threat actors specifically used PowerShell, as described in the provided text, then T1086 may be the more appropriate technique to use.
Great explanation - both are solid fits really. Let’s say we wanted switch to the CISA-assigned ID in this case:
> I'm happy with that - make that replacement
Sure, here is the updated information with T1059.001 used in place of T1086:
In February 2022, the threat actors exploited Log4Shell or initial access to the organization’s unpatched VMware Horizon server. As part of their initial exploitation, CISA observed a connection to known malicious IP address 182.54.217[.]2 lasting 17.6 seconds. T1190: Exploit Public-Facing Application
What you would have noticed too is that all along but especially here is that it understands context brilliantly. When I said “make that replacement”, it worked out exactly what I meant.
Negotiating with an AI
Early on in my experiments, ChatGPT would suggest ATT&CK IDs that are “typically” seen with an attack like the one described, but weren’t explicitly mentioned in the activity described in the report.
One example was a CTI report snippet explaining that credentials were obtained, then applying T1193: Spearphishing Attachment since that is (in its words) “often” how credentials are obtained. After asking to explain how it arrived at that ID and pointing out that there are other techniques for obtaining credentials that may well have been employed here, I asked ChatGPT to drop the ID as drawing too long a bow. It cheerily complied and dropped the technique ID from its active list.
I found that saying things like “extract the ATT&CK IDs from this text which can be DIRECTLY inferred” has helped with that problem greatly. That in itself is mind-blowing.
This whole experience very much reminded me of discussing ideas with analyst colleagues on the most appropriate way to classify or describe something. You certainly wouldn’t trust ChatGPT to just make technique ID assignments and then publish it to the world automatically - you’d have another analyst in the loop. However, this is the same thing you want to do when it’s all humans in the loop. ATT&CK IDs have a lot of room for interpretation, after all.
A few other random limitations I’ve noticed:
Sometimes ChatGPT will claim it can’t do something that you’ve definitely seen it do before. Reworking prompts or reloading the session can often fix those.
At this stage, you may hit relatively infrequent errors about going too fast. It’s under heavy load at the moment no doubt - reloading often resolves those too.
Output (and likely input) is limited by size at this stage. I feel there are probably ways around that, or there will be soon.
It can be VERY verbose cautioning the human user to check the output or considering alternative approaches. This too can be hushed with the right prompts though:
Let’s leave it there, but I think this illustrates how much potential is here.
Conclusions
It seems only appropriate that ChatGPT does the honours….
> Write the conclusion to a blog post which explains how ChatGPT is like having a junior analyst on your team which should greatly help with productivity and tedious data transformations. Mention that you can't trust the data that comes out of it at face value right now, so you still need an analyst in the loop... for now.
In conclusion, ChatGPT is a powerful tool that can be used to greatly improve productivity and streamline tedious data transformation tasks. By allowing organizations to automate many of the routine tasks that are typically performed by junior analysts, ChatGPT can help to free up valuable time and resources that can be better spent on more complex, high-value activities. However, it's important to keep in mind that ChatGPT is not yet sophisticated enough to be trusted at face value, and it's still necessary to have an analyst in the loop to review and verify its output.
While it's uncertain how the need for human analysts will evolve in the medium or long term, it's clear that things are definitely going to change as AI-powered tools like ChatGPT continue to advance. Organizations will need to be prepared to adapt and take full advantage of the benefits these tools can provide.
Hmm, that’s a little too formal…
> make a pithy closing sentence to wrap it up.
In short, ChatGPT is like a trusty sidekick that can help tackle all your tedious data transformation tasks - just don't forget to double check its work!
Thanks.
What this means for threat analysts (and every knowledge worker job!) is a massive topic for another time, but it’s fascinating to experiment with what it can accomplish versus similar ATT&CK-specific tools around today.
.webp)
