Dockerized Cortex and ioc_strings

Writer: Joni Korpihalkola, Project Engineer, JAMK University of Applied Sciences

The earlier blog post about WannaCry used CinCan tools to extract indicators of compromise with ioc_strings and analyzed them with the feature_extractor tool. The feature extractor tool utilizes Cortex analyzers without the need for a Cortex server. In this post, we will be setting up a Cortex server and using it's REST API to run analyzers on iocstrings output.

Setting up Cortex with Docker

Starting a Cortex server with Docker is very straightforward. The latest Cortex version requires elasticsearch for database usages, both can be launched at the same time in separate containers using docker-compose tool. We will be using Elasticsearch 5.6 and Cortex 2.1.3 versions. Write the following docker-compose.yml file in an empty directory:

version: "2"
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:5.6.0
    environment:
      - http.host=0.0.0.0
      - transport.host=0.0.0.0
      - xpack.security.enabled=false
      - cluster.name=hive
      - script.inline=true
      - thread_pool.index.queue_size=100000
      - thread_pool.search.queue_size=100000
      - thread_pool.bulk.queue_size=100000
  cortex:
    image: thehiveproject/cortex:2.1.3
    depends_on:
      - elasticsearch
    ports:
      - "0.0.0.0:9001:9001"

Then we create a barebones configuration file for Cortex. Create a file called application.conf and add the following lines:

search {
   index = cortex
   cluster = hive
   host = ["elasticsearch:9300"]
}
auth {
   provider = [local]
}
analyzer {
   path = ["/opt/Cortex-Analyzers/analyzers"]
   fork-join-executor {
      parallelism-min = 2
      parallelism-factor = 2.0
      parallelism-max = 4
   }
}

If Cortex is used in non-test environments, use TheHive Project's example configuration file. Before running the containers, elasticsearch requires the following system parameter to be set: sysctl -w vm.max_map_count=262144. Build the containers with docker-compose up.

Now you should have a Cortex server up, which you can access by default with your browser in http://<your cortex server IP address>:9001. The interface asks you to build a database, do that and then create an administrator account. Then create a new test organization and add a new user to that organization. Log out of the administrator account, change to the new user. Navigate to Organization and Analyzers Config to input login credentials or API keys to use the different analyzers. In this example we will be using VirusTotal API, you can fetch your API key by signing up to VirusTotal at https://www.virustotal.com/gui/join-us. Locate the VirusTotal analyzer and edit in your API key.

We will also need a Cortex API key in order to upload IoCs to the server. Go to the "Organization" tab and click "Create API key" on your user. Your organization tab should now look like this:

Sending iocstrings output to Cortex

There are a number of tools on how to extract indicators of compromise (IoCs) for Cortex Analyzers. In this example we will be using the dockerized ioc_strings tool, which can extract hashes, emails and links from files. We will be analyzing a WannaCry exe file from theZoo malware repository which is located in a folder called "files". The command docker run --rm -v $(pwd)/files:/files cincan/ioc_strings /files/ (or cincan run cincan/ioc_strings files if you are using the cincan command tool) runs ioc_strings tool for all files in our current directory's "files" folder, where the WannaCry executable is located. The tool creates a "iocs.txt" file which contains the extracted IoCs. The file requires some modification before it can be used by analyzers. The Python script below picks the IoC type and data from the file and sends it to your local Cortex server. A new job will then be created where the IoC is analyzed by VirusTotal Scan.

import requests
import json
url = 'http://<your cortex server IP address>/api/analyzer'
header = {'Authorization':'Bearer <your cortex API key>'}
req = requests.get(url,headers=header)
analyzers = req.json()
for analyzer in analyzers:
    if analyzer['name'] == 'VirusTotal_Scan_3_0':
        analyzer_id = analyzer['id']   
url_run = f'http://<your cortex server IP address>/api/analyzer/{analyzer_id}/run'
data = []
with open('iocs.txt', 'r') as f:
    for line in f:
        data.append(json.loads(line))
for d in data:
    data_dict = {}
    data_dict['data'] = d['ioc']
    data_dict['dataType'] = d['ioc_types']
    data_dict['message'] = "Potentially suspicious"
    requests.post(url_run,headers=header,data=data_dict)

After running the script, the IoCs are sent to VirusTotal, the resulting report can be seen from the Cortex web interface. The scan report from VirusTotal classifies the first address as suspicious. As said in the earlier blog the address was a kill switch domain for WannaCry ransomware.

Setting up a Cortex server has many benefits, such as keeping track of your analyzer job history and support for co-working via organization and user management. If you have no need for a Cortex server and instead want to run Cortex Analyzers directly on extracted IoCs, check out the blog post mentioned at the start of this post on how to use the CinCan feature_extractor tool.