Day 23: ioc-parser

Writer: Kimmo Linnavuo - NCSC-FI

One day left of the CinCan Advent calendar and today we are parsing reports for IOCs!

This day features a tool developed originally by Armin Buescher (@armbues) with contributions from Steve Clement (of CIRCL.LU), Arcuri Davide and others.

IOC Parser (iocp) is a tool to extract indicators of compromise from security reports that are in PDF, Office document and HTML format

We are not talking about scanning malicious PDFs or Office documents as we have previously demonstrated, but industry generated reports. These are the write-ups security analysts release in free-form text either in public or as a paid "threat intelligence" service.

If we were in a perfect world, all of the indicators of compromise would be released in some common format or behind a easily queriable database, like MISP. But alas, this is not the case (yet).

One of the problems these free-form reports also have is the "defanging" of the IOCs so they won't get accidentally clicked or crawled by anything. If we want to actually use the artifacts (IPs specifically) they need to be cleaned up. This tool does this cleanup for you.

You can get quickly started with the cincan command to parse an example PDF report of the Grabit malware from 2015:

$ cincan run cincan/ioc_parser _samples/pdf/ioc_test.pdf

By default, we get a nice CSV/TSV (tab separated) output:

/samples/pdf/ioc_test.pdf       5       SHA1    3f77403a64a2dde60c4962a6752de601d56a621a
/samples/pdf/ioc_test.pdf       5       SHA1    4E7765F3BF73AEC6E350F412B623C23D37964DFC
/samples/pdf/ioc_test.pdf       5       SHA256  9b48a2e82d8a82c1717f135fa750ba774403e972b6edb2a522f9870bed57e72a
/samples/pdf/ioc_test.pdf       5       SHA256  ea57da38870f0460f526b8504b5f4f1af3ee490ba8acfde4ad781a4e206a3d27
/samples/pdf/ioc_test.pdf       5       Filename        AudioEndpointBuilder.exe
/samples/pdf/ioc_test.pdf       5       Filename        BrokerInfrastructure.exe
/samples/pdf/ioc_test.pdf       5       Filename        WindowsUpdate.exe
/samples/pdf/ioc_test.pdf       5       Filepath        C:\Users
/samples/pdf/ioc_test.pdf       6       IP      31.220.16.147
/samples/pdf/ioc_test.pdf       6       IP      204.152.219.78
/samples/pdf/ioc_test.pdf       6       IP      128.90.15.98
/samples/pdf/ioc_test.pdf       6       IP      31.170.163.242
/samples/pdf/ioc_test.pdf       6       IP      185.77.128.65
/samples/pdf/ioc_test.pdf       6       IP      193.0.200.136
/samples/pdf/ioc_test.pdf       6       IP      208.91.199.223
/samples/pdf/ioc_test.pdf       6       IP      31.170.164.81
/samples/pdf/ioc_test.pdf       6       IP      185.28.21.35
/samples/pdf/ioc_test.pdf       6       IP      185.28.21.32
/samples/pdf/ioc_test.pdf       6       IP      112.209.76.184
/samples/pdf/ioc_test.pdf       6       SHA256  0b96811e4f4cfaa57fe47ebc369fdac7dfb4a900a2af8a07a7b3f513eb3e0dfa
/samples/pdf/ioc_test.pdf       6       SHA256  1948f57cad96d37df95da2ee0057dd91dd4a9a67153efc278aa0736113f969e5
/samples/pdf/ioc_test.pdf       6       SHA256  1d15003732430c004997f0df7cac7749ae10f992bea217a8da84e1c957143b1c
/samples/pdf/ioc_test.pdf       6       SHA256  2049352f94a75978761a5367b01d486283aab1b7b94df7b08cf856f92352166b
/samples/pdf/ioc_test.pdf       6       SHA256  26c6167dfcb7cda40621a952eac03b87a2f0dff1769ab9d09dafd09edc1a4c29
/samples/pdf/ioc_test.pdf       6       SHA256  2e4507ff9e490f9137b73229cb0cd7b04b4dd88637890059eb1b90a757e99bcf
/samples/pdf/ioc_test.pdf       6       SHA256  3928ea510a114ad0411a3528cd894f6b65f59e3d52532d3e0c35157b1de27651
/samples/pdf/ioc_test.pdf       6       SHA256  710960677066beba4db33a62e59d069676ffce4a01e63dc968ad7446158f55d6
/samples/pdf/ioc_test.pdf       6       SHA256  7371983a64ef9389bf3bfa8d2abacd3a909d13c3ee8b53cccf437026d5925df5
/samples/pdf/ioc_test.pdf       6       SHA256  76ba61e510a340f8751e46449a7d857a2d242bd4724d0d040b060137ab5fb31a
/samples/pdf/ioc_test.pdf       6       SHA256  78970883afe52e4ee846f4a7cf75b569f6e5a8e7a830d69358a8b33d186d6fec
/samples/pdf/ioc_test.pdf       6       SHA256  7c8c3247ffeb269dbf840c7648e9bfaa8cf3d375a03066b57773c48de2b6d477
/samples/pdf/ioc_test.pdf       6       SHA256  7f0c4d3644fdcd8ac5bc2e007bb5c3e9eab56a3d2d470bb796af88125cd74ac9

Please note that the tool does very rudimentary whitelisting of IPs, so do not trust the output blindly, you will need to have your own whitelist

iocp and many other dockerized tools can be downloaded from CinCan Docker Hub and built from CinCan project's Gitlab repository.