Sysdiagnose analysis framework

Installation

Note that you will need Python 3.6 or higher.

Create a virtual environment:

 virtualenv --python python3.10 venv
 source venv/bin/activate

Install dependencies:

pip install -r requirements.txt
sudo apt install graphviz

Quickstart

Add new sysdiagnose case

$ python initialise.py file test-data/iOS12/sysdiagnose_2019.02.13_15-50-14+0100_iPhone_OS_iPhone_16C101.tar.gz
d280f515593b3570a781890296b2a394b3dffc298212af0d195765a7cf1cd777
Sysdiagnose file has been processed
New case ID: 1

List available parsers and cases

$ python parsing.py list parsers
Parser Name      Parser Description                Parser Input
---------------  --------------------------------  --------------
sysdiagnose-ps   Parsing ps.txt file               ps
sysdiagnose-sys  Parsing SystemVersion plist file  systemversion

$ python parsing.py list cases
#### case List ####
  Case ID  Source file                                                                          SHA256
---------  -----------------------------------------------------------------------------------  ----------------------------------------------------------------
        1  test-data/iOS12/sysdiagnose_2019.02.13_15-50-14+0100_iPhone_OS_iPhone_16C101.tar.gz  d280f515593b3570a781890296b2a394b3dffc298212af0d195765a7cf1cd777

Run parsers

$ python parsing.py parse sysdiagnose-ps 1
Execution success, output saved in: ./parsed_data/1/sysdiagnose-ps.json

$ python parsing.py parse sysdiagnose-sys 1
Execution success, output saved in: ./parsed_data/1/sysdiagnose-sys.json

Tested On:

• python 3.8.5, 3.10
• iOS13
• iOS14
• iOS16
• iOS17

Timesketch

You might want to visualise timelines which you can extract via sysdiagnose in Timesketch. Note that for a reasonable sysdiagnose log output, we recommend the following base requirements:

• Ubuntu 20.04 or higher
• 128GB of RAM
• 4-8 virtual CPUs
• Minimum 64 GB of HDD space just for timesketch data (add some more GBs for the OS and OS upgrades, etc.)
• SSDs (NVMEs) for the data.

UnifiedLogs

This unifiedlogs parser tool is natively provided on a MacOS system. Fortunately some entities developed a linux compatible parser.

By default sysdiagnose will use the Apple unifiedlogs log binary. On linux it expects the Mandiant developed UnifiedLogs tool to be present in the path. Follow below instructions to compile and install it on your system.

Building macos-UnifiedLogs for linux

First, ensure cargo is installed so you can build rust projects.

sudo apt install cargo

Now you can download and compile the code:

git clone https://github.com/mandiant/macos-UnifiedLogs
cd macos-UnifiedLogs/examples/unifiedlog_parser_json/
cargo build --release
sudo cp ../target/release/unifiedlog_parser_json /usr/local/bin/

See unifiedlog_parser_json --help for more instructions to use the tool, or use it directly through sysdiagnose.

Contributors

• Dario BORREGUERO RINCON (European Commission - EC DIGIT Cybersecurity Operation Centre)
• David DURVAUX (European Commission - EC DIGIT Cybersecurity Operation Centre)
• Aaron KAPLAN (European Commission - EC DIGIT Cybersecurity Operation Centre)
• Christophe VANDEPLAS (European Commission - EC DIGIT Cybersecurity Operation Centre)
• Emilien LE JAMTEL (CERT-EU)
• Benoît ROUSSILLE (European Parliament)

License

This project is released under the European Public Licence https://commission.europa.eu/content/european-union-public-licence_en