Skip to content

This script it's used to disable the attribute 'to_ids' on MISP events, features removal of the IDS tag on old events or based on Reputational datasets results.

License

Notifications You must be signed in to change notification settings

LZappy87/toids_remove

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

LZappy87 - toids_remove License Languages CodeFactor.io issues - toids_remove LastCommit

stars - toids_remove forks - toids_remove

MISP IDS Tag Remover

CREATED BY: LZappy87

LAST VERSION: 2.0

CREATED ON: 03/02/2022

UPDATED ON: 07/04/2022

FILES USED

  • toids-remove.py (this script)
  • keys.py (the configuration file)

TESTED WITH

  • MISP 2.4.152
  • PyMISP 2.4.152
  • Python 3.8
  • VirusTotal APIv3
  • AbuseIPDB APIv2
  • Greynoise APIv3

LIBRARIES USED

  • contextlib (contextmanager)
  • sys
  • os
  • shutil
  • time
  • argparse
  • prettytable (PrettyTable)
  • pymisp (ExpandedPyMISP)
  • urllib3
  • requests
  • base64
  • requests
  • re

DESCRIPTION

This script it's used to disable the attribute 'to_ids' on MISP events, features removal of the IDS tag on old events or based on VirusTotal scan results. An idea developed from this article: https://www.vanimpe.eu/2019/09/24/tracking-false-positives-and-disabling-to_ids-in-misp/

USAGE & DEMO

Help Menu

help

Remove Mode (--mode rem)

rem

Reputation IDS Removal Mode (--mode reputation)

reputation

Sights Only Mode (--mode reputation --sightsonly True)

sightsonly

Sights IDS Removal Mode (--mode reputation --sightsrem True)

sightsrem

Demo (v 1.6, Reputation Mode)

toids_removev1.6.mp4

CHANGELOG

v 2.0 (07/04/2022):

  • Implemented sightsonly for Reputation mode: populate only sightings without removing IDS tags;
  • Implemented sightsrem for Reputation mode: remove IDS tags based on sightings\false positive percentage;
  • Removal of redundant code;
  • Overhaul of script messages (for better understanding).

v 1.8 (06/04/2022):

  • Testing sightings mode;
  • Further code optimization.

v 1.7 (29/03/2022):

  • Added sightings based on reputation results.

v 1.6 (27/03/2022):

  • Minor changes to the code;
  • Added further VTotal tags to maltag.

v 1.5 (25/03/2022):

  • Added Greynoise API to reputation mode;
  • Added tabled results at the end of the script;
  • Removed unnecessary script header informations.

v 1.4 (21/02/2022):

  • Added AbuseIPDB to the vt mode;
  • vt mode now is reputation mode;
  • remold mode is now rem mode;
  • Added minimal error handling for AbuseIPDB API.

v 1.3 (08/02/2022):

  • Implemented VirusTotal Mode (vt);
  • Implemented Remove Old Mode (remold);
  • Included arguments to launch the script;
  • Moved some variables to keys.py for better configuration;
  • Included the 'published=True' search constraint (this should speed up the queries);
  • Removed mintime and maxtime from keys.py, substituted with arguments --mintime --maxtime;
  • Overall revamp of the code.

v 1.2 (05/02/2022):

  • Preparing for VirusTotal API implementation

v 1.1 (04/02/2022):

  • Removed old search string (it was not getting all the attributes);
  • Added filtering based on event tags through build_complex_query (thanks Giuseppe for the idea);
  • Various code revamp (not necessary linked to the aforemended changes);
  • Moved misp_client_cert to keys.py;
  • Added: misp_excluded_tags (for tag exclusion), mintime and maxtime (for time reference regarding the query on MISP) on keys.py;
  • Added basic error handling;
  • Added the creation of a default 'keys.py' if not present.

v 1.0 (03/02/2022):

First release

TODO:

  • Add more API's (for domain\url);
  • Better error handling.

About

This script it's used to disable the attribute 'to_ids' on MISP events, features removal of the IDS tag on old events or based on Reputational datasets results.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages