Log HTTP/S traffic

[chrisfrederik]

Hello,

I am trying to use mitmproxy as tool to log HTTP/S traffic. I read through the documentation and got it to run. I aim to schedule mitmdump to run on reboot with crontab. A log file is being created, but how can I change it to a human readable format? Basically I only need the date/time, method and requested URL in CSV or any other format that is easily read- and searchable. Similar to the view you get, when you just start mitmproxy on the command line. How can I achieve this and is mitmproxy the right tool for this?

Thanks for your time :)

[emanuele-em]

Hi! I think that you can use mitmproxy addons, I write a working but simple script for you that should do what you need:

from mitmproxy import ctx

class TrafficLogger:
    def __init__(self):
        self.log_file = open("traffic_log.txt", "a")

    def response(self, flow):
        self.log_file.write("=============================\n")
        self.log_file.write(f"Request: {flow.request.method} {flow.request.url}\n")
        self.log_file.write(f"Response: {flow.response.status_code}\n")
        self.log_file.write("=============================\n\n")
        self.log_file.flush()

    def done(self):
        self.log_file.close()

addons = [
    TrafficLogger()
]

save it as traffic_logger.py. After that you can just run it with

mitmdump -s /path/to/traffic_logger.py

this is the output inside traffic_log.txt

[chrisfrederik]

Thanks a lot, I will try it out and report back. But it looks like this is exactly what I need. So this is the preferred/intended way of manipulating the output to the log files?

[rosydawn6]

This was an example to show what can be done. Typically leveraging python's logging framework should be considered. Based on your requirements the addon could be extended to use https://docs.python.org/3/library/logging.html