Skip to content

j-moriarti/pDNSf-Hosts-collection

Repository files navigation

In this Repo, I've created a shell script to automate the process of collecting and compiling all the Hosts from various sources, then cleaning and optimizing it to be used in "PersonalDNSfilter"
This collection is suitable for blocking the domains containing ads, tracking, malware, phishing, ransomware, spyware, trojans and virus (for PC and Smartphones).

ℹ️ Updates & News πŸ“’

2021/07/27
πŸ“Œ Added support for converting sources from [AdBlockPlus/uBlock/AdGuard] format to Domains/Hosts format!
πŸ“Œ Added many new sources!
πŸ“Œ Improved the code to reduce false positives.

Expand to see previous changes!

2021/07/18
πŸ“Œ Improved checking TLDs of domains + fixes
πŸ“Œ Implemented "idna to dns (utf8)" converting function for domains and TLDs.
πŸ“Œ Optimized the code speed.
πŸ“Œ Added support for checking more TLDs (IANA / Blockchain-registered / OpenNIC / FurNIC / Emercoin / New Nations / Onion).

2021/07/10
πŸ“Œ Implemented personal Allow-List for falsely blocked domains.

2021/07/03
πŸ“Œ Implemented parallel downloading of sources.(disabled by default due to github limitations)
πŸ“Œ Separated the sources list from code.

2021/03/18
πŸ“Œ Improved removing redundant subdomains! (thanks @badmojr for reporting πŸ˜‰)

2021/03/15
πŸ“Œ Implemented removing redundant subdomains!
πŸ“Œ Greatly reduced blocklist size without affecting the quality!(same blocking power)
πŸ“Œ The original blocklist (without whitelisting / subdomains removing) is also available in Section "B: (Manual Method)"

2021/03/13
πŸ“Œ Implemented Whitelisting: For now I apply mkb2091's whitelist source!
πŸ“Œ Also, the original Hosts is preserved, too (without applying whitelists)↩️
   You can find it in Section "B: (Manual Method)"
πŸ“Œ To get statistics about the (input) sources used in this Repo, check 'stats' file.
πŸ“Œ The previous Hosts releases and backups of the input sources (in compressed format)↩️
   can be found in 'Releases' page, under tha tag 'v1.0.0-backup'.



⚠️ Disclaimer:

Click to expand!

The Hosts sources that are used in this Repository, have their own Licenses. The links of the HOSTS sources that are used in this repo (which includes their own licenses) can be found here: "sources.conf". The compiled hosts in this Repo are meant for my personal usage only. I have no responsibility about others misusing of these files.

⚠️YOU HAVE BEEN WARNED⚠️ πŸ˜‰

ℹ️ Notes:

Click to expand!

This hosts file is specially optimized for "PersonalDNSfilter" app. This Open-Source app can handle Huge number of Domains without affecting on battery or ram usage! So there would be no problems adding these hosts to it!

!!! THIS HOSTS FILE IS EXTREMELY LARGE, and only pDNSf can easily handle it !!!
!!! Although applied whitelists to remove falsely blocked domains, but There may be a possibility of FALSE-POSITIVE or UNWANTED BLOCKING !!!
!!! Be prepared for manual whitelisting in pDNSf !!!
!!!YOU HAVE BEEN WARNED AGAIN!!!


ℹ️ Hosts Info

[Original Blocklist]
Blocked Domains: 4,135,875
Blocklist size: 94.5 MB

[Optimized Blocklist πŸš€]
Blocked Domains: 2,211,465
Blocklist size: 43.4 MB

Number of Blocked IPs: 246,518
IP Blocklist size: 3.4 MB

Auto Updates: Every 24 hours



A: (Set & Forget Method)

Download Hosts in multi-part (Domains-only Format)

(Add these links directly to pDNSf config)

(Additionally, you also can add IP blocklist.(links can be found at 'Extra Hosts' table below))

Name Download Link Mirror Link Size
pDNSf-hosts-part0.txt Domains (Part 0) Mirror (Part 0) 14.5 MB
pDNSf-hosts-part1.txt Domains (Part 1) Mirror (Part 1) 14.5 MB
pDNSf-hosts-part2.txt Domains (Part 2) Mirror (Part 2) 14.5 MB


B: (Manual Method)

Download GZip Compressed Hosts (Domains-only Format)

(First, download and extract (uncompress) 'pDNSf-hosts.txt.gz' to somewhere, then add the LOCAL PATH of 'pDNSf-hosts.txt' to pDNSf config)

Name Download Link Mirror Link Size
pDNSf-hosts.txt.gz Download Mirror 13.3 MB
pDNSf-hosts_no-whitelist.txt.gz Download Mirror 23.6 MB

ℹ️ Notes:

Click to expand!

πŸ“Œ Example of a LOCAL PATH format in pDNSf:
PC version: file:///D:/Downloads folder/pDNSf-hosts.txt
Mobile version: file:///sdcard/Downloads/pDNSf-hosts.txt

πŸ“Œ Use "pDNSf-hosts_no-whitelist.txt.gz" if you want to use the Hosts without applying "whitelists" or "redundant subdomains removing".



Download Extra Hosts (IPs-only & Domains-only Format)

Name Download Link Mirror Link Size # of Lines
CIDR-IPs Block-List Download Mirror 84.6 KB 5,344
Wildcard Block-List Download Mirror 8.1 KB 441
IP Addresses Block-List Download Mirror 3.4 MB 246,518