[OpenWrt-Devel] Slow DNSMasq with > 100, 000 entries in additional addresses file

[Dave Taht]

On Tue, Dec 27, 2016 at 11:03 PM, TheWerthFam <thewerthfam at gmail.com> wrote:
> Thanks for the feedback, I'll look into NFQUEUE.  I'm forcing the use of my
> dns by iptables.  I'm also using a transparent squid and e2guardian to
> filter content.  I like the idea of the dns based blacklist to add some
> filtering capabilities since I don't want to try and filter https types
> sites.  I know no solution in perfect.

I've been thinking about this, and given the large amount of active
data in a very small memory space have been thinking that another
approach would be more fruitful. Convert the giant table into a
"minimally perfect hash", and mmap it into memory read-only, so it can
be discarded under memory pressure, unlike ipset, squid, or dnsmasq
based approaches.


> Cheers
>  Derek
>
>
>
> On 12/27/2016 01:53 PM, philipp_subx at redfish-solutions.com wrote:
>>>
>>> On Dec 26, 2016, at 10:32 AM, TheWerthFam <thewerthfam at gmail.com> wrote:
>>>
>>> Using the adblock set of scripts to block malware and porn sites. The
>>> porn sites list is 800,000 entries, about 10x the number of sites adblock
>>> normally uses.  With the full list of malware and porn domains loaded,
>>> dnsmasq takes 115M of memory and normally sits around 50% CPU usage with
>>> moderate browsing usage.  CPU and RAM usage isn't really a problem other
>>> than lookups are slow now. Platform is cc 15.05.1 r49389 on banana pi r1.
>>>
>>> The adblock script takes the different lists, creates files in
>>> /tmp/dnsmasq.d/ entries looking like
>>> local=/domainnottogoto.com/   one entry per line.  The goal is to return
>>> NXDOMAIN to entries in the lists. Lists are sorted and with unique entries.
>>>
>>> I've tried increasing the cachesize to 10,000 but that made no change.
>>> Tried neg-ttl=3600 with default negative caching enabled with no change.
>>>
>>> Are there dnsmasq setting that will improve the performance?  or should
>>> it be configured differently to achieve this goal?
>>> Perhaps unbound would be better suited?
>>>
>>> Cheers
>>>     Derek
>>
>>
>> Not to rain on your parade, but the obvious defeat of this solution would
>> be to point to an external website which does DNS lookups for you, and then
>> edit the URL to have an IP address in place of the host name.
>>
>> I would use netfilter’s NFQUEUE and make a user-space decision based on
>> packet-destination (since it seems you’re filtering outbound traffic
>> requests).
>>
>> After all, it’s not the NAME you don’t want to talk to… it’s the HOST that
>> bears that NAME.
>>
>> -Philip
>>
> _______________________________________________
> openwrt-devel mailing list
> openwrt-devel at lists.openwrt.org
> https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel



-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org
_______________________________________________
openwrt-devel mailing list
openwrt-devel at lists.openwrt.org
https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel