DNS: Size of content interception

DNS: Size of content interception

From TV and mobile phones to smart light bulbs and smart cars, all things worldwide can be connected to the Internet. Given that advertising and advertising trackers can be seen everywhere on the Internet, it is not enough to use browser-based advertising interceptors. They only open a small window through “pure Internet”, can only avoid annoying banners and pop-up ads. However, if the user wants to change this small window to a door, what should I do?

Tighten the seat belt and ready to travel through the DNS filtration, now and future time travel. why? Because DNS is our answer!

What is the DNS level interception?

DNS refers to the “domain name system”. Its goal is to convert the name of the name into the IP address that the browser can recognize. That is, DNS can be said to be an Internet “address list”. Therefore, each time the user access the website, the browser can send DNS requests to a particular server (DNS server) to determine the IP address of the website. General DNS parsers will return directly to the IP address of the requested domain.

User’s device always uses a DNS server to get an IP address of the domain name you want to access.

Some DNS servers provide interception of DNS levels. General, interception is conducted by DNS Sinkholing. DNS Sinkholing technology refers to an address that is assigned to a blocked domain. When the user device sends “bad” request, including the advertisement or tracker, the DNS server responds to the blocked domain to the 0.0.0.0 IP address. Therefore, App will not connect to this address, and the result we have acquired is to avoid connectivity.

An example of using DNS server from accessing domain access

Another way is to return nxdomain (this means that the domain does not exist) or refused (this means that the server refuses to handle a request). Everyone may not understand the differences of them, but some old devices may misunderstand the refused and attempt to use the alternate DNS server.

DNS predecessor – Hosts file

Amazing is that the DNS level intercepts the oldest method of advertising interception. how is this possible?

Back to the Internet is also very “young”, called Apa Net, has a design name system that matches the computer’s IP address with a unique ASCII-based identifier. At that time, only a few computers were connected to the Apa net, the US Department of Defense Network Information Center supported the so-called hosts.txt file. This is the main list of each computer address and its hostname. The system administrator will periodically download hosts.txt. At that time, if you know the name of the Apa net, you can view its IP address on your hosts file.

This is the appearance of the Apa.com Hosts.txt file

But soon, people have found that because the number of hosts has increased, the NIC (Network Information Center) computer is too high. Everyone wants to connect their computers into Apa.com, but they all have to wait for NIC update hosts.txt files. Therefore, Paul Mockapetris in 1987 invented the DNS domain name system.

Although he invented a domain system, the Hosts file continued to exist. After that, when the Internet ad will appear, the user discovers the hosts.txt file can be used as a prevention list to avoid advertising loading (this method is very similar to DNS depression).

One realize this, some users start to create the first prevention list in history and share them with others. We have something worth seeing 20 years ago, but it still continues to be improved. For example,, or list.

Peter Lowe Block List Segment

When the first version of the Adblock browser appears, some users don’t understand their tasks. Users are very satisfied with the effect of using the HOSTS file.

Part of the 2003 Forum

However, Hosts files have some difficult problems.

Distribute an updated version. There is no effective way to provide updated HOSTS files. Users need special software to do this, or users need to be patiently updated regularly.

Hosts file is very large. Since users cannot use wildcards and need to be included in each domain, HOSTS files are often very easy. The Hosts file syntax should not be used in this way. The following is an image of the longest preactive list (ENERGIZED Unified). As you can see, it contains more than 700,000 domains. Its size is about 21 megabytes. How to share 21 MB of users every day is not a simple problem. Energized Unified Blocks list

Later, the intercepting public DNS server appeared as an alternative to using the HOSTS file. Just let the configuration of a device use the server, you can no longer process the update of the HOSTS file. One of the first public DNS servers is in 2016, called Alternate DNS, only one person handles it. Since 2016, we also released the AdGuard DNS.

New beginning of DNS interceptors

Until recently, DNS has almost snowed as industry standards. In recent years, only the small repair minus. But recently this new standard presents an explosive growth. It is mainly due to the rise of DNS encryption, including DNS-OVER-TLS, DNS-OVER-HTTPS, DNS-OVER-Quic. They have become mainstream, and the content interception has produced a major impact!

DNS encryption is hot in front of ordinary users, the way is too complicated:

Use the HOSTS file and try to update the file when statutory.

Using a normal DNS server and accepts the limits they exist, such as those who cannot be used on mobile phones. Or, configure each network adapter at the desktop side and understand the lower level settings.

Use VPN (or local VPN) software.

After DNS encryption, some technical obstacles disappear directly. Now configuring DNS servers relatively simple and it can be used for any network. Of course, users still need to understand device settings, but they are not as complex as before. DNS encryption is available in Android 9+ (DOT, will have DOH), iOS 14+ and MacOS 15+ (DOT, DOH), Windows 11 (restricted) has local support. All of this has played a huge role in the popularization of DNS interception.

In addition, the DNS encrypted invention allows the public DNS server to provide custom options to users, in other words, each user has an opportunity to select content to be shielded or not shielded. Custom rules that have previously used DNS-based server domains can only be performed by “connection” user IP address.

Use DNS to intercept content: profit and disadvantages

There is also a shortcomibility of using DNS to intercept content. Let us list some of the pros and cons:

advantage

No need to install additional software

Do not depend on the browser or operating system

There is no effect on performance

Enabling public DNS servers allow users to view full internet. If the user has a prevention list, the above will be helpful. Users can cancel the rules that are not used and promptly understand new threats. DNS has no blind spot because it observes all devices, not just browser.

Centralized solutions are more advantageous when dealing with some problems.

For example, let’s take a look at CNAME camouflage. This is some strategies used by some trackers to avoid interceptors. Through the CNAME DNS record, they hide the actual third-party domain name, disguise it into the first domain name. Using AdGuard DNS we can find all camouflage domains and publish their list, and even content interceptors that cannot access DNS can also intercept these domain names.

Another example is that the agent is used to disguise the tracker. Basically, you can use CloudFlare or CloudFront to configure a server-free agent, and it hides the original third-party domain. In general, DNS can’t help users solve this problem, but it can be a good starting point for detecting such agents.

shortcoming

Unable to handle the first advertisement. For example, users cannot block YouTube video advertisements because they are on the same domain with legal video hosting.

No modified filtering. Use DNS to intercept users without most of the ads, but there are damaged frames and advertising placeholders.

The possibility of damage is higher. For example, some applications or websites have been damaged due to Google Analytics, and you are very passive.

More easily bypass. Applications can easily choose another DNS server.

Can improve items

DNS filtration is some of the disadvantages that are easily repaired or at least alleviated.

Reject access error (Access Denied Error Page)

Currently, if there is a domain being intercepted at the DNS level, the error page will be displayed to the user. It is miserable to the user. For example, the user may want to temporarily unpack, but there is no way to do it. The good news is that there is an RFC document (a series is scheduled). The aesthetic error page will be allowed to display the user instead of throwing an ugly solution (such as requiring the user to install the HTTPS protocol) to the user.

Detecting the working principle of encryption DNS

Now you have to configure DNS on each device. So is it more convenient to set DNS on a router? Some new routers have supported local DNS encryption. However, many old routers still only support ordinary DNS servers. There is a working group called Adaptive DNS Discovery, which is committed to improving this situation.

Really payable RFC project:

The encrypted DNS server is allowed to detect the encrypted DNS server through a particular DNS record. Its working principle should be, the user first configures a normal DNS server, then the operating system sends a specific DNS request to it, and ask if there is available encrypted DNS request.

Another worth noting RFC project is used. This option will allow the router to set the encrypted DNS to the computer connected to the router.

APP detection

There is still something that can be improved. We need to learn which application sends which DNS request sent. There is no RFC that can help us, but we can try to build a “DNS fingerprint”. At least a hot application is prepared.

If we know which app sent DNS request, we can more flexibly control the intercept process.

For example, we want to intercept Facebook tracking, but comprehensively intercept it will result in an error over the entire Facebook application, so we want to do this only in applications that do not belong to Facebook.

Another realistic issue is that intercepting AppsFlyer (mobile analysis system) has caused some popular applications to be destroyed. We tend to selectively allow Appsflyer run – just for affected applications, otherwise it will intercept.

Building this “DNS fingerprint” needs to analyze the network behavior of each popular application, we are trying to find a solution to achieve this goal.

What is AdGuard DNS?

If you want to use “better Internet”, then DNS and VPN and advertising interceptors must be your left right arm. Let us read it carefully.

So what is AdGuard DNS? This is one of the most refined privacy DNS services on the market. It supports reliable encryption protocol, including DNS-OVER-HTTPS, DNS-OVER-TLS, and DNS-over-Quic. It can identify requests for advertising, tracker, adult website (selective) and respond with blank. AdGuard has its own hosted advertising, tracker, fraud website domain name, and we will update this list frequently. In addition, users can add their own custom rules in the prevention list.

In order to let everyone know what AdGuard DNS is, I will mention some points:

It sets multiple servers in 14 locations.

These servers are “declared” the same IP address through BGP (boundary routing protocol).

Its current load is about 300,000 DNS requests per second.

The AdGuard DNS writes in golang language.

75% of the up and down DNS traffic is encrypted. This is the difference between the DNS server and other servers. If the user looks at CloudFlare or Quad9’s statistics, it can be found that encrypted DNS traffic accounts for only a small part of all requests. But for the ADGUARD DNS, the situation is completely different. Most of the traffic is encrypted.

DNS filter rule grammar

The ADGUARD DNS blocks the list with special components. We don’t like the preactive list restriction of the HOSTS file type, so we use the syntax of advertising interception of advertising rates in AdGuard DNS.

Example of DNS filtering rules. Complete syntax rules can find on Github

Let us show some differences. The left is the statistics of the AdGuard DNS filtered (we use by default). It is relatively short, but it actually prevents approximately 900,000 domain names. On the right, another block list – Energized Ultimate. It contains a HOSTS file for 500,000 domains, you can intercept 50,000 domains.

Statistics according to AdGuard DNS data Adguard DNS server node

Each ADGUARD DNS server node includes several parts. On the client interface We have a DNS forwarder written in Golang language. It implements all logic, including DNS filtration, caching, and more.

In terms of hardware and software of the service, we have a non-bound server instance. It provides DNS recursive. The load of each server node is between 5,000 and 40000 responses per second. This depends on the position of the node (some positions are more popular than other locations).

DNS filtered engine

The DNS filter engine is open source, so everyone can view on GitHub. There are two, the first one is. This is our Golang library we used on AdGuard DNS and AdGuard Home. The first engine provides relatively high performance and lower memory usage. The second is. This is the C ++ library we used in the client application. In addition to DNS filtration, the engine also allows DNS encryption.

If you want to protect yourself from ads and trackers with AdGuard DNS, you can follow it.

Looking to the future

Browser-based content interceptors are more popular among ordinary users. But for filter list makers, they are not mainstream. The HOSTS file and domain name list created by volunteers and the domain name are far more than the browser interceptor filter list. This shows that there is a big foreground in the DNS level.

According to Filterlists.com, statistics

All in all, what is the future of DNS content interception? First, DNS content interception will continue to grow at high speed. Windows 11 brings support for DNS encryption, and the new router also supports DNS encryption, allowing users to set up and use DNS to intercept.

The ratio of the interception of the DNS content is also increased, but will still be slightly. Bypassing it requires “multi-party cooperation”, but if you want to make this thing, DNS content should be popular like a browser interceptor to promote “multi-party cooperation.” As far as it is, although the interception rate of DNS content is high, that high penetration rate may still need to practice for a long time.

DNS will not replace the reason for the browser-based interceptor, and its quality is not so good. Despite this, DNS interception will continue to exist and occupy their own position in terms of users’ privacy and security on the Internet.

Uncategorized