DNS_PROBE_FINISHED_NXDOMAIN error

[kkw0432]

Page to Investigate

https://queens-web.vercel.app

Steps to Reproduce

my vercel app has DNS problem.
it work well yesterday but when I tried to access it now, it failed.
our app already deployed using that domain, so I can not switch to other domain now..
Please check and resolve this issue quickly.

Thank you

[spoyodevelop]

Seems like and somehow our korean ISPs blocked vercels DNS(vercel.app) for some dumb reason. so basically you can use any https bypass app like icloud Private Relay or unicorn https to connect in your website.
which is not a solution or anything, but still we can know where exactly problem is.

[spoyodevelop]

Use https://www.digitalocean.com/ or https://netlify.com/ for alternative.
Here is link for who are interested in digitalocean deployment https://youtube.com/watch?v=1ldjq5M7cbw

[spoyodevelop]

or use custom domain whichever not using (vercel.app) if you can.

[kkw0432]

oh... is there no way but to wait?
as I metioned my app is using this domain..
I needs app update to change the domain url.. it will take few days. in addition it is weekend.
if it takes a lot of time, I'll prepare an app update

[kkw0432]

�I'm sorry service has been stopped so long time..

[kkw0432]

now it is working well.
I will change to custom domain so that I can deal with like this status.

[SeokyoungYou]

I am in Korea and also could not access the preview URL using the Vercel domain.

[safethecode]

The project with the connected domain seems fine, but accessing it via *.vercel.app address doesn't seem to work.

[idchoi2]

fuck korean ISPs

[PurpleSand123]

Me too. My vercel app has same issue

[makepado]

Korea did Korea

[Nangniya]

Me too. I purchased a different domain name and connected it newly. 😢

[Socrad]

Me too. I had to change nginx config of backend server for CORS.
Now I'm afraid of missing something

[hiddenest]

For the people who are experiencing this issue:
Could you run this bash commands on your computer and share the result here?
It would be really helpful for Vercel team to resolve this issue.

https://gist.github.com/smaeda-ks/5997018d5fe5dce44d047299829288e1

[hiddenest]

@spoyodevelop It looks like you are using U+(LG Dacom) ISP.
In the last 2 week, KT/SKT/SKB had blocked 76.76.21.22 domain, which is locating to one of Vercel CNAME IP address. (Now it is resolved) (Issue Link)

My guess is that the Korean government asked ISPs to block that IP addresses.
...and it's a problem now because U+ reflected it slowly compared to KT/SKT.

[Socrad]

No. My vercel domain also has same problem from today morning, and everyone in Seoul(maybe whole Korea) can't connect to the service. I'm using SKT, and the other using KT, and everyone has same problem.

[spoyodevelop]

I am used that website for better visual representation, not for very accurate information so take them with grain of salt. I am currently using KT and I tried other ISPs and seems no one works.

[hiddenest]

Hmm... It could be quite difficult to debug it then.
Because when I made this comment, I've tested it on both KT and SKT and there was no problem (it is working now, too)

[sko-kr]

Seriously, I feel like we have dictatorship when it comes to ISPs in Korea.
They blocked r2storage (cloudflare r2 storage) now vercel.app.

[henrynoowah]

Hi, guys, I just checked and want to let y'all know that it is now working

[spoyodevelop]

It do not works on my machine, i am using KT as ISP now too.

[spoyodevelop]

No idea but this might give you hint
https://www.reddit.com/r/nextjs/comments/pl11tc/vercels_ip_is_blacklisted_and_inaccessible_from/

[Socrad]

dnschecker.org still shows the domains don't working in Korea.

[Socrad]

And other people says it doesn't working.

[kkw0432]

my project is working now.

[mcsdevv]

Hi all, sorry to hear you're once again having difficulty - we're more than happy to take this to the ISP once we have a little more detail.

As @hiddenest mentioned, can those unable to access vercel.app domains run the following commands? https://gist.github.com/smaeda-ks/5997018d5fe5dce44d047299829288e1

This will allow us to take more evidence to the ISP that there is indeed an issue and aim to resolve it quicker.

[smaeda-ks]

@spoyodevelop Your dig command is wrong. It shouldn't be a URL, just the domain name. Could you get that?

$ dig didibringmyumbrella.vercel.app

$ dig didibringmyumbrella.vercel.app @8.8.8.8

Anyway, seems they put DNS filtering for *.vercel.app domains. If switching to a public DNS service (e.g., 8.8.8.8) works then that's most likely it. I'm not seeing it from @winterlood 's latest result, but if you're still seeing issues please get fresh one so we can reach out to the ISP as needed.

[spoyodevelop]

Spoy@Marvins-MacBookAir ~$ dig vercel.app

; <<>> DiG 9.10.6 <<>> vercel.app
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49765
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;vercel.app.			IN	A

;; ANSWER SECTION:
vercel.app.		600	IN	A	76.76.21.164
vercel.app.		600	IN	A	76.76.21.9

;; Query time: 34 msec
;; SERVER: 192.168.50.1#53(192.168.50.1)
;; WHEN: Sun May 12 12:23:57 KST 2024
;; MSG SIZE  rcvd: 71

Spoy@Marvins-MacBookAir ~$ dig vercel.app @8.8.8.8

; <<>> DiG 9.10.6 <<>> vercel.app @8.8.8.8
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 22301
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;vercel.app.			IN	A

;; ANSWER SECTION:
vercel.app.		555	IN	A	76.76.21.93
vercel.app.		555	IN	A	76.76.21.61

;; Query time: 33 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sun May 12 12:24:02 KST 2024
;; MSG SIZE  rcvd: 71

sorry, i am not sure i got them correctly because i am complete newbie and this is result you want i think.

[smaeda-ks]

if DNS_PROBE_FINISHED_NXDOMAIN was happening, I'd assume it was a typical DNS blocking. But even in the last one from @spoyodevelop, I don't see anything wrong.

@Socrad @unhyif do you know if they're also using Korea Telecom as well? And if so, could you get the same ones as well?

$ dig foo.vercel.app

$ dig foo.vercel.app @8.8.8.8

[Socrad]

I already changed domain name, and ~.vercel.app domain works well and redirects to new domain now. But dnschecker.org still shows that some DNS don't work in Korea.

[smaeda-ks]

So far, no one has provided clear evidence of blocking, which is a problem. If DNS blocking is live, those two dig commands will show different outputs.

@Socrad @khj0426 Does the issue resolve if you switch to a public DNS service? 8.8.8.8/1.1.1.1 if so, it's most likely DNS filtering. But then we need the evidence. Please help us collect the logs so we can escalate it to the ISP.

[Socrad]

Today I checked connection via dnschecker.org, and it seems that some Korean DNS don't work.

[unhyif]

2 users of my service reported having trouble accessing the page on May 12th at 18:11 and May 13th at 09:30 (UTC+9).

[khj0426]

Today I checked my all vercel domain not work.. it seems that some DNS not works

[smaeda-ks]

@khj0426 What's your ISP? Could you please run these and report back to us?

$ curl https://ipinfo.io

$ dig vercel.com

$ dig foo.vercel.app

$ dig foo.vercel.app @8.8.8.8

[khj0426]

 {mongdiwoo} 🎉   ~/Documents/codingProject/dreaming_ver2/dreaming   feature/51 ±  curl https://ipinfo.io                    
{
  "city": "Paripark",
  "region": "Seoul",
  "country": "KR",
  "loc": "37.4883,126.8255",
  "org": "AS3786 LG DACOM Corporation",
  "postal": "08359",
  "timezone": "Asia/Seoul",
  "readme": "https://ipinfo.io/missingauth"
}%                                                                                                                                                                  
 {mongdiwoo} 🎉   ~/Documents/codingProject/dreaming_ver2/dreaming   feature/51 ±  dig vercel.com

; <<>> DiG 9.10.6 <<>> vercel.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62224
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;vercel.com.                    IN      A

;; ANSWER SECTION:
vercel.com.             12      IN      A       76.76.21.9
vercel.com.             12      IN      A       76.76.21.61

;; Query time: 47 msec
;; SERVER: 10.0.100.1#53(10.0.100.1)
;; WHEN: Mon May 13 16:18:10 KST 2024
;; MSG SIZE  rcvd: 71

 {mongdiwoo} 🎉   ~/Documents/codingProject/dreaming_ver2/dreaming   feature/51 ±  dig foo.vercel.app

; <<>> DiG 9.10.6 <<>> foo.vercel.app
;; global options: +cmd
;; connection timed out; no servers could be reached
 ✘ {mongdiwoo} 🎉   ~/Documents/codingProject/dreaming_ver2/dreaming   feature/51 ±  dig foo.vercel.app @8.8.8.8

; <<>> DiG 9.10.6 <<>> foo.vercel.app @8.8.8.8
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 58213
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;foo.vercel.app.                        IN      A

;; ANSWER SECTION:
foo.vercel.app.         600     IN      A       76.76.21.9
foo.vercel.app.         600     IN      A       76.76.21.22

;; Query time: 68 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Mon May 13 16:18:38 KST 2024
;; MSG SIZE  rcvd: 75

[hhkwon]

Now, It works on LG ISP. Probably solved.

[r-4bb1t]

In my case, all services deployed using Vercel are still inaccessible via the LG network.

[Update 2024-05-19] I found the occurrence of this in the domain 76.76.21.22, 76.76.21.241.

[Huinno-ParkJinHyun]

again.., it happend

fyi : 2024 05 20 17 50 ~ 18 30 (utc +9 seoul) vercel serverless ip was blocked by isp (kt & lgu+)

[hiddenest]

Could you run this command and share the result?
#6807 (comment)

[code-xhyun]

I hate Korea ISPs

[kgneng2]

The ISP has started blocking again. It's LG U+.....

FUCK.....

[developerjhp]

https://github.com/developerjhp/vercel-dns-isp-block-watch-dog
KT, SKT, LGU+ DNS 서버를 통해 Vercel로 배포된 도메인의 접근 가능 여부를 모니터링하고, 문제가 발생하면 Slack으로 알림을 보내는 코드를 만들었습니다.

(isp에 의해서 block된 경우 aws나 netlify 로 배포된 리소스에 도메인을 연결하시는 방법이 그나마 가장 나은 해결책인 것 같습니다.)

github action은 cron 스케쥴링이 지멋대로여서.. 로컬에서 돌리시는 걸 추천드립니다~