#Setting the DNS Resolver (Tailscale)

1 messages · Page 1 of 1 (latest)

radiant cypress
#

Hello, I currently trying to reach resources that are under a private network, reachable through tailscale. (AWS)
I'd like to use the AWS resolver in order to use private route53 records. Is it even possible?

Thank you

ornate hill
#

you need to be aware that docker is quite sensitive to network changes so you need to make sure your Dagger engine is started after the machine is connected to tailscale

radiant cypress
#

Thank you for your answer @ornate hill , however my question is how to make that works. I mean how can I set the container resolver?

ornate hill
#

as Dagger will use your host's DNS server to resolve whatever destination you're trying to get to

#

and I'd assume that when you're connected to tailscale, they end up changing your DNS resolver so you can actually resolve your tailscale network hosts

radiant cypress
#

Well actually tailscale is started from a container too and I use a service binding. I'm going to commit something so that you'll understand better 🙂

radiant cypress
#

Now that I'm digging further, I'm not sure this is even possible. Even though I managed to forward all the traffic through the Tailscale container and let's say that I can change the resolver, I still don't know how to accept routes in the application container so that I could use a subnet router.
To sum up:

  • networking is working, I mean I can reach tailscale devices
  • magic dns doesn't work
  • I can't reach the private dns resolver because it needs a route to the AWS network that is set using the --accept-routes
    Here is my experiment: Service definition, Used here.
#

Maybe that would have been easier if I could share the tailscale socket instead of a service binding.

#

Setting the DNS Resolver (Tailscale)

radiant cypress
#

Update: nevermind, the routing works great. Now I just need to set the resolver

# 
dagger /bass/work $ dig @10.0.0.2 +short google.fr
142.250.179.99
#

Well after restarting Docker, it tooks the tailscale resolver and everything works as expected. Sorry for the inconvenience. That is working fine right now. Note: Docker should be started after Tailscale. Thanks again

#

However I'll need to check how would that works within Github Actions

#

Ideally I would need a parameter to configure dagger dns resolver

radiant cypress
#

I configured my local docker daemon.json so that I won't need to start tailscale at all

{
  "dns": [
    "100.100.100.100",
    "1.1.1.1"
  ],
  "dns-search": [
    "tail9c3xx.ts.net",
    "yourdomain.tld",
    "private.yourdomain.tld"
  ]
}
#

Then I'll need to figure out how to get this working in Github Actions

ornate hill
#

as long as you're using the ALL_PROXY option and the SOCKS proxy, the DNS resolution should still be done by the tailscale Dagger service

#

when you're using the ALL_PROXY variable, the dns resolution is not being done by the client. The hostname is sent "as is" to the proxy service (tailscale dagger service) and tailscale performs the dns resolution

radiant cypress
#

Indeed I was searching too far 😄 , that works without any changes and without running tailscale at all in the host. I had issues probably because I hit a rate limit of some kind, but after regenerating the api key and cleaning up things, everything worked.

ornate hill
#

I'm glad that your'e good to go. Don't hesitate to ask if you have further questions!

radiant cypress
#

Well, actually I figured out my issue, because in the end I still have an issue
Without having the host (my laptop) routing through the subnet router I can't reach the private resolver at all

dagger /bass/work $ dig @10.0.0.2 vault.priv.cloud.ogenki.io +short
;; communications error to 10.0.0.2#53: timed out
;; communications error to 10.0.0.2#53: timed out

Even though tailscale is started in my laptop I still not use the private resolver by default (empty result)

dagger /bass/work $ dig vault.priv.cloud.ogenki.io +short

The only way to make it work is to target the resolver

dagger /bass/work $ dig @10.0.0.2 vault.priv.cloud.ogenki.io +short
10.0.39.62
#

I'm gonna open an issue in their github repo to see what Tailscale suggests

ornate hill
#

I'm thinking about the dagger specific use-case of spawning tailscale as a service and use the userspace-networking

radiant cypress
#

I'd say that the main issue is that the routing through the subnet router isn't working through the Tailscale container. It worked because I've started my laptop's Tailscale instance.

tailscale-userspace-routing.png
#

The use case would be to be able to use a subnet router from the CI too and reach private resources in the cloud.

#

I'll improve this drawing this evening

ornate hill
#

I'd say that the main issue is that the routing through the subnet router isn't working through the Tailscale container.

it should work as long as you're using the ALL_SOCKS variable and your app can actually make use of that SOCKS proxy to access whatever service it needs to access in the private DNS zone

#

you shouldn't have to start tailscale in the host for this to work

#

as the Dagger tailscale service is the one that handles all the routing for you

radiant cypress
#

I'm probably missing something, I'm gonna try again then, starting everything from scratch

ornate hill
#

will report in a bit

radiant cypress
#

😢 The machine (the tailscale container) appears connected a few seconds in the console, then it disappears. There must be something here

#

opening a terminal in the tailscale container works, so I need to check the serviceBinding again. That's strange, I thought it was working but maybe through my local tailscale

#

my tailscale configure seems f**** up lol, having a deeper look this evening

ornate hill
#

ok, making progress here

#

ok @radiant cypress managed to make it work

#

your tsScript needs a change given that since you're starting tailscaled in the background, the service container exists as soon as your process finishes. So i've changed the script like this:

    tsScript := `#!/bin/bash
tailscaled --tun=userspace-networking --socks5-server=0.0.0.0:1055 --outbound-http-proxy-listen=0.0.0.0:1055 & \
tailscale up --authkey "$TAILSCALE_AUTHKEY"

wait $(jobs -p)
#

and this is an e2e example on how I use it:

func (cr *CloudNativeRef) Test(ctx context.Context) *dagger.Container {
    tsKey := dag.SetSecret("tailscale_key", "<redacted>")
    svc, err := tailscaleService(ctx, tsKey, "tail450d.ts.net")

    if err != nil {
        panic(err)
    }

    return dag.Container().From("alpine").
        WithServiceBinding("tailscale", svc.AsService()).
        WithExec([]string{"apk", "add", "curl"}).
        WithEnvVariable("ALL_PROXY", "socks5://tailscale:1055").
        WithEnvVariable("CACHE", time.Now().String()).
        WithExec([]string{"sleep", "10"}).
        WithExec([]string{"curl", "-v", "https://hello.ts.net"})

}

^ I'm adding the sleep 10 as tailscale takes a bit to register the node in the network and given that the healthcheck is currently set to 1055, that port is opened as soon as the tailscaled process starts

radiant cypress
#

yes I just made it works, I'm gonna commit my changes
But I'd say that the UDP doesn't

dagger /bass/work $ ping 10.0.22.112
PING 10.0.22.112 (10.0.22.112): 56 data bytes
^C
--- 10.0.22.112 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss
dagger /bass/work $ dig @10.0.0.2 vault.priv.cloud.ogenki.io
;; communications error to 10.0.0.2#53: timed out
dagger /bass/work $ curl 10.0.22.112:8200
curl: (1) Received HTTP/0.9 when not allowed
#

I pushed my changes and I'm gonna look for issues related to UDP

#

(DNS and PING don't work)

#

I had to remove the parameter --outbound-http-proxy-listen=0.0.0.0:1055 , I don't know why so far but that did the trick

radiant cypress
radiant cypress
#

I guess I have to wait for the next release. I'll keep you updated

#

thanks for your support!

ornate hill
ornate hill
ornate hill
radiant cypress
#

I can give it at try 💪

radiant cypress
#

unfortunately it doesn't work

image.png
# 
dagger /bass/work $ ping 100.114.240.15
PING 100.114.240.15 (100.114.240.15): 56 data bytes
^C
--- 100.114.240.15 ping statistics ---
4 packets transmitted, 0 packets received, 100% packet loss
dagger /bass/work $ curl 100.114.240.15:8200
curl: (1) Received HTTP/0.9 when not allowed
#

or I missed a new parameter

ornate hill
#

I don't think ping should work

#

that's ICMP

#

not UDP

radiant cypress
#

same with a call to a DNS server

ornate hill
#

i'd try something like netcat or telnet

#

I'm not sure if dig honors the ALL_PROXY variable

radiant cypress
#

strange because the curl works but not the netcat 🤔

dagger /bass/work $ nc -zv -w 3 100.114.240.15 8200
nc: connect to 100.114.240.15 port 8200 (tcp) timed out: Operation now in progress
dagger /bass/work $ curl 100.114.240.15:8200
curl: (1) Received HTTP/0.9 when not allowed
radiant cypress
# 
ncat --proxy tailscale:1055 --proxy-type socks5 -vz 100.114.240.15 8200
Ncat: Version 7.94 ( https://nmap.org/ncat )
Ncat: Connected to proxy 10.87.0.43:1055
Ncat: No authentication needed.
Ncat: Error: general SOCKS server failure.
#

I give up for today, I'll dig further tomorrow

ornate hill
#

nc -u -x localhost:1055 -X 5 100.83.213.57 1234
nc: no proxy support for UDP mode

seems like netcat doesn't support socks5 in udp mode

radiant cypress
#

I tried again this morning with the latest version but that didn't work or I don't have the proper test command.
Installed using

    // Installing unstable tailscale binaries
    binDir := dag.Arc().
        Unarchive(dag.HTTP("https://pkgs.tailscale.com/unstable/tailscale_1.71.72_amd64.tgz").WithName("tailscale_1.71.72_amd64.tgz"))
    tailscaledBin := binDir.File("tailscale_1.71.72_amd64/tailscaled")
    tailscaleBin := binDir.File("tailscale_1.71.72_amd64/tailscale")
ornate hill
#

@radiant cypress all this UDP dancing is to get DNS working inside the tailscale network, correct?

#

are you using MagidDNS in your case?

radiant cypress
#

Yes I want to be able to reach my private DNS resolver

ornate hill
#

@radiant cypress not sure if DNS is going to work though. Even if we get DNS to work in tailscale, you'd require your DNS client to honor the proxy settings and linux doesn't do that by default

#

so not sure how you were planning to overcome that

#

"ideally" tailscale userspace networking should support socks5h so their proxy performs the hostname resolution so you don't have to deal with that

#

@radiant cypress ok, managed to make it work

#

so tailscale also implements socks5h (even though they don't mention it in their docs)

#

so as long as you're u sing magicdns, it will resolve the hosts for you automatically

#

just set the socks server to ALL_PROXY=socks5h://$IP:$PORT

#

^ note the socks5h at the end

#

cc @gentle valley @urban void all these findings here are community call material 🙌

radiant cypress
#

👁️ I'll tell you in a few minutes, thanks @ornate hill you rock!

radiant cypress
#

Have you confirmed that on your side? unfortunately that didn't change anything on my side

env | grep socks
all_proxy=socks5h://tailscale:1055
ALL_PROXY=socks5h://tailscale:1055
ornate hill
#

are you using magicdns in your case?

radiant cypress
#

It is enabled, but that's not my target and I didn't manage to get the hostname resolved. Question: In your example above, you don't even declare the http_proxy environment variable and that worked.
In my setup I must declare it...

dagger /bass/work $ curl 100.93.210.48:8200
socks5: client connection failed: connect tcp 100.93.210.48:8200: connection was refused
curl: (97) cannot complete SOCKS5 connection to 100.93.210.48. (1)

dagger /bass/work $ export http_proxy=http://127.0.0.1:1055
dagger /bass/work $ curl 100.93.210.48:8200
socks5: client connection failed: incompatible SOCKS version
curl: (1) Received HTTP/0.9 when not allowed
ornate hill
#

$HOSTNAME should be your machine's magicdns hostname

radiant cypress
# 
ALL_PROXY=socks5h://127.0.0.1:1055 curl 100.93.210.48:8200
socks5: client connection failed: connect tcp 100.93.210.48:8200: connection was refused
curl: (97) cannot complete SOCKS5 connection to 100.93.210.48. (1)

 netstat -laputen|grep 1055
tcp        0      0 127.0.0.1:51444         127.0.0.1:1055          TIME_WAIT   -
tcp        0      0 :::1055                 :::*                    LISTEN      189/tailscaled

ps aux|grep tailscale
  189 root      0:01 tailscaled --tun=userspace-networking --socks5-server=:1055 --outbound-http-proxy-listen=:1055
# 
ALL_PROXY=socks5h://127.0.0.1:1055 curl ip-10-0-22-139:8200
wgengine: Reconfig: configuring userspace WireGuard config (with 2/5 peers)
[unexpected] magicsock: derp-18 does not know about peer [BfY3O], removing route
socks5: client connection failed: context deadline exceeded
curl: (97) cannot complete SOCKS5 connection to ip-10-0-22-139. (1)
ornate hill
#

is ip-10-0-22-139 the magicDns hostname?

#

that's the hostname you see in the tailscale dashboard for that machine?

radiant cypress
#

yes it is

ornate hill
#

@radiant cypress I have 10 minutes if you can jump into #911305510882513037 to check it together

#

since you're around..

radiant cypress
#

Sorry I'm at the office, I won't be able to find a quiet place right now :/

ornate hill
#

np

radiant cypress
#

Actually the dns resolution works

ALL_PROXY=socks5h://127.0.0.1:1055/ curl -fsSL --proxy-insecure -v https://ip-10-0-26-168:8200
* Uses proxy env variable ALL_PROXY == 'socks5h://127.0.0.1:1055/'
*   Trying 127.0.0.1:1055...
* Connected to 127.0.0.1 (127.0.0.1) port 1055
* SOCKS5 connect to ip-10-0-26-168:8200 (remotely resolved)
* cannot complete SOCKS5 connection to ip-10-0-26-168. (1)
socks5: client connection failed: connect tcp 100.93.210.48:8200: connection was refused
* closing connection #0
curl: (97) cannot complete SOCKS5 connection to ip-10-0-26-168. (1)
#

But there's something weird with the proxy itself

ornate hill
#

seems to be resolving the host correctly

#

maybe restart the proxy?? I was able to connect to an http endpoint without issues

radiant cypress
#

seting the http_proxy seems mandatory:

dagger /bass/work $ export http_proxy=http://127.0.0.1:1055
dagger /bass/work $ curl ip-10-0-26-168:8200
socks5: client connection failed: incompatible SOCKS version
curl: (1) Received HTTP/0.9 when not allowed
#

that means that magicdns works. That's cool, however that's not my target :p

ornate hill
#

lol. What's your target?

radiant cypress
#

I have a dns resolver on aws side, at the address 10.0.0.2

#

and I want to be able to resolve records from this server

#

(note that it works on my laptop)

ornate hill
#

yes, that works in your laptop because you're using the /dev/tun approach which is not proxied

radiant cypress
#

and looking at tailscale logs it takes the configuration

dns: Resolvercfg: {Routes:{.:[1.1.1.1 1.0.0.1 2606:4700:4700::1111 2606:4700:4700::1001 10.0.0.2] ts.net.:[199.247.155.53 2620:111:8007::53]} Hosts:6 LocalDomains:[tail9c382.ts.net.]+65arpa}
ornate hill
#

I think this will be more difficult in this case because DNS clients don't generally use proxy settings

#

so unless you have some custom code, your DNS queries won't be able to access your tailscale resolver

radiant cypress
#

ok, but really big thanks for your help there. I will still look for a way if that's possible

#

anyway, note that I had to also set the http_proxy env var, I don't know why the socks5 doesn' t work

#

oh I got this error

socks5: client connection failed: incompatible SOCKS version
ornate hill
#

so you can setup a split dns config in your tailscale account and have tailscale use a different resolver for specific dns suffixies

radiant cypress
#

This is what I already do and yes this seems to be the same problem

image.png
ornate hill
radiant cypress
ornate hill
#

👍

ornate hill
radiant cypress
#

I just tried to use this patch on top of the main branch and it didn't work. I mean I'm not able to reach my dns resolver neither. I'm going to give a try to loft's fork

#

That's not better

radiant cypress
#

Hey @ornate hill , hope you're doing great! Just to let you know that I achieved a first version for bootstrapping my platform. https://github.com/Smana/cloud-native-ref/tree/main/dagger
I still have the Tailscale issue which prevents me for running it on github actions but I'll try to find another solution 🙂

GitHub
cloud-native-ref/dagger at main · Smana/cloud-native-ref

Opiniated Cloud Native Platform Reference. Contribute to Smana/cloud-native-ref development by creating an account on GitHub.

#

Btw given the fact that I'll be presenting on the next community call, I'll send you a few slides asap if you have a few minutes to review them. Thanks!

ornate hill
radiant cypress
#

Yes that's it but that will be an opportunity for me to share about my use case and learn from the community too 🙂