#My Dark Journey Into Configuring Azure CDN...

1 messages · Page 1 of 1 (latest)

kind flax
#

Over the course of last weekend, that extended into Monday followed by Tuesday, I struggled to get Azure CDN to deploy a series of 10 different custom DNS domains pointing to a single WordPress server (11 if you count preserving the old, pre-legacy website). This is the story of how I got it to work.

#

Okay, so my saga begins here: #hangout message where I have successfully gotten a single Web App running WordPress-as-a-Service up on Azure, with a Multi-site network running 10 different vanity domain projects that all have different content, branding, etc.

#

For reasons that are hard to explain or justify, what we ended up with was a subsite-network using subfolders BUT each subsite has a unique DNS name and a path of /. The original intent was to have one DNS name and different paths, however this caused problems from some plugins and after struggling with it most of Thursday and Friday, it was decided to simply do the thing that works.

#

So, this left us in the unfortunate position that we had now 10 domains on the origin side that need to route independently to the front end of the CDN.

#

We also have the blob storage optimizations to take into account as well, which are configured as the default origin group of the main CDN endpoint with a rule to switch to the Web apps origin group whenever the URL path does not match the storage container prefix "<container path>/wp-content/uploads" specifically.

#

At first, I tried to do everything in a single CDN endpoint with rule set to change the origin group to different DNS according to the request path that came in. This didn't really work out, so I quickly gave up and moved to doing a single endpoint for each unique DNS name. (For the record, I think it could've worked with some effort, but it was not very elegant anyway.)

#

So, now for the first caveat in Azure CDN that you have to watch out for when doing multiple sites on a single server: The CDN Endpoint Type of "Web App" is just a thinly vailed convenience. It doesn't appear to do anything special. Worse yet, if you have configured your Web App with 2+ custom DNS names like I had done, then the name it will so for the Origin Host will be randomly selected from that list. This is obviously not desirable if you intend to have each endpoint specifically pass through to its designated DNS name, which is how WordPress knows what content to serve.

#

Note that it would not have been any issue if we could have simply used subfolder paths, since there'd be just one DNS name at the origin - but this was not working for us.

#

So, here's the thing. If you configure a Web App endpoint, and then you explicitly change the Host / Origin values, what you will get is a Custom Endpoint. Once I realized this, the scaled fell from my eyes and I knew that Web App endpoint is not something magic or special. It's just a fancy way or automatically setting the Host value.

#

So tip #1 is that you can simply make 10 Custom Endpoints pointing to 10 Custom Domains (or more, if you have aliases, more later). The base endpoint created by the WordPress QuickStart can be reconfigured so it is a Custom not Web App and you change the host to match whatever you have specified for WP_HOME / WP_SITEURL on your WordPress server, or the root site.

#

On to the next problem. As we configured Custom DNS for each endpoint, it quickly became clear that we would be able to easily add those like "www.mydomain.com" but had problems with root level domains such as "domain.com". This was a real issue for us, since we have search engines and print magazines published using the root level domain for the company.

#

Microsoft says that Azure CDN no longer supports root level custom domains, but this is not entirely accurate. However, they are tricky devils.

#

A root level domain can't have a CNAME. If your DNS host supports it, you can create an ALIAS (aka ANAME) record, but Azure CDN will not recognize it is correctly configured. What you have to do to bypass this is create a CNAME record in the format of "cdnverify.mytld.com" pointing to "cdnverify.<myazureedgename>.azureedge.net". Once you add this record, the Custom DNS screen will allow you to get past verification so you can create the root domain alongside the www, etc.

#

A caveat: you can't create any Azure CDN managed SSL certs for a TLD either. You'll have to go buy one like the RapidSSL for $25/year. Pretty annoying if you already paid for a wildcard cert, but hey. Basically managed certificates are giving us a dozen SSL certs for free, and they were set up so we'd have to use like a dozen SANs to cover the spread. So we're good!

#

So we have 10 CDN endpoints, with 10 sets of custom domains and almost 10 sets of managed SSL certificates. Each endpoint has an origin that points the Host and Origin to the custom DNS name supported by its WordPress subsite. (You did remember to add those custom DNS on the back-end in the WebApp too and test them to ensure they are working ok, right?)

#

Tip #2: Understand how to overcome the limits of TLD in Azure CDN Custom DNS and SSL/TLS certs by using alias record and "cdnverify" trick.

#

On to the next problem.

#

So, I'll take a moment to walk three sides around the barn. When we started this configuration, I thoughtfully disabled HTTP on all 10 of my Endpoint Origins. Why? Because I have HTTP to HTTPS redirect on the WordPress application and also the Web App itself is set to Always HTTPS. It's quite difficult to get WP to only serve HTTP anyway.

#

Later, for testing, I did find a magic 3 line code in wp-config.php that will spit out a simple message if HTTP is being used:
if (!(isset( $_SERVER['HTTPS'] ) && 'on' === strtolower( $_SERVER['HTTPS'] ))) {
// Commented to test if redirection on the back end messes up the CDN
die('Page Intentionally Left Blank!');
//throw new Exception('Page Intentionally Left Blank!');
}

#

But that's not important right now!

#

So, in thinking the Origin can't support HTTP anyway, why not turn it off at the front end of the CDN?

#

Simple answer:
The "Our services aren't available right now" error message.

#

Which you will get on any front end HTTP request if the service has no HTTP at the origin.

#

Why did they configure it this way? I don't know. It seems stupid.
What's worse is for a while it seemed like my HTTP to HTTPS redirect rule (more in a few moments) was working reliably in spite of this, and so there was no issue with it.

#

The above error message is terrible. Microsoft did not even try to offer a message like "This website does not support HTTP" or the like.

#

Premium versions of Azure CDN will do this redirection automagically. But we are cheap bastards, so we're going to hack around the problem.

#

Long Story Made Short, My First Attempt:
CDN Endpoint > Rules Engine > Add Rule...
Condition: Request Protocol equals HTTP
Action: Redirect URL 301, Protocol HTTPS.
Worked sometimes, not all the time. Spooky behavior and a half a day or head scratching.
Later Attempt That Works Reliably:
Condition: Request URL begins with "http://"
Action Redirect URL 301, Protocol HTTPS (and if you wand to prove it is working, add Domain as "www.somedomain.com" so you can see it doing its thing in F12).

#

Why does request protocol sometimes not work yet url does?

#

I do have a theory, but it has to do with a different rule.

#

As y'all know you can't redirect a page if there are changes to the headers.

#

Saying this now, no, on second though it doesn't make sense.

#

The way CDN is doing rules to change request headers, they are done between the proxy and the origin. So, let's say you add a rule to add "Access-Control-Allow-Origin" = *...

#

You will never see this in F12 on the browser, because it happens in the backend when the call is made to the origin server.

#

A practical example of this is when you need to do cross-origin calls to the azure edge (CDN endpoint) for pulling assets from blob storage. In this case, the Url will match the root name of the endpoint (edgename.azureedge.net not www.mycustomdomain.com) and the content will be blocked unless CORS allows the reference. Our easy trick is to have a rule that if the Host is edgename.azureedge.net and the Origin is www.mycustomdomain.com then we open the origin list to * since we know it is on the whitelist. (Adding each domain explicitly is complex and seems unnecessary.)

#

So, what happens when Rule Engine is configured for HTTP to HTTPS redirect and not working correctly?

#

A: spooky stuff that is hard to troubleshoot.

#
  1. You may get the "Our services are not available" message, which isn't much to go on - if you have HTTP disabled, or maybe some other times too.
  2. You may get a 302 redirect from your HTTP to the HTTPS endpoint on the back-end of the WebApp. This happens when WordPress receives the HTTP request from the CDN and decides to send the user to HTTPS at the WP_HOME location. (See my magic wp-config.php hack about for how to prove it was WordPress.)
  3. You may see some but not all of your redirects working randomly or not. So, for example, HTTPS redirect was working on the first of 4 custom domains on the endpoint, but being ignored for the other 3. This is seemingly without rhyme or reason, as 2 domains were TLD and 2 were www. All DNS was configured identically to point to the Azure Edge via CNAME/ANAME.
#

Side note: resolving the ALIAS/ANAME produces a different IP address than the CNAME does. So, wtf? Why is that anyway??

#

Tip #3: Test your rules are working as expected and not some rando stuff that is being done by the origin server or Azure cloud voodoo. Even if you have to hack the origin to force it to say "Hello world, I am HTTP/HTTPS!" message you will unravel a lot of mystery BS and settle on rules that work properly much faster if you know what the origin is doing.

#

Tip #3a: Do not trust Azure CDN Rules Engine to work correctly for any given combination of conditions or actions. There are lots of examples of Microsoft coming back and saying something to the effect of "just because you can configure it that way doesn't mean we implemented it." HTTP is a weird protocol with its own quirks, and a given condition you are trying to do may not fire because in some circumstances the header you test is not sent by the browser or some other BS reasons. Where possible, stick to the ones you know work, like the URL.

#

So let's see now... I have wandered off the forest path a bit so let me review and see if we have a working configuration now or is there something else we needed to cover.

#
  1. Multiple Endpoints for multiple Custom Origin with unique DNS.
  2. Top level DNS in Azure CDN and certs.
  3. Rules to redirect to HTTPS and allow cross origin as needed for blob store.
  4. Make sure HTTP is turned on for Origin - even if you aren't going to use it.
#

That last one bothers me a lot, but it seems like the way things stand now, you can't count on the CDN processing any HTTP request even for redirection if you turn it off at the Origin. wtfms

#

In the end, this entire nightmare took me 4 days of basically working around the clock to implement. (To be fair, about halfway through there was a little detour into deploying WP themes while I thought all this was working correctly.) So, probably approx. 30-40 hours worth of effort to find the magic configuration to deploy the entire front end for our WordPress site network of about a dozen sites.