Introduction
CORS (Cross-Origin Resource Sharing) has always been a classic problem in front-end development. Simply put, due to some security considerations of the browser, you will encounter some restrictions when loading resources from other domains. The solution is simple, just add some response headers such as Access-Control-Allow-Origin on the server side. With this header, the browser will recognize that you have been verified and there will be no problem.
I have written an article about this problem before: Understanding Ajax and Cross-Origin Requests, which details the problems encountered and their solutions.
I thought that since I had delved into this problem last time, CORS would never be a problem for me again, and I would never see the error of “forbidden to access cross-origin” in the console.
But I was wrong.
This time, I stumbled in a specific use case, but I also learned a lot from it. This experience also reminded me of what I wrote before: The most difficult cookie problem I have ever encountered.
Great, there is something to share with you again!
The Tragic Beginning
The thing is, a while ago, the company’s product redesign entered the final stage. The serious bugs were almost fixed, and the next step was to start adjusting some performance and testing the most important new feature of this redesign: PWA!
For those who don’t know what PWA is, let me explain briefly here. PWA stands for Progressive Web App, which is simply to make your Mobile Web more like an App through some browser support. The most important thing is that you can use Service Worker to cache any request (even API requests). If done well, you can even open this webpage offline.
In addition, through the browser, you can add your website to the main screen, just like installing it in your phone, becoming no different from an App.
Below are three screenshots that will give you a better sense of PWA. First, you can add this webpage to the main screen:
The second one is that this PWA will be like other Native Apps, existing in your phone. You can’t tell whether it is a Native App or a PWA just by looking at this page.
The last one is that after you open this PWA, it will become full screen. Just by looking at this screenshot, it is no different from a Native App.
In short, you can think of PWA as: existing website + new technology (Service Worker, manifest.json…), combined to become PWA.
That’s all for the simple introduction to PWA. If you want to learn more, you can refer to what @arvinh wrote: Will Progressive Web App be the future trend? or When React web app meets Progressive web app.
For PWA, the most important thing is actually this Service Worker (hereinafter referred to as SW). Chrome’s built-in Lighthouse can give a PWA score for the webpage. SW is one of the considerations, because you must implement SW to cache files and implement the offline opening App function.
The following figure shows the items that Lighthouse will check:
Alright, the preface is over, let’s get to the point.
We’ve done everything we need to do for our PWA, including registering the service worker and implementing offline functionality. However, there’s one thing that keeps failing in Lighthouse’s tests: registering the service worker.
No matter how many times we test it, Lighthouse keeps saying that our website doesn’t have a registered service worker.
It’s really strange. I tried testing it manually in a clean Chrome window in incognito mode, and I confirmed that the service worker is definitely registered. But no matter how I test it in Lighthouse, it keeps saying that it’s not registered.
So what should we do?
Fortunately, Lighthouse is open source and provides a CLI version that you can run on your own computer.
So I thought, since Lighthouse says it’s not registered, let’s take a look at how Lighthouse is testing it. I did a little research on the source code of Lighthouse and found that the testing method seemed fine. So I decided to modify Lighthouse to prevent it from closing the window after running the tests, so that I could see if there was any useful information in the console and if the message that should be printed when registration is successful was printed.
I made a few changes:
- Added a configuration file to only run the service worker test.
- After running the test, Chrome won’t be closed.
- Added a log in the service worker check.
If you need it, the parts I changed are here: PR for the changes
After making the changes, I ran the tests again. And at that moment, I remembered the fear of being trapped by CORS:
Clearing the clouds and seeing the sun
Since we have some clues, we should investigate them thoroughly. From the screenshot, it looks like the service worker is registered successfully, but there are some errors when using the cached files with the service worker, which seems to affect the entire test. Anyway, as long as we solve this CORS problem, everything should be fine.
Let me give you some background information first. We store all our static files on Amazon S3 and use Cloudfront in front of it. We have followed Amazon’s instructions to add what we need to add, so if the request header has an origin, the response will definitely have the CORS header. So there should be no problem.
And when the service worker caches files, it uses fetch, so it will definitely add the origin header, and there is no reason for it to fail.
After being stuck for an hour or two, I decided to take a look at the network tab and found more clues:
The following is a request sent from the service worker. The header does have an origin, but the response does not have Access-Control-Allow-Origin!
In addition, I found an identical request earlier. Since this request was sent by <script>, it did not include the origin, so the response did not have the CORS header.
It’s worth noting that the second response is from disk cache (although both are in the screenshot, that’s because I didn’t clear the cache when taking the screenshot, in fact, only the second one should be).
After investigating these clues, I have a rough idea of what’s going on.
In-depth investigation
Alright, let me explain.
The file that the service worker needs to cache is one of the JavaScript files that the page will load. Since the page will load it, we put a <script> tag in the HTML to load this file. From the screenshot, it looks like the browser loaded this JavaScript file first, and because it wasn’t sent via AJAX, it didn’t include the origin. According to S3’s rules, there was naturally no Access-Control-Allow-Origin.
Next, after successfully registering SW, we started executing the code inside it to cache the list we prepared in advance, one of which is this JavaScript file. However, when we used fetch to retrieve this file, the browser directly used the cached previous response (because the URL and method are the same), and this response did not have Access-Control-Allow-Origin! Therefore, the cross-domain error we saw at the beginning occurred.
The truth is revealed here, all due to browser caching issues.
Why couldn’t I find this problem when I tested it myself before? As a front-end engineer, it is reasonable to check “Disable cache” in devtool, so no matter how I tried, I couldn’t find this problem.
After knowing the cause of the problem, it is relatively simple. I searched on Google and found this Chromium ticket: CORS Preflight Cache Does not Consider Origin
The problem encountered inside is basically the same as what I encountered. The solution given in the end is to add a Vary: Origin to the response, so that the browser knows not to use the cache if the Origin is different. However, I found that we had already added it but didn’t know why it didn’t work.
In addition, I found several similar problems:
- Chrome S3 Cloudfront: No ‘Access-Control-Allow-Origin’ header on initial XHR request
- S3 CORS, always send Vary: Origin
Later, I adopted one of the solutions inside: “Since S3 needs an origin header to enable CORS, let’s send a fixed origin to it using Cloudfront! This way, every response will definitely have Access-Control-Allow-Origin!”
You can refer to this article: AWS CloudFront + S3 + Allow all CORS, which is basically just adjusting a setting.
This trick sounds quite effective, but it is not the best solution. It feels a bit dirty, after all, origin is not used in this way. It doesn’t seem too good to do this for the S3 mechanism.
So in the end, I thought of something that also solved a doubt in my mind.
That is to add crossorigin="anonymous" to <script>, so that the request sent by <script> also has an origin header!
I have seen some places add this before, but I still don’t understand why it needs to be added, because scripts can be unrestricted by domain. Why do you need to add a tag to make it look like an ajax request?
But unexpectedly, this attribute helped me. Once I added it, the script loading would attach Origin, and S3 would return Access-Control-Allow-Origin, so I wouldn’t encounter cross-domain issues later!
As for the other functions of this attribute, you can refer to: Purpose of the crossorigin attribute …?
Conclusion
To encounter the problem I encountered, you must meet the following four conditions at the same time:
- You put the static files on S3
- You did not check the browser’s Disable cache
- You used script and SW to load the same file
- The browser uses the cached script response to respond to the SW request
If any of the conditions are not met, this problem will not occur. In other words, it is quite difficult to encounter this problem.
But the more pits you step on, the stronger you become. Solving one problem means you have one less problem to encounter in the future. After solving this CORS-related problem, I think I won’t encounter related problems in the future… I hope.
Comments