Last week, Microsoft showed off the results from testing different web browsers' effects on battery life. Web browsers can be among some of the most resource intensive applications PC users frequently rely on, and Microsoft wanted to show that their development of Edge in the Windows 10 Anniversary update will drain your battery the least compared to Chrome, Firefox, and Opera.
Microsoft ran their experiment on four similar Surface Books that looped common activities people perform in a web browser. In Microsoft's tests, Edge lasted 70% longer than Chrome, 43% longer than Firefox, and 17% longer than Opera (with Battery Saver enabled).
The Program Manager for Microsoft Edge talked about how his team accomplished this. They relied on learning from customer telemetry to optimize the browser, made more efficient background tasks, and improved the efficiency of Windows networking to name a few things.
However, the team over at Opera was displeased with the results. And so, they ran their own test with a clearly laid out methodology of testing specific websites and looping certain actions. In their head on head comparison between Edge and Opera, Opera won.
But there were also disputes over this single test by Opera. They might not have been using the current and most stable Redstone build of Edge. And Microsoft had actual real-world use telemetry to construct their test, which was backed up by a parallel test by the Wall Street Journal.
PC World's latest battery life test
PC World decided to do their own test and announced the results earlier today. Gordon Mah Ung of PC World actually began testing battery life differences a few months back.
One of his concerns with testing browsers is that the web is constantly changing. If you run a test at one time, a few hours later you could get completely different results because websites are always changing their content. For example, one time you visit there could be a ton of ads and embedded videos which use up a lot of processing power. Then a few hours later there might not be as many. This leads to inconsistent testing.
To circumvent this testing inconsistency, Gordon Mah Ung used EMBC's BrowsingBench which controls for all of that variability. He does note there is a big downside to BrowsingBench--it only tests simulated single-tab browsing, which just isn't how most people browse the web today. But at least BrowsingBench does offer a more apples-to-apples comparison.
PC World's Results
So PCWorld tested Edge 13.1, Chrome 50, Opera 37, Firefox 46, and Internet Explorer 11 using EMBC BrowsingBenchmark mostly on a Toshiba Radius 12 laptop running Windows 10 Home. What he found was Edge does indeed take first place out of all five browsers. But the lead is not as dramatic as Microsoft's own test might lead you to believe.
Chrome actually came in second place, followed by Opera, then Firefox, and lastly IE 11. The PCWorld test concludes that Edge "has a clear power advantage in light browsing chores; it’s just not as dramatic as Microsoft’s own tests." But Gordon Mah Ung distinguishes a very important caveat that should be kept in mind when reading about all of these tests.
But the truth is actually more complicated because our browsing habits are so different, and can change from day to day.... Do you sit with 10 tabs of Flash- and video-heavy webpages open all day? Or do you sit in Google Docs for eight hours?... All three of those use cases will likely have very different effects on battery life and going by anyone’s generic 'browser battery-life' figures doesn't make much sense.
These tests can establish important benchmarks. But you have to know what they are testing and how a test's criteria are relevant to how you use your device in order to evaluate a test's significance.