Performance issuses

We recently had a performance issue specifically on chrome that essentially made the Gotland map unusable. In this post I'll go over what happened.

So What happened?

After we added forests (and support for hills and mountains) to the game we also re-wrote the renderer from a single threaded solution to a worker based approach. This offloads a lot of the rendering and keeps the UI and game feel responsive, even on lower end devices.

Prior to deploying we conducted a lot of manual tests on all major browsers as well as on android and iOS. Everything looked good on our development servers.

But when we did eventually deploy, especially on chrome, the map slowed to a halt.

At first we debugged the new render setup thoroughly but after many hours decided that the problem must be elsewhere.

Long story short

During debugging the game ran smoothly on our development servers. This lead us to believe that the issue was some server specific configuration that differed from localhost.

The issue was also complicated by the fact that even on our production server, firefox and safari ran fine. Maybe not great, but fine. This narrowed it down to a server specific, chrome specific issue.

Without going into to much detail, it turned out that an old server configuration still forced the game to use a very old SVG estate render that was drawn underneath the canvas based estate markers, and it also rendered a drop shadow effect underneath the estate marker.

The drop shadow specifically forced a complete re-build of the DOM every time the camera panned or zoomed, every frame.

Drop shadows and SVG are handleded differently on chrome than on other browsers.

All in all, a hard bug to find but one that made the game unplayable for a couple of days.

Hard lesson learned.