The technical misery of ecommerce-shop front-ends

Thanks to my previous employments I got some insights into the technology of online shops and in particular the condition of their front-ends. I got those insights by integrating graphical elements into them and extracting product data out of them using an external JavaScript. Additional I had to clone single product pages to my server with small modifications for demonstration purposes.

Working this close with that shops front-ends got me a good insight into their technical quality and the way they were done. In fact, I have gotten in close contact with the front-end code of several hundred shops. I think that my impressions might be worth sharing.

This articles is for people that either want to feel the pain with me or are interested in some hints what they do better within their shops front-end.

At this point I’m sorry that I neither can give exact numbers nor provide many names.

Findings

There are actual a few good shops out there, but there are much more with severe issues. Most (not all, unfortunately) of them look okay when you just look at them in the browser.

The misery starts when you open developer tools and are greeted with many JavaScript errors and warnings. Although some creative developers care for that problem by overwriting console.log or adding a global error catcher. Problem hidden, problem solved, I guess.

Semantics and SEO

(better: Google, leave me alone!)

DSC02435
Still a lot of water needs to pass until SEO is done right

Online shops want visitors to sell anything. Usually you would expect that they invest much work into SEO optimization to make more people find their shop. In reality many shops fail to even cover the very basics:

  • schema.org is an industry standard to provide semantic metadata about web pages (i.e. for search engine robots). You see itemprop attributes in the HTML code when it is used. Only a minority of shops use it correctly, most use it incomplete, wrong or ignore it completely.

    From my impression awareness for that feature is even decreasing. I have seen many shops that relaunched to support mobile screens and forgot the semantic annotations they had in their previous implementations.

    A few shows use Open Graph instead, but that is a standard for social networks. Meaning those shops support Facebook posts (for the people sharing their articles, what in my impression no one does) but they ignore search engines.

  • I even had found a big online shop from Germany having a robots NOINDEX meta-tag on their product pages for years what makes search engines ignore those pages. Actually I did not know the shop before because I have never seen it in my search results. They fixed this recently  but still – oh boy.

  • Semantic URLs are URLs containing product names instead of product numbers. E.g

    semantic URL: /mobiltelefone/samsung-galaxy-s8-64-gb-P24397

    non-semantic URL: /product/320346555/

    The quality of semantic URLs varies and many shops still do not use them. Even big German ones.

  • In rare cases shops even put the filter settings from their product list into the path. That means the URL of one product page is different depending on the filter settings that have been used to find it. Good by search engines.

Performance

(better: Only what takes long is good)

DSC04607

Shops are increasingly using responsive websites (probably thanks to Google rankings). Four years ago only good ones had a mobile variant, today most of them have.

While it is good that they adapt to the change of user behaviour being, they still often miss to provide a usable performance. A nice looking responsive page is useless when people accessing it on mobile conditions wait more than ten seconds to see it.

There are tons of good resources out there explaining why being fast is so important and how much users you loose with every 100ms of increased loading times. Shops are also affected by this effect. A shop needs damn good prices in case they fail to deliver a good user experience.

While many shops seem to miss basic understand of how to achieve acceptable loading times, some have heard about its importance and try to improve it without any understanding. I’ve seen shop developers believing that performance is all about cutting bytes down. While size is important for performance, what matters more is network latency and the time when a browser actually starts looking for a file. There are developers actually making their page slower by using custom scripted loading mechanisms and breaking browser’s prefetching optimizations. There are so much half-knowing assumptions about loading performance out there that I often want to bang my had against something hard.

Here is a list of common fails:

  • Shops often use a big amount of script and style files. I have often found more than hundreds of them being loaded. While loading many files can be okay when you use HTTP/2, the summed size of those files is often a performance breakermanyfiles

  • In many cases shops actively slow down loading by using some kind of a script loader, be it a custom solution or the GoogleTagManager (GTM). Deferring the load of scripts can be a way to speed up initial rendering, but it is a bad idea to use it for scripts that build up the UI. I’ve seen custom script loaders to be used for every script, what actually makes page loading slower by working around browser pre-fetching mechanisms. Also the usage of GTM for script injection is in many cases no good idea because it is blocked by some ad-blockers. Nonetheless most shops use it heavily.

  • Additionally most shops developers seem not have a good knowledge of asset loading in general. I.e. they use CSS imports (horrible for loading performance) and do not put the images they intend to load in image tags but use JavaScript to build up a gallery. The async and defer attributes of the tag seem no to be common knowledge yet. We have 2017 and people seem to ignore any technology not supported by IE8, although their page is completely broken in that browser nonetheless.

  • Shops do often miss the easy optimization to minify their scripts and styles (properly) to get rid of comments, shorten variables and remove whitespace. That is a technique that is in available for more than a decade and still not everyone got it. In fact, it is getting better now compared to three years ago, probably thanks to much better tools.

    unminified
    That file is even static, you could even minify it without tool chain.
  • Another easy optimization is the use of HTTP/2. I call it easy because you do not need to change any code to use it but basically have to update your webserver – or use one of the better CDNs. Still, a small majority of shops ignore it.

  • Another performance nightmare is the implementation of A/B tests using external services. I have seen a big German shop first loading the page conventionally with HTML and styles, afterwards loading a script which contained the HTML and the styles for EVERY test case running. From all that code only one variant got applied on the page. Performance was horrible thanks to the additional loading and all the unused code.

    Other solutions seem to be more optimized and load only the code being required, but still doing A/B tests often means to first load the page and afterwards restructure it with JavaScript. This creates a big delay before users can see the final page.

  • Most shops use jQuery and although I would not build a UI with it any more, that is okay in some cases. But I really ask myself why people still use the bigger jQuery 1 instead of smaller newer versions. jQuery2 is jQuery1 one minus IE8 support, what makes it smaller. Many shops miss this damn simple optimization although they do not support older browsers anyway. Many developers seem not care for their jQuery version in any way, as they use an outdated version of jQuery like 1.7 and do not even update it when they relaunch. I cannot remember having seen jQuery 2 or newer in any shop although it would be the correct choice for most of them.

  • A special bad technology often found in bigger shops is Scene7 from Adobe. It is used for showing product images from an image database within some kind of a gallery on product pages.

    scene7
    Watch your scene7 gallery to appear 5 seconds after the rest

    I myself used it five years ago and judging from the shops today it has not become much better. Apart from many issues it is a performance nightmare because it is a script which loads scripts (which load scripts again, I believe), which load the images from adobe server.

    Whenever the product image on a product page appears seconds after the page has finished loading, probably Scene7 is used. I consider the product image as an important part of product presentation and shops should tune it accordingly. But no, they use Scene7. At least, it is used more rare today.

IMG_9703
Got headaches? Take some breath and look on clouds

More pain

  • Wrong or no use of cache-control header. Using cache-control you can make browsers load a file from disk cache immediately without checking for a newer version on the server before. This usually heavily speeds up loading of a file. Many shops do not make use of this feature. On the other hand many other shops use it but in a harmful way. To use this feature correctly you need to suffix the requested files with a version number. In case you update your shop, you update your version number and browsers will fetch this file from the server. The mistake many shops do is to not use suffixes but put this header on standard files name like screen.css or app.js. Whenever you do updates on such shops and have changes in those files, browser will still used the old cached versions what probably will break something. I have seen one shop which told the browsers to cache every file for a whole year. By doing things like that you are on the best way to break the page for some customers while you do never notice it because developers are used to do hard reloads.
  • Not very surprising, but I have seen broken HTML structure in some cases. Not sure what browsers and robots do with it, but it is not good for sure.
  • No understanding of grid systems. You still often see pages making use of absolute pixel positioning. I have even seen shops doing a responsive version with a broken grid and crazy pixel positioning.
  • Broken styling which means developers having strong opinions about the look of their standard elements. Such opinions are great for breaking any third-party functionality or UI library. Worst one I have ever seen:

    * { margin: 20px; padding: 20px; }

Quality by Size

The size of a shop in terms of sales seem not to have much influence on its technical quality. There are bad shops in all sizes.

They are especially bad when shops with a lack of IT competence use a custom shop system. Many big shops suffer from this.

On the other hand I saw many shops from Europe which use one of the available shop systems in a effective way. Those are shops are unobtrusive. Not particularly good, but no fail either. Best front-ends are made by big shops using a custom shop system.

The best shop systems I’ve seen were custom made ones by bigger companies.

Quality by region

I noticed an undeniable difference in technical quality of shops between different regions of the world. This is a pure observation and I do not speculate about the reasons. Of course my sample of shops is incomplete and my ranking is not well defined, but there is a difference for sure.

  • India: good
    I only worked with a few online shops from India meaning my overview is limited. But all the shops I worked with were very good. They even use React based progressive web apps many month before they started to appear in Europe. The shops from India were technically the best ones I’ve seen at all.
  • Western Europe and Australia: PoorWestern Europe provides a mix of different qualities and most of them are not veray good. There are only a few progress web apps in service here. I have seen one at cromwell.co.uk and a new one at microspot.ch/interdiscount.ch.
  • USA: BadMany big shops that work somehow, but make me me of the web of the years 200* instead of 2017. They stick with the known technology even after relaunches and seems not be interested in further development. That means they are using dinosaurs like Demandware, Oracle Commerence and Scene7 at a high density. One exception I have seen is walmart because they use something modern like React. Unfortunately even they can compete with the Indians due to the lack of server-side rendering.
  • Eastern Europe: Very bad. Not much to say about them. There is not much deviation in quality here.
  • Middle and South America: Even worse
    This is the region with the shops I might tell my children about to frighten them. Most of the issues I wrote above can be found here. Some special things I’ve only seen here: Loading 3000px * 2000px big images to show them as thumbnails in 100px * 100px.  Another shop created the script loading cascade in use as the main image of this post.

meh
Meh!

Conclusions

I did in no way do security checks on the shops. The adoption of HTTPS is still not complete, but most shops have it, probably thanks to Google. But HTTPS is only protecting the connection to the shop.

It is just hard to believe that shops missing many basics are in any way able to protect themselves from hackers or even simple remote exploits.

There are obviously not enough capable developers out there to run that many shops.

In case you have read until here and you ask yourself how you should do an online shop without me ranting about it, I want to provide some hints.

Your general options are:

  • Build one yourself.

    That is not the worst option, but it requires a reasonable amount of resources. The worst shops out there have been coded by someone years ago and are simply kept running without any technical enhancements. On the other hand there are some big shops out there running their own software and they do the job in acceptable way. If you do not want something generic, you do not have much of an option but have to go this way.

  • Build atop an existing shop system

    Most commonly there is a shop software installed on a custom server. This approach might work, but unfortunately shop owners want customization and that is where the trouble starts. You often have to do heavy changes and hacks in the shops codebase what might even work at first. But: the most important thing when you run a shop software, is keeping it updated for security reasons. Those updates often do not happen. This might be out of ignorance but can also be because the customizations changed the shop in a way that it is not compatible with the update any more. Here you end in some kind of a deadlock.

  • Use hosted shop software

    Big shops often find this approach smart to avoid operations, but technically it is not much better than the previous ones. The trouble are the customizations again. As service provider is hard to build front-end code which the your customers can modify and still be compatible with your updates. It is close to impossible. An attempt to solve that is to leave all generic shop code intact and just add in additional code and styles. That is no good approach from a performance perspective and does not save you from problems with future updates done by the provider.

    Those shop systems cannot develop with the technical progress because they have to stay stable to keep users code running. In short, you cannot update a jQuery-based front-end to a react-based one without killing the customer code that worked atop the jQuery front-end. And because customer is king hosters will not do any heavy updates any more.

So, my advice in case you want to start an online shop is:

Either do it yourself by some great developers or use an existing shop software but do not change any more than its colours.

Unrealistic, I know.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s