Crashing Apps, Buffering Streams, and Fuming Fans: FIFA WC 2022 Streaming Saga

FIFA World Cup 2022 was a massive spectacle for football fans worldwide, with 26 million viewers of the final alone in the United States. It’s the most digital and technologically advanced world cup ever hosted. It ended on 18th December 2022 with a fairytale final as the ‘messiah of football – Leo Messi,’ lifting the crown for Argentina.

In terms of marketing and revenue, FIFA World Cup 2022 generated around $6.5 billion through advertising campaigns. Google CEO, Sundar Pichai also mentioned that FIFA World Cup 2022 generated the highest search traffic in the last 25 years.

Elon Musk, the Twitter CEO, also tweeted about the global event, with the traffic hitting almost 20,000 tweets per second.

With such a scale of events both in viewership and marketing, streaming applications faced multiple issues. Unfortunately, many of them crashed as well.

So, we’ve put together a guide to what you should know about the tech blunders of streaming apps.

As the audiences were enjoying the football on display and having a gala time, many streaming apps that got the right to broadcast the world cup had massive nightmares. They crashed, faced streaming issues, and were reprimanded by the fans for their poor performance and average user experience.

Reliance Jio won the right to showcase the marquee event for the first time in India.

It decided to showcase the live streaming free of cost to all users on their flagship app – Jio Cinema. However, the start was like a dreaded dream!

Jio Cinema’s service crashed and lagged during the opening game between Qatar and Ecuador.

Fans got upset due to the poor quality of streaming and various lags during the crucial juncture of the match.

In reply to these tweet attacks by the end-users, the Jio Cinema admin had to issue an apology.

After this incident, Jio Cinema worked on its user experience, and within a couple of days, they were giving a satisfactory streaming service.

But Jio Cinema forgot, “first impression is the last impression.”

As a result, users couldn’t forget the day-one affair, and Jio Cinema’s brand value got a severe hit.

Potential cause of tech blunder

From the engineering perspective, Jio Cinema failed in scalability and performance!

What they could have done is—consider the number of users that may stream their service. They should have a mechanism where they can scale up or down as per the demand for the service. Moreover, they could have opted for observability to track issues that could hamper their user experience continuously.

Like Jio Cinema, ITVX, a free-to-watch service launched by ITV faced many streaming issues during Saudi Arabia’s vs. Argentina match. The streaming service crashed during the national anthem of Saudi Arabia, which irritated the fans. They felt ITV had disgraced the occasion by providing a poor user experience.

Nevertheless, they shared their thoughts on Twitter…

And reported problems sarcastically:

Even as ITV tried to recover from these angry reactions, the lightning struck for the second time. The experience was a nightmare for English fans watching the quarter-final against old-rivals France. First, the service crashed, and then, there was an apparent blackout. In addition, the English fans complained about the poor user experience and numerous error messages.

Some fans also said that the stream was several minutes behind the live-action in Qatar. Then, adding injury to insult, England lost their match 1-2 against France and got knocked out of the FIFA World Cup 2022. Consequently, ITV faced a barrage of angry tweets and the blame for England’s loss.

No one would have thought that a tech blunder could hamper brand identity to such an extent! But it did for ITV and other streaming apps, including the official FIFA app.

Where did ITV fail?

ITV failed in sustainable performance and resiliency.

When handling such a massive global event, some failures are bound to happen. But, ITVX could have made their app fault-tolerant by implementing chaos engineering principles. Also, they could have opted for observability tools for sustainable performance as it helps to find the root cause of issues hampering user experience.

Various apps crashed while streaming the FIFA World Cup 2022. So, how can the official FIFA app lag? They also had their fair share of issues in the ticketing system during England vs. Iran game. Due to the problem in the ticketing system, the fans had to wait at least a couple of hours outside the Khalifa International Stadium.

A similar issue also occurred during the US and Wales match—tickets “disappeared” from the FIFA ticketing app, leaving United States fans unable to attend the team’s opening match against Wales. In addition, the app reportedly crashed, preventing those in attendance from accessing their account or transferring tickets to others.

FIFA had to issue an apology statement on both these incidents for the poor experience fans had outside the stadium.

How could FIFA have avoided this blunder?

FIFA could have implemented complex engineering principles of reliability and resiliency. Reliability means performing the function as desired. They could have conducted regression testing before finally releasing the ticketing system in front of fans. Also, in terms of resiliency and fault tolerance, the FIFA ticketing system should have prepared a backup plan for transferring the user to a safe state in case of failure. They could also have included CDN to cache the last stable version of the page and show that to users.

TSN Sports is one of Canada’s leading sports entertainment companies, which acquired the right to stream the FIFA World Cup 2022. With so much experience hosting major sports events, Canadian fans expected that TSN Sports would do a fine job. However, it didn’t work as desired.

Many fans experienced streaming problems and frequent app crashes. For some, the playback freezes every few seconds or minutes, while others experienced audio issues during live streaming, which worked well during commercials.

Here’s how the fans showed their displeasure on Twitter:

Moreover, the frustration increased for fans as there was no apology or notification from TSN Sports acknowledging the issue. However, it did set a terrible example of professionalism.

What could they have done better?

They could have emphasized rigorous testing practices from an engineering perspective. From time to time, the app crashed or failed to synchronize audio and visuals. This tells that various testing methodologies have yet to be implemented before the final release. Also, from the user experience perspective, they could have worked on performance engineering principles to ensure a smooth selling ride.

As the FIFA World Cup 2022 started, millions of fans chose the Peacock app streaming service to watch their favorite players and the match. However, their excitement turned into disappointment when the app crashed/froze while streaming live content.

The viewers had paid a hefty amount to have a next-level and engaging streaming experience. However, the outcome was a stark contrast. Many people experienced an issue where the audio and video got out of sync every time an advertisement ended.

Some also faced app compatibility issues with Amazon Fire TV Stick. Moreover, there were issues related to low video quality and connectivity errors.

Unlike TSN Sports, Peacock responded to fans’ problems with streaming services and crashes by taking a case-by-case approach.

Peacock did take a spontaneous step to help mitigate user challenges. Instead, they should have taken a proactive approach.

How they could have ensured performance, reliability, and resilience?

To ensure peak performance, Peacock could have opted for observability tools that find the root cause of issues hampering user experience.

Reliability is performing the functions as desired, and to make that happen, Peacock TV could have tested its app on various platforms, avoiding compatibility issues. As for software resiliency, it could have undergone resilience testing to test system behavior under realistic disruptive conditions.

SBS On Demand provides free, unlimited streaming of TV shows, films, and major sporting events in Australia. Like many apps discussed above, SBS On Demand was a streaming partner for FIFA World Cup 2022 in Australia.

However, football fans around Australia were left fuming after SBS On Demand’s stream of Argentina’s quarter-final clash against the Netherlands crashed late in the first half. Angry viewers showed their frustration on Twitter after getting an error message with steam crashing.

SBS On Demand responded to the fan’s request by improving the stream in the second half. However, it was a little too late for fans who didn’t want to miss any piece of the action.

How SBS could have dealt with the engineering mishap?

The engineering team at SBS On Demand could have focused on two complex engineering principles – resiliency and fault tolerance. The stream of SBS on demand was crashing way too often, showing that they haven’t put enough effort into making their service fault-tolerant. They could have worked on making their service resilient by adding backup servers. They also could have analyzed the number of expected users and, based on their installed backup servers in various regions of Australia, provided a seamless experience to users.

BBC iPlayer was the major broadcaster for the UK in the FIFA World Cup 2022. It had the experience of handling plenty of live events across various sports. So, fans in the UK expected it to be a silky smooth affair. However, it turned out to be a bumpy ride in the first English game.

England was playing Iran in their opening world cup clash, and minutes after the kick-off, the streaming service from BBC iPlayer crashed. Many fans reported connection issues and the BBC iPlayer failing to load, with an error message appearing for desktop and mobile users.

Here’s how the fan reacted to this event on Twitter:

The official BBC Sport handle on Twitter responded to fans’ angry tweets and eventually re-established the service on BBC iPlayer.

BBC should have focused on its engineering practices

Despite being a major player in the streaming industry, it failed in performance, scalability, and resiliency.

Considering the huge customer base, they could have tested the application on various devices and platforms they expect users to stream. And utilized observability tools to continuously track issues and resolve them before they become fatal errors hampering user experience. Moreover, they could have tested their network connections and had a backup plan in case a particular streaming link failed.

The only way out is through the performance engineering principles.

Focus on the functional and non-functional requirements. Identify the cause apps crashed during live events. And adopt the best practices covered below.

One of the most common issues streaming platforms face during live events is processing high-quality content. During the live event, video, sound, subtitles, and many other parts get transmitted separately and then synchronized on the device. If that synchronization gets delayed, the live stream will buffer and disrupt the whole user experience.

One of the possible solutions to avoid such a scenario is to opt for adaptive bitrate. Adopting this approach will help you adjust the video quality depending on the speed available for a particular user. It also ensures a smooth live-streaming experience without any interference. Lastly, you can also set up a backup for crucial components of your streaming software.

In a scenario like the FIFA World Cup, millions of viewers consume your content through various devices. So, ensuring compatibility with multiple devices is key to success. The streaming protocol can help you in this regard as it optimizes content delivery to avoid video crashes. So, choosing the suitable streaming protocol and encoding standard can ensure compatibility.

Install the latest version of the streaming application on your device’s OS to ensure you have relevant software. Also, ensure that the stream can have several encoding devices, from a PC to a laptop, a smartphone to a tablet to GoPro. Streaming should support landscape and portrait video modes to avoid compatibility issues.

Any live event will attract massive eyeballs, which challenges your security and data transfer speed. Therefore, you must ensure that closed commercial streamings like interviews, private appointments, sneak previews, demos, etc., are only accessible to the premium subscribers.

Built-in token authentication is one of the possible ways to deal with this situation. It ensures that users can only access the stream with a token, meaning the subscribers. A registration page in your streaming app can help you in this regard. In addition, it allows you to have a list of subscribers in your database and ensures channel safety.

Sometimes, what happens is that you’re the only primary streaming provider, but you’ve won a right for a particular region. In that case, you rely on the primary content provider. Now content providers will have servers for processing the audio and video. But, the servers have some limits, and streaming gets delayed if the received data exceeds the server’s processing capacity.

One of the possible solutions to this problem is owning a secure server. Having your own server allows you to scale and manage the quality and speed of your live streaming. However, server management is more challenging than it sounds. So, many businesses rely on cloud computing services for that purpose as they’re cheap, reliable, and well-scaled according to demands.

A stable internet connection is one of the keys to success when live streaming from your phone, laptop, tablet, or computer. During live streaming, your camera will feed video to your video processing device (local or cloud). Sometimes, your camera will communicate with the device where processing occurs before the video gets delivered either locally or on the cloud.

In both these situations, if the internet connection is weak, the whole pipeline of video streaming gets disrupted. So, always ensure that you have a solid internet connection with multiple networking options (Wi-Fi, Ethernet, or a 4G hotspot/dongle), configured, tested, and ready to use. Preparing yourself for the worst-case scenario in advance is vital.

When handling a global sporting event viewed by millions of people, some lags are bound to happen. Everything doesn’t go as planned.

The best streaming providers manage all lags and issues professionally by opting for a CDN (Content Delivery Network). It caches your videos on servers around the world. So, when a user requests a particular video segment, CDN serves it immediately without reaching web servers. Using CDN allows you to manage a significant traffic spike that can sometimes crash a web server. It also reduces delays in live streaming.

These are some of the ways through which you can save your streaming apps from getting crashed or causing lagging issues.

In the upcoming section, we’ll look at some of the streaming services that implemented these suggestions and ensured an incredible streaming experience.

Disney+Hotstar is the no.1 OTT platform in India and home to Indian cricket. It also had the right to telecast the ICC Cricket World Cup 2019 in England. In 2019, Hotstar had 350 million downloads and 100 million unique visitors engaging with the platform daily.

Source: Hotstar Tech

This is what the Hotstar concurrency pattern looked like during the India-New Zealand semi-final. With 25.3 million concurrent users, Hotstar managed 1 million requests per second, 10 Tbps bandwidth consumption, and 10 billion clickstream messages. It was a massive scale.

Hotstar achieved it by launching Project Hulk. It comprises load generation, performance tests, chaos engineering, and traffic pattern analysis using machine learning. It involved 108K CPUs, 216 TB RAM, 200 Gbps network out, and 8 regions geo-distributed. The tech team at Hotstar didn’t rely on a typical autoscaling approach that you use in AWS.

The engineering people here relied on keeping the AWS Lambda instances warm and avoiding the cold start. They used to pre-warm the Lambda and EC2 instances before the match time. Moreover, the engineering team relied on the ‘provision concurrency’ solution to scale up resources based on demand. Also, the Hotstar team used Akamai, a leading CDN provider, to cache the video on eight geo-distributed regions and availability zones.

So, the lesson here is to prepare for all types of failures. There will be a sudden increase or decrease in traffic during the live events. How you manage those spikes to ensure a good experience matters a lot. Lastly, understanding your user’s journey is extremely important. Hostar knew the points or events when users were likely to grow or shrink, so they were prepared to handle those events well.

Channel 7 is Australia’s #1 free-to-air commercial television network and home to the most loved news, sports, and entertainment programs. It had the right to stream and televise mega sporting events such as AFL Grand Final, the 2020 Tokyo Olympics, and the 2022 Winter Olympics.

To capture the imagination of millions of people, it had to ensure higher uptime and an A-grade streaming experience. For that purpose, knowing scalable infrastructure and components was paramount.

However, its homegrown tools for simple monitoring and troubleshooting were insufficient with the scale of mega sporting events. This led to the shift to observability.

Channel 7 can now capture rich and valuable data at the application level to understand the customer journey and identify user experience issues. It can determine the number of users the infrastructure can support at a particular time.

Opting for observability was a record-breaking win for Channel 7!

It ensured 100% uptime and a flawless user experience. As a result, Channel 7 could stream 4.7 billion minutes during Tokyo 2020 Olympics, 32.6 million minutes during AFL Grand Final, and 376 million minutes during the Winter Olympics 2022.

Developing a streaming application may sound easier, but providing a hassle-free experience for your apps as the user scales during a live broadcast is not a piece of cake. As we have seen during the FIFA World Cup 2022, various streaming apps crashed or faced streaming issues.

Even the official FIFA ticketing app faced problems, which tells you that you need to manage your functional and non-functional requirements.

To do so, you need a knowledgeable technology partner with expertise in digital product engineering; Simform could be your one-stop destination!

Contact us if you want to build products that scale seamlessly.