Category: Load Testing

The Big Billion Day Sale Fiasco – Flipkart’s Software Testing Issue in 2014

Posted on by Yogesh Kumar Gupta

On October 6, 2014, Flipkart, one of India’s leading e-commerce giants, conducted its first Big Billion Day sale. The goal was ambitious: achieving ₹600 crore (approximately $100 million) in gross sales within a single day, a staggering 20 times their regular daily sales of ₹30 crore. This event was designed to be a major milestone, both in terms of sales and market penetration, planned meticulously over several months. 

Pre-Event Preparations 

Flipkart’s preparation for the event was extensive. The finance and sales teams worked to place large orders with vendors, expand warehouses, and hire thousands of temporary delivery workers. The marketing team launched a massive advertising blitz across newspapers, billboards, and TV channels. The technical team, led by senior engineers, worked overtime, reserving additional server space and testing the robustness of their systems to handle the anticipated surge in traffic.

The image is a projection summary for a sales event by Crestech Software. It states: Target Sales: ₹600 crore ($100 million), Regular Daily Sales: ₹30 crore, Duration: One day, Key Resources: Expanded warehouses, temporary delivery workers, and additional server space. The Crestech Software logo is displayed in the corner and there is an icon resembling a shopping app.

 


The Day of the Sale

The sale commenced at 6 a.m. on October 6, 2014, initially for Flipkart employees, and was opened to the public at 8 a.m. Within minutes, the website and mobile app were overwhelmed by the deluge of customers. Despite the extensive preparations, Flipkart’s systems could not handle the traffic, and the site experienced significant downtimes. The initial two hours saw half of the sales target achieved, but the technical issues persisted. 

System Failures

By 8:10 a.m., Flipkart’s website and mobile app had crashed. The company received a volume of traffic in an hour that the Indian Railway Catering and Tourism Corporation (IRCTC) typically received over several days. Despite optimistic initial sales figures, the situation soon turned grim as technical issues continued. The payment system broke, and the warehousing software crashed, causing further delays and customer frustration. 

Customer Backlash

Customers faced multiple issues, including: 

  • Inability to access the website 
  • Products being marked as ‘out of stock’ within minutes 
  • Prices being marked up before the sale to make discounts appear larger 
  • Orders being abruptly canceled 

Less than 10% of the visitors could make a purchase, leading to widespread dissatisfaction and anger among customers. 

Post-Event Apology and Internal Reflection

By the end of the day, Flipkart had achieved its sales target in just ten hours but at a significant cost to its reputation. The management acknowledged the failure, with a public apology stating, “We did not live up to the promises we made and for that we are really and truly sorry.” The internal review highlighted a crucial lesson: when technology becomes subservient to business ambitions without adequate testing and preparation, the consequences can be severe. 

Importance of Software Testing in E-Commerce

The Flipkart fiasco highlighted the critical importance of software testing for e-commerce giants. Companies like Blinkit and Zepto, which promise ultra-fast deliveries, and ride-sharing apps that handle millions of transactions daily, must invest heavily in rigorous testing to ensure their systems can handle peak loads. These companies employ strategies such as: 

  • Load Testing

    Simulating high traffic conditions to test system performance. 

  • Stress Testing

    Pushing systems beyond their operational capacity to identify breaking points. 

  • Scalability Testing

    Ensuring systems can scale up seamlessly during peak times. 

  • Real-Time Monitoring

    Continuously monitoring system performance to address issues proactively. 

The Ticketmaster Example: A Persistent Issue

Despite the lessons from the Flipkart incident, similar issues persist. In 2022, Ticketmaster faced a massive backlash during Taylor Swift’s Eras Tour ticket sales. The system failures were strikingly similar: long wait times, site crashes, and an inability to handle traffic surges. This resulted in significant revenue loss and legal actions, yet underscores a troubling trend: even major players continue to compromise on adequate testing and preparation. 

Conclusion

The Big Billion Day sale of 2014 serves as a cautionary tale for e-commerce and tech companies. The incident emphasizes the importance of investing in robust software testing and infrastructure to handle high-traffic events. As the Ticketmaster example shows, the failure to do so not only leads to immediate financial losses but also long-term damage to brand reputation. For companies aiming to thrive in today’s digital marketplace, ensuring their technology can meet business demands is not just a priority—it is a necessity 

Who should test your application? A developer or a tester?

Posted on by Yogesh Kumar Gupta

Who should test the application? The deciding battle whether to hire a tester or a developer is never-ending. The objective is to verify and validate the application while finding the defects before its release and ensuring its quality.  

While developers aim at creating and developing the application to its best, testers aim at ensuring the application design is of good quality.  

One of the most important factors differentiating a developer and a tester is that the developer stops testing when the application works once while the tester starts testing when the application works. Also, it depends on the mindset that reflects their attitude toward the development of the application.  

Hence, if you are struggling with who can test your application better, then you need to understand how they work. We’re sure at the end of this write-up, you will know the answer.  

How does a tester test? 

A tester… 

1) Tries out both the beaten path and the “odd ways” of testing an application 

  • Testing may sound like a common process. A tester is a person who is responsible for trying out all the necessary usage scenarios for the best working of an application.  
  • Testers follow both the regular testing process and their unconventional ways to ensure that the application works as expected.  
  • A tester is more focused on addressing defects and resolving them before the application can be deployed to users.  
  • Hence, a tester follows both the beaten path and odd ways to try out several different approaches to do the same thing. The agenda is to determine whether a specific combination of steps may lead to application failure or unexpected results.  

2) Tests the same thing over and over again until gets 100% of the expected results 

  • Tester idealizes the process of continuous testing. A tester starts the testing the moment it becomes available.  
  • This type of application testing can also rely on test automation that is integrated with the deployment process.  
  • Though automated testing enables the application to be validated in realistic test environments. However, an ideal tester urges to test the application over and over again.  
  • A result-oriented tester is focused on improving the application design and reducing risks.    

3) Doesn’t limit to the usual process of what needs to be tested and how it needs to be tested 

  • A tester is involved in assessing many stages. Ideally, organizations maintain test assets to track what an application builds to test.  
  • However, a tester is not limited to what needs to be tested. A tester gains access to assets such as requirements, codes, models, test scripts, design documents, and test results.  
  • A tester is completely aware of the parameters of what needs to be tested and how it needs to be tested.  
  • An ideal tester focuses on user authentication and audit trails to help companies meet compliance requirements with minimal administrative effort.  

4) Don’t assume that it will work every time and everywhere 

  • A tester is determined to offer perfection when it comes to testing results. Hence, a tester analyzes the success of testing based on reports and analytics. It eventually helps other team members to share status, goals, and results.  
  • A tester never assumes that the same process will work every time and everywhere. Thus, a tester incorporates advanced tools to integrate project metrics and present results in a dashboard.  
  • This particular practice makes the tester super confident and lets the teams quickly see the overall health of the project. 
  • A tester tests to establish the parameter that defines the development of the application while monitoring relationships between development, test, and other significant elements.  

5) Never satisfied even if it works in the most ways, needs the application to work in every way 

  • A tester knows that testing can be time-consuming. Still, the tester is never satisfied with regular procedures even if it works in most ways.  
  • Several automated software testing tools are used to complete the testing process. However, a tester feels incomplete with automation and runs manual testing or ad-hoc testing to be 100% sure.  
  • The tester is more focused on making sure that the application works in every way irrespective of any circumstances.  
  • A tester is not ready to accept that automated testing helps implement different scenarios and test differentiators. The tester never feels satisfied until the application works in every bit possible way.  

How does a developer test? 

A developer… 

1) Follows the obvious way like how an application is meant to be used 

  • A developer is quite practical in the process and works on the application the way it is meant to be used. Once the features of the application have been nailed down, the developer is supposed to convert them into an actual application. 
  • The developer uses a variety of tools that include programming language, integrated development environments, data structures, staging serves, and more to get the application started.   
  • Once the primary development of the application is completed, a developer tests the application in a regular, necessary way to make sure the application runs the way it should.  

2) Tests once and gets satisfied if the feature works fine  

  • Mostly, a developer follows a definite testing process where the developer writes down and executes basic test cases.  
  • Ideally, this process helps to determine whether the application is structurally sound and performing properly or not. Once the results are in favor, the developer makes the end call and finalizes it.  
  • The developer would only test once if the results are accurate in one try. The developer feels satisfied if the features of the application work fine and even work in usually used ways.  

3) Doesn’t explore and is limited to what needs to be tested  

  • Ideally, a developer is focused on unit testing that is not similar to the way a tester does. The process is followed by developers to determine any necessary bugs while ensuring the application works as expected.  
  • If everything seems good, a developer won’t make extra effort to explore more possibilities of any glitches while using the application.  
  • Developers are aware of bugs that can’t be identified by them. Their mind-sets are focused to follow the usual testing process and report the functioning of the application.  

Conclusion:  

During the testing process, both the tester and developer work in their best possible ways to give 100% favorable results. However, certain parameters can’t be examined by the developer. And this is where the tester’s role comes in.  

What businesses need to know about LOAD TESTING and why they should go for it

Posted on by Yogesh Kumar Gupta

With the upsurge in the adoption of technology in various business environments and organizations increasing reliance on software applications for core operational transactions, the need for fast, reliable and stable performing applications has become an absolute necessity.

The possible drawbacks on an organization’s business due to performance issues are revenue loss, poor end-user satisfaction, high maintenance cost and increased cost to scale up.

There is an increasing demand from customers for fast and reliable applications not only to meet their growing business needs, but also to gain a competitive edge in the market.

Software load testing is the discipline concerned with designing and executing non-functional tests to know the current performance of an application vis-à-vis the speed, scalability, stability, and other performance parameters.

Load testing is apt for software solution providers as a diagnostic tool to locate bottlenecks and performance issues within an application.

In Load testing we endeavor to question ourselves the following and address the same by evolving an effective testing methodology.

  • Whether the application meets the defined response time requirements of my customers?
  • Will my users experience acceptable response times even during peak hours?
  • What system capacity is required to handle expected load?
  • Where are the bottlenecks in my multi-user environment?
  • How can we detect defects that only occur under real-load conditions?
  • How can we optimize the multi-user application before it goes live?

What is Load Testing?

Load Testing is a testing process that determines how software applications affect the user experience when they are used simultaneously by many users. With application load testing, we will be evaluating the system behavior/degradation with various loads and/or configurations to ensure:

a) that the system meets the performance requirements as defined,
b) determine the capacity of existing systems and
c) create benchmarks for future systems.

Why is Load Testing important?

Load testing service is important because sometimes popular sites often suffer from outages when they draw unforeseen traffic on their pages. To avoid such outages, via the various tools we use, we ensure glitch free usage of an application.

What is the process involved in Load Testing?

To ensure that the load testing is conducted in a systematic, well documented with timelines, the following process is adopted:

  • Evaluate requirements and identify scenarios
  • Develop load test scripts by emulating end user actions
  • Automate and Execute the test scripts with different volume of data and concurrent users
  • Monitor the tests with automation
  • Analyze results and verify across the SLAs
  • Craft load test report

What is performance benchmarking and baselining?

  • Benchmarking:
    A set standard that helps to determine the quality of the performance of the product. We establish high-quality metrics of an application to meet various business standards.
  • Baselining:
    To compare the performance of a new application with a known standard of system performance. We ensure that the application doesn’t degrade with any new updates over time.

Performance testing helps assess your application through the speed, effectiveness, scalability and reliability.

What are some important metrics captured during load testing?

There are curated metrics that are captured while performing load testing. These metrics are captured by analyzing products.

  • Response time
  • Transactions per second
  • Hits per second
  • Throughput
  • CPU utilization
  • Memory usage
  • Network load

What are the popular testing tools used in performance testing?

  • Jmeter:
    An open-source tool to test load and measure performance by simulating heavy loads. Mainly used for testing web applications.
  • LoadRunner:
    An automated load testing tool which can create virtual users and scale up the capacity of the virtual users, identify processes having trouble and diagnose a problem to find out the root cause.
  • Neoload:
    It is a proprietary automating tool used for performance testing on web and mobile applications.
  • Silk Performer:
    Load and Performance testing tool for Java and .Net applications.
  • Rational Performance Tester:
    An automated testing tool for performance testing from IBM.

When is the right time to get load testing done?

When the application is relatively stable and before rolling it out to production would be the right time to get load testing done for your application. Also, load testing should be done at an early stage to assure performance consistency across user loads, to ensure speed and scalability with current infrastructure and for continuous stable performance.