Flipkart chose to partner with Moolya in the year 2012:
Via one of the Flipkart members having attended one of Pradeep’s sessions at a Conference
For testing a couple of their applications - An Offline Reading Application and Flyte
Included testing for their Web, Android, IOS, Mobile Web and Windows platforms
The Main focus when Moolyans started testing were functional and UI tests
For UX, our team partnered with the Flipkart design team :
To collect information from real users, based on their reviews on Play Store and App Store
During the design phases for wireframes and mock-ups, the ideas shared by the test team were considered and incorporated
While planning for New Releases and the release scope, Moolyans were involved in the competitor application analysis for features undergoing changes or new releases. Based on this exercise the test team many valuable suggestions. A classic example for this are the first few Flash Sales launched by Flipkart in the past
The test team keeps a constant check on the Analytics Data, when ever there is a slow down WRT users interaction with certain features, then :
Analysis is done by the team and the product owners to perform an RCA to find out on what went wrong
Example: The number of Flipkart’s users making use of the Exchange feature dipped after a Major Release rollout. This was because of users being unable to relate with the usage of a certain checkbox on the Exchange screen before proceeding. The team identified this and was eventually fixed, which led to an improved conversion rate for the feature.
User Review Analysis
The practice of Regular Monitoring of App reviews are in place
Understand the real user struggles by scheduling time with them and connecting with them directly
Pro-actively take up and prioritize user issues along with the product owner
Flipkart Vernacular Testing
Before kicking off with the Vernacular testing for Flipkart, the test team spent time on testing one of the Competitor applications (Amazon), which had already adopted multi-language support. This analysis report was shared with the Product team. A lot of Cues were takeaways as a result of this analysis for introducing this feature on Flipkart
Main focus of Vernacular testing was for the application to have the same UI/UX as offered when the language default is set to English
Following are some of the snapshots for the translation quality scores and the competitor analysis.
Below is the complete competitor analysis report for reference.
Loading…
Release success metrics
0
How we prevented bugs ?
The Moolya team had themselves involved from Phase 0 of the SDLC itself - Identifying and Gathering Requirements and Designing
The Team:
Understood the system from the scratch
Could comprehend multiple aspects of the application better than Flipkart’s own testing folks
Were able to figure out a lot of root causes and potential risks upfront
Example 1: During a Flash Sale, there is a limit with the number of sellers selling a particular product. This was identified early in the release cycle and conveyed to the product team, which led to a redesign of the flash sale screens.
Example 2: When users add items to their cart during a flash sale, they are expected to purchase these items before the reservation time is expired. Few of the users were found adding items to their cart but not purchasing within the set reservation time. This resulted in these items showing up on the respective user carts during the next flash sale. Bringing up this observation to the fore, led the Flipkart team to build a prevention mechanism to avoid the repercussions.
Note: The finer details of the bugs we prevented for Flipkart were not shared due to security reasons
Tech value
The Moolya team has been a major contributor to build a proper test process across multiple teams
The Bug reports produced by the team were highly regarded due to the many inclusions. Helped the dev team to understand and fix the bugs faster
Charles proxy reports
Device and Console logs
Complete bug analysis and details
Detailed troubleshooting
The team completely owned the deployments to various environments via Jenkins. They kept constant checks on the PR’s (with constant dev team interaction) across all the project Git repositories and maintain a correlation with the release scope and releases
Automation was not in-scope initially, but Moolya testers upskilled themselves to bring in UI and API Automation using slightly customized Open-Source Frameworks
Helping/training devs to mock API response through charles proxy in their local machine to reproduce a particular scenario
Conducted charles proxy sessions for devs and product managers
Identified and logged a whole lot of bugs early in the product lifecycle which helped to strengthen the quality of the product
Bug bashes for new features were conducted across the team for better device coverage and for for bringing in varied perspectives
Product value
Being involved in the requirement design discussion helped in preventing issues and to give our ideas or solutions to problems
Regular discussions with product owners and analysis of the analytics on a regular basis
Researched on UX related issues when drops were noticed through analytics and brought in A/B experience (User experience research methodology) when needed
Suggestions on feature enhancement and UX improvements
Making the address fields dynamic, to incorporate other local languages
Layout & Vernacular translate change in GST invoicing feature
Test Value
The Moolya team:
Has been a part of all the major releases
Credited for contributions to the launch of Flipkart’s new version of website and mobile applications. This was a mammoth effort involving the entire engineering team putting in relentless efforts for a month
Brought in good practices for exploratory testing
Aligned the way of working with test teams from Flipkart and the other vendors
Mentored and paved the way for new members on the test team
The team lead from Moolya was entrusted with the responsibility of handling the whole of the functional and platform test teams at one point of time
Brought in a visibility of the testing activities to stakeholders
Ensured to involve the right stakeholders while devising the test coverage and scope , fine-tuning based on the suggestions/feedbacks