Implementing MagicPod has allowed us to allocate more development resources, enabling us to produce more content.

Weathernews Inc.

Mobile app testing Browser testing

In the 19th edition of our user interviews, we had the opportunity to speak with representatives from Weathernews Inc., alongside Mr. Ito, CEO of MagicPod. We discussed various topics, including specific use cases and the decisive factors for selecting MagicPod.

Weathernews Inc.

Since its establishment in 1986, Weathernews has been providing meteorological technology and services that support people's lives. In its corporate business, it offers content that assists risk management across various industries, including sea, air, and land transportation, as well as in the distribution and energy sectors. For individual users, it provides participatory weather information through the smartphone app "Weathernews" and its 24-hour live broadcast "Weathernews LiVE".

KEY POINTS

Prior to implementation, everyone was involved in regression testing.
The decisive factor was the strong capabilities in mobile app testing.
UI changes can be corrected with just one click.
Automated testing allows for development up until the last minute.

From left to right:
Mr. Tomoaki Koga, App Development Manager
Mr. Takayasu Morishita, iOS Engineer
Mr. Nozomi Ito, MagicPod CEO

Mr. Koga (hereafter referred to as Koga): As one of the world's largest private weather information companies, we operate in BtoB for corporate clients, and BtoS for individual users. While BtoC typically refers to individual users as "customers", at Weathernews we refer to our customers as "supporters", and call it BtoS. For example, by receiving information about the local weather from our supporters, we can provide more accurate weather information.

Since joining Weathernews in 2009, I have been involved in BtoS app development, primarily handling Android development. Since the end of 2023, I have shifted the development focus to iOS, while also managing both iOS and Android teams.

Mr. Morishita (hereafter referred to as Morishita): Since joining Weathernews in 2022, I have been working on iOS app development. Specifically, I handle the development of the "Weathernews" app, its user analysis, and automated E2E testing.

We have about 30 engineers working on BtoS system development, including four members each specializing in iOS and Android app development.

Journey to Implementing MagicPod

Mr. Ito (CEO of MagicPod): Was the QA function handled by the app development team?

Koga: Yes. Our release cycle is about once a month, but the "Weathernews" app has a vast amount of content that in the past, all our engineers would spend two to three days conducting regression tests.

Indeed, it was too burdensome for the engineers, and desiring a "system that would allow us to focus more on app development", we initially introduced Appium for Android apps and began working on automated testing.

However, at that time, there wasn't a high level of understanding about testing within the company, and I started learning it on my own initiative. I managed it in a way that I would handle maintenance whenever something broke down. This was in 2021, and the following year, Morishita joined the company and extended the implementation to the iOS apps.

Morishita: I had experience with web-based tools like Selenium and Cypress, so I was able to apply that knowledge moving forward.

Ito: You had implemented Appium for both Android and iOS, and I believe you transitioned to MagicPod about a year later. What challenges did you face?

Morishita: Appium is written in JavaScript, which differs from the app's primary language. This required a catch-up effort, and as a result, the number of people who could write and maintain the test code was limited.

Also, the transition from Appium 1.x to 2.x took time to research due to the lack of documentation. It felt wasteful spending so much time to set up the testing environment, when we simply wanted to test the apps.

Ito: MagicPod also uses Appium in the background, so I understand the difficulties well.

Koga: We also faced challenges not being able to integrate into CI/CD environments like Bitrise or GitHub Actions, and being unable to conduct regular regression tests on a daily or weekly basis.

Decisive Factor in Choosing MagicPod

Morishita: Actually, while doing the automated testing of our iOS app, I learned about MagicPod and suggested, “Let’s implement MagicPod”. However, the timing wasn’t right…

Koga: It was our peak period, known as the flood season, spans from June to October, a time when heavy rains, typhoons, and floods are frequent. We avoid app maintenance during this period.

Ito: I see, so there's such a peak season as well!

Morishita: When things calmed down, I proposed again to my boss at the time, saying, "There’s a tool that can make us more efficient", and that's when we decided to implement it.

We considered other options besides MagicPod, but we chose MagicPod because "support is readily available in Japanese", "the pricing is not dependent on the number of tests run", and "it has strong capabilities in mobile app testing."

Moreover, I trusted it because Ms. Tamagawa of MagicPod had translated "The Way of the Web Tester", a book I read during my student days, and I also own two other books written by the MagicPod team.

Ito: Thank you! How’s your actual experience using MagicPod?

Morishita: The GUI is intuitive to use and significantly lowers the learning curve, which is great.

Koga: Last year, our new employees visited each department as part of their on-site training, and when the app team had them try automated testing with MagicPod, they managed to operate it to a certain extent.

Morishita: With code-based tools, catching up on the code is usually necessary, but being able to skip that and easily create test cases is really impressive to see. In fact, other teams have been able to run it successfully with little to no instruction from me.

Ito: It's great to hear such ideal stories (laughs).

Utilization of MagicPod

Ito: How do you typically use MagicPod?

Morishita: We use it to ensure that no regressions occur in our daily development. Our builds are scheduled in Bitrise around 5 AM, finishing in about 30 minutes, and are then transferred to MagicPod for testing.

The entire process takes 2 to 3 hours, so just before our workday begins, we receive notifications on Slack. Our day starts by reviewing these notifications. So far, there haven't been any critical issues. However, sometimes when the number of API accesses changes due to weather conditions, the display gets disrupted, so we reach out to the API team.

For the iOS team, I'm mainly in charge, but if I'm not available, another member may take over.

Koga: Similarly for the Android team, as Morishita's counterpart, I handle test case creation and maintenance for Android.

Ito: How is the progress with automation?

Morishita: We had a list of about 450 manual test items, and we've automated about 85% for iOS and 75% for Android. The remaining parts are those that require integration and communication with other devices. Since we originally had Appium code, we managed to complete 50% in about a month.

However, while code-based tools allowed for the flexibility to set up our own shortcuts, and MagicPod, being GUI-based, we had to change our approach. Which took some time to adjust.

Ito: Are there features of MagicPod you find particularly convenient?

Morishita: The self-healing feature. It manages adjustments needed due to minor version updates or changes to the internal UI structure of the OS, which often disrupt the display.

When I was using Appium, I had to fix at the level of updating selectors, but with MagicPod it automatically suggests, "This selector has changed to this", allowing me to fix it with just one click. This feature has been the most valuable to me.

Additionally, the prompt response to inquiries is extremely helpful. When I encounter issues due to UI changes, I often reach out, and support would respond within a few hours, sometimes in just one hour. It's incredibly fast, and I’ve also heard other teams praise how "the response to inquiries is fast".

In Conclusion

Koga: Testing is crucial yet can easily become a burden for developers. MagicPod simplifies this process, allowing developers to focus more on implementation.

In fact, I’ve heard feedback such as “Because automated tests are running, we can confidently develop up until the last minute” and “We can release earlier with confidence". Implementing MagicPod has allowed us to allocate more development resources, enabling us to produce more content.

Morishita: MagicPod being GUI-based is intuitive to use, making it easy to run automated tests. The barrier to entry is very low, where in fact, during our recent new hire onboarding, even non-engineers without testing experience can operate it easily. It's a tool I recommend for both experienced and new to automated testing!