Empowering the Development Team to Focus On Development. How MagicPod Streamlined Outsourcing Tests with Minimal Training and Resources.
GeoTechnologies, Inc.
In this 16th installment of our user interviews, we share a discussion between MagicPod CEO Nozomi Ito and GeoTechnologies, Inc. They talked about specific use cases, the deciding factors in choosing MagicPod, and more.
GeoTechnologies, Inc.
Founded in 1994 as a multimedia software development company, GeoTechnologies released the world's first car navigation system. As a major maps company, GeoTechnologies leads the Japanese map industry by providing map data and location information solutions for corporate clients, as well as high-precision 3D data maps that are indispensable for advanced automated driving. The "Trima" application, which allows users to earn points just by moving around, has been downloaded over 16 million times (at the time of the interview), and many people enjoy the application for daily travel and various content.
KEY POINTS
- Before MagicPod, the development team handled testing.
- The main criterion for selecting a testing tool was accessibility to those without development experience.
- GeoTechnologies was able to reduce revisional work, and achieve time-savings of approximately 45 hours a year.
- This outcome allowed the development team to dedicate themselves entirely to development work.
Takuya Sato; TCE Quality Assurance, Process Assurance
Implementing MagicPod
Mr. Sato (below, “Sato”): I joined the company as a new graduate in 2016 and have been involved in quality assurance and testing related to combining map data with the car navigation app since being assigned to the QA department. Currently, I am involved in the testing of the points activity app "Trima." We have a large department called TCE (Total Customer Experience), and I belong to the Process Assurance (PCA) department within it, which is responsible for quality assurance of services and apps.
Our company's core business is the production and sale of map data, and "Trima" is a service that started as an internal venture. Initially, it progressed as a project team where development and planning were integrated, but since the establishment of PCA in March 2022, PCA has been responsible for quality assurance for all our services. Currently, three employees are assigned to PCA, handling test design and process management. Test execution is entrusted to a test vendor.
Ito (MagicPod CEO): Before the establishment of PCA, did the development team for the "Trima" app also perform testing?
Sato: Yes, that's correct. Since testing was done on the development side, we started thinking about whether we could allocate those resources to other development tasks. Also, as the service grew larger, the QA department began to handle quality assurance for the entire company's projects, but it was judged to be difficult in terms of resources to carry out the actual testing. Therefore, we considered introducing automated testing and outsourcing the testing, allowing QA members to focus on test design.
Decision Factors for Choosing MagicPod
Ito: How was the selection process for an automation tool carried out?
Sato: Firstly, since nobody in QA had experience in mobile app development, we considered tools such as Selenium or Appium to have a high barrier to entry, given the need for coding. We chose low-code tools as candidates. After comparing several options, MagicPod was the only one that supported Android at that time. The fact that we had members in the company with experience using MagicPod, and their positive evaluations, was also a deciding factor.
Ito: Was it also your first time using a test automation tool?
Sato: Yes, I hadn't been involved in mobile app testing, so it was a completely new experience for me. I’d seen a colleague struggling with selecting a tool that required coding for mobile app test design before, so when I used MagicPod, I felt that its no-code platform was very convenient. The UI is also visually straightforward, so some people might be able to start designing test cases on their first day.
Current Operation of MagicPod
Ito: How is MagicPod currently being operated?
Sato: I handle test design, and test execution and maintenance are entrusted to test vendors. Even though they were new to automation initially, they are now creating tests for new features as well.
One advantage of MagicPod is the ease of training and handover when there are changes or additions to the team. The fact that the execution environment is in the cloud is also beneficial. Just creating an account completes the environment setup, which is convenient. We are considering introducing MagicPod for the overseas version of the "Trima" app, and it seems like we can start smoothly.
Ito: It's good to hear that you are using it more often! How is it used on a daily basis?
Sato: Once the build is done, we incorporate MagicPod into the build flow on the development side using scripts. On the QA side, we run it regularly every night. This is not so much for quality assurance but more for running it daily to understand the updates to MagicPod itself and identify trends such as where an issue is likely to occur. It's for maintenance purposes.
The intention behind such usage is to avoid unnecessary rework within the limited testing effort. It might not be exactly how MagicPod recommends using it. How do other companies use it?
Ito: Many companies run it frequently with the purpose of identifying unstable tests. I think your approach is somewhat similar. It reduces rework during releases, leading to fewer setbacks. So, it seems like you're making good use of it.
Sato: That's reassuring. We were able to reduce manual testing and rely solely on checking test results, and MagicPod can run in the late hours unlike humans. Initially, the lack of manpower was the reason for introducing it, so it contributes to cost reduction.
Due to the nature of the app, there are few cases where automation is possible. Considering the time it takes to check results, it may not lead to significant savings, but it has reduced about 3 hours per release. Since our goal is to have releases around 15 times a year, it adds up to approximately 45 hours of reduced effort per year.
Use Cases of MagicPod
Ito: Do you have opportunities to discuss MagicPod with the development team?
Sato: They got hands-on experience during the introduction, but now that the development team no longer handles testing, they are able to focus on development. There was a member who said, "If it's okay with the automated test, then it's fine," when there were small fixes after a rebuild due to a bug. So, the fact that MagicPod is working allows them to concentrate on development.
Ito: That's a positive story. In practical terms, are there any features that you find useful?
Sato: To be specific, features like the auto-save function for the editing screen and the history restoration function contribute to a sense of security in our operation. Also, the existence of execution captures makes it easy to check how long it was running normally in case of incidents.
However, since we are currently outsourcing operations, I am in more of a managerial position than an operator. From a management perspective, it’s beneficial that MagicPod allows testing not to be influenced by the skill level of the tester.
The development side also had concerns about entrusting testing to external people who might not have a high understanding of the service. However, MagicPod has features that detect image differences, matching text, etc. So, I think there are many aspects that we can “see” with MagicPod compared to the human eye.
Ito: It's been about a year since the introduction. Are there things you would like to tackle in the future?
Sato: The number of users for "Trima" is increasing, so I want to create a more stable operation mechanism. For example, the backend on the server side is well-monitored by development, but we directly look at the app for the front-end behavior. We are considering using MagicPod to automate this as well. Also, concerning unit tests, we are thinking about incorporating MagicPod into CI.
Ito: Many companies integrate it with tools like Bitrise, CircleCI, GitHub Actions, so ultimately aiming for integration with CI would be good.
Final Thoughts
Sato: When people hear "AI test automation," many may imagine that it can automate everything, and I also thought that before getting involved in test automation. However, in reality, there are many things it cannot do, and it becomes challenging depending on the compatibility with the test target. It requires the creativity and ingenuity of the designer. I don't recommend approaching it with the misunderstanding that you can automate everything, but there are certainly things that you can reap the benefits of if done correctly. It's a technology whose capabilities become apparent only when you give it a try. I think it's best to start by trying it out and considering what it can do.
Ito: That's right. Recently, Magic Pods user community has become active, and there are many cases where questions and answers are resolved between users. Of course, we will also support you if you contact us. As you mentioned, starting by trying it out is the way to go.
Thank you for today’s interview!
