Creation of a test automation cycle that successfully involves Product Development teams. Among others, MagicPod improved productivity of entire team.
MEDLEY, Inc.
I went to Medley with Ito, the MagicPod CEO.
We heard a lot about how MagicPod went through the examination stage, how it is currently operated, and from the people working on test automation going forward.
Company overview of MEDLEY, Inc.
MEDLEY was founded in 2009 with the mission of "creating the future of medical healthcare" and aiming to achieve "convincing healthcare" through technology-enhanced businesses and projects. The platforms that we are developing consist of a human resources platform business that operates the "Job Medley" recruiting site and a medical platform business operating the "CLINICS Online Medical Care" online medical care system, as well as the "MEDLEY" medical information service aimed at patients.
KEY POINTS
- The strength of support where consultations are met with an instant response
- The reason why many company staff members are able to use MagicPod
- If you see a solution to a problem, do this first
First Development Group, Product Development Office, Medical Platform Headquarters, MEDLEY, Inc.
QA Engineer Yoshiaki Yoneyama
Stable provision of high-quality products
I joined the company in 2020. My responsibilities cover the entire QA process, including QA and regression test maintenance for new feature development. There is one other QA engineer, and that engineer joined the company a little earlier than I did. Until then, there were no dedicated QA members, and the development members related to the project were all manually running scenario tests, as well as regression tests.
It is not that there were any particular problems occurring with this structure, but since we are developing products in the sensitive field of medicine, it was decided that a QA team should be arranged to ensure a comprehensive QA system capable of providing customers with stable, high-quality products in the future.
Automated testing had been a big help in my previous job, so as soon as I joined the company, I started to think about how it could be used to improve product quality.
The strength of support where consultations are met with an instant response
When thinking about automation, we examined several tools for achieving this, including MagicPod. We had discussions in-house where we considered the fact that "We need to do a lot of things in the overall QA process, but can we start by automating it?" Making our own version was certainly an option.
However, when we examined the three platforms of iOS/Android/Web, we realized that it would involve a major workload to create this in-house, and we were also concerned about parts of it that might become dependent on individuals’ skills in the future.
If we were able to reduce the workload involved, we would still have to take this forward in parallel with other processes. From a cost perspective as well, when we examined the expense required for outsourcing the testing or hiring a dedicated automation engineer, we reached the conclusion that we would get far better cost vs performance by deploying an automated testing tool.
On top of all this, some acquaintances of mine had told me about MagicPod. They had been using the tool for some time. I also had the opportunity to meet Ito at an event. As there was a trial period available, we decided among ourselves to “give it a try.” I remember that when we tried out the software and had questions, we would receive an instant response, and this was extremely helpful. We felt the strength of their support and technical ability.
We liked the fact that the Web and app could be used with the same UI. It meant that the onboarding costs were low for staff other than the QA engineers. (By “onboarding costs,” I mean the costs for introducing the service so that people can understand it and become familiar with operating the software.) We were also able to find a solution to the specifications that were acting as a blocker for automated testing, including client certificates and two-step authentication on the Web.
Anyway, in our case, we approached automation from the perspective of mobile apps, and when we were examining the alternatives, we found that MagicPod was almost the only SaaS testing tool that was competitive for mobile apps as well.
Left: MagicPod CEO, Ito / Right: MEDLEY, Inc. Yoneyama-san
The reason why many company staff members are able to use MagicPod
When we first deployed it, we performed automation based on the scenarios that we tested manually every week. Currently, we are implementing the parts of the iOS and Android native apps, and Web browser regression tests that can be automated as E2E tests, and these are run either daily or when performing QA before release.
By running these on a daily basis, we are able to detect degradations at an early stage, and when the results of past runs can be checked by saying, “This is not the latest fix, but when did it start?”, you are able to check the results of past runs, and it is relatively easy to troubleshoot the problem. Of course, for the part that could be automated, it is possible to reduce the manhours that would be necessary when executing manually.
*Web test results
*Medicine notebook function test results (mobile)
Basically, when I encounter a failure, I try to isolate the problem on my side, and if it is a problem on the product side, I inform the development team members.
After we deployed the tool, there was a request from the development members to provide notification of the run results on Slack, and the test result notifications were provided to the team channel using Slack.
The UI tests are more liable to fail than the code-based tests, but as it is not good to get accustomed to failure, we performed maintenance every day to try and improve the rate of success just a little. Sometimes, when a test had failed, the development team members would check and investigate and correct the problem in advance, and I felt that this was the ideal situation.
We try not to individualize the implementation and maintenance as much as possible. This is because MagicPod requires almost no environment to be constructed and it can be maintained with basically no code. This means that even if there are no QA engineers on the team, the planners and development engineers can take charge. I think there are many members who can use MagicPod within the company. Of course, we need to do a certain amount of onboarding, but MagicPod was a good selection from the perspective that anyone could operate it.
Reference: “Stories about deploying MagicPod for UI Test Automation” (Medley Developer Blog)
If you see a solution to a problem, do this first
At our company, we often hear the argument that “before doing automation we should OO,” and this can lead to people neglecting problems that automation would solve. If you have some issues currently, and feel that automation might solve the problem, you should first try it out, check to what extent your tests could be automated, and make a decision after first confirming what the implementation/maintenance costs would be incurred.
Of course, it is difficult to automate all tests, and there are some tests that are better confirmed manually. We cannot leave everything to the judgment of AI, and we need to consider carefully what needs to be checked when we create test cases. If you are facing problems with the tests you are trying to realize, there are many reliable people in TRIDENT, starting with Ito, so you should discuss any issues with them.
When using the test tool to the maximum, not only quality, but attempts to increase team productivity will differ greatly depending on the company. We are very interested in case studies from other companies and we would welcome the opportunity to share them in the future.
Notifications
We are taking applications for QA engineers.
・Recruitment page
・Tech blog page
A word from the Editorial Department
We interviewed them in their cool white office, from which there is a view of Tokyo Tower.
Yoneyama, whether in his blog or in interviews, always thinks in a logical way and explains things clearly in an easy-to-understand manner.
As a QA engineer, and despite the fact that there are only two of us in the company, it was truly impressive to be able to have so many members in the company be able to use MagicPod after we had already set automation on track.