The real deciding factor in the selection was that “there is no limit to the number of tests you can run.” Daily testing leads to the early detection of issues.
K.S. Rogers, Inc.
For the ninth user interview, we spoke to the MagicPod CEO, Ito, and K.S. Rogers, Inc. We discussed a wide variety of matters, including actual use cases and the deciding factors in the selection.
Company Profile of K.S. Rogers, Inc.
Since its establishment in 2017, the company has adopted a “fully remote, fully flexible, and free employment format,” looking to promote new ways of working. This is a community-based company that primarily comprises freelance and side-business engineers who work remotely and are not bound by their environment. The company supports the promotion of various new businesses in Japan through its “Startup Studio business,” and the Chief Technology Officer provides hands-on and systems development support for websites, applications, etc., to startups.
KEY POINTS
- Our quality assurance was not keeping pace with our business expansion.
- The real deciding factor in the selection was that “there is no limit to the number of tests you can run.”
- MagicPod does the work for you, so you have far more time for other things.
- Daily testing leads to the early detection of issues.
K.S. Rogers, Inc.
Kazuhiro Yoshino (Freelancer / Test Engineer)
Mr. Yoshino and Koyuki (left), MagicPod CEO Ito (right)
Circumstances behind MagicPod implementation
Yoshino-san (hereafter “Yoshino”): I joined K.S. Rogers in the fall of 2021. Before that, I had been in charge of quality assurance at Panasonic for around 40 years. For around 10 years, I had been working on testing automation. Moreover, for network camera QA, we used Selenium for WebUI testing. Since retirement, taking care of my dog has become my day job, but I now work at the QA unit at K.S. Rogers doing quality assurance for contracted and in-house development approximately two days a week.
When I joined the company, the QA unit at K.S Rogers was not keeping pace with its business expansion, and bugs were often overlooked. Tamiwa, the CEO, was aware of the problem and was in the process of introducing automated testing, believing that this was a necessary mechanism for allaying quality concerns. When I joined the company, the tool selection process had just finished, and two tools had been shortlisted. It was at that point that I was asked to trial MagicPod.
Reasons for deciding on MagicPod implementation
Yoshino: I had written Selenium test codes at my previous job, so I was surprised to find that, with MagicPod, I could perform automated testing simply by dragging and dropping the elements displayed on the screen. It struck me that we were in a fantastic time in history. Another thing I was surprised at was the self-healing function of the locator. In the case of Selenium test codes, if the locator changed even slightly, it would stop working. With MagicPod, the AI would tell you that “this is probably what you want to do.” In addition, there is a function to check for differences in images. The thumbnails displayed on a site can be verified without using a code. The development environment is unstable and specifications change frequently with MagicPod, but its AI has been extremely helpful in following up with me on many occasions.
In the end, however, the deciding factor in deploying MagicPod was that “you could run unlimited tests.” To be honest, I had never imagined that I would be able to repeat a test so many times each day under different conditions. Being able to do this made me glad that I had chosen MagicPod. Moreover, I liked its extensibility, which allowed me to integrate it with other tools such as Slack, email, and BrowserStack. We discussed these things in the QA unit and decided to “go with MagicPod.”
MagicPod usage cases
Yoshino: Currently, the QA unit at K.S. Rogers uses MagicPod for internal development. Specifically, we check industry association websites and the accuracy of the information provided regarding various site user demographics. For example, we verify whether information with publication deadlines is being provided only during the publication period rather than after the publication period and whether the functions available on the administration screen are controlled according to each role, both for PCs and smartphones.
Because the attributes of our customers vary so widely, we use MagicPod’s scheduling feature to run tests under a variety of conditions at specific times on specific days of a week. However, with our current contracts, we will have a scheduling problem if we need to separately configure all the test cases before running them. In addition, we want to make use of the daytime environment to implement test steps and wish to run E2E tests on implementation. Therefore, we created a weekly test planner that allows as much of the work as possible to take place during the daytime.
Currently, MagicPod is running around 80% of our regression tests, which is extremely helpful. Testing is an important component of quality assurance work, but there are many other tasks under its purview. I feel that I can afford to spend time on other tasks only because MagicPod does some of the work for us.
Recently, I have been considering introducing a test management tool, and I would like to share comprehensive information about QA activities in the K.S. Rogers’ Slack community. Being able to appear in user interviews while holding my dog is only possible because MagicPod is running automated tests for me right now (laughs).
Ito (MagicPod CEO): Have you been able to find any actual bugs using MagicPod for testing under various conditions?
Yoshino: Generally speaking, our testing pyramid is to “do unit testing extremely thoroughly,” and E2E tests are really the last tests that we run. Thus, I think in many cases E2E testing is done on a stable version in a staging environment. Having said that, I use MagicPod during the development phase before releasing the code to the staging phase, so I am able to find bugs ahead of time through daily testing. I think that developers would welcome having tests running daily during a code development phase. This is because if E2E could run the code deployed on a day during that night and could provide answers by the next morning, issues (bugs) could be detected early.
Ito: I think that is very useful. It seems you are utilizing the “magicpod analyzer” published by Kishi-san of Locoguide when checking the test results.
Yoshino: In addition to listing test-run results on the screen, MagicPod also provides notifications via email and Slack. However, if you just follow the list of test results every day, that may be all you achieve that day. You may be buried in the list, and failures may go unnoticed. That would be a waste of a powerful tool. Of course, MagicPod also has a Web API feature. However, with “magicpod analyzer,” if a project name is written in YAML, the test results are obtained in JSON format, which can then be deployed in a Google spreadsheet. Please click here if you are interested in getting more details about how we deployed the code.
*Reference: “With automation, regression tests increased from just once to 50 times. By creating an environment in which we could develop with peace of mind, the way we utilize time has also changed.” (Locoguide, Inc.)
Ito: It is true that once you become used to it, you can do it without thinking, so it is valuable to run tests every day and gain awareness in this manner.
Yoshino: At the MagicPod 5th Anniversary Event, you said that “the greatest feature of MagicPod is that testing can be done every day,” and I think that’s definitely true.
Finally…
Yoshino: Last year, I was invited by Ito-san to take the stage at the “Software Test Automation Conference.” According to the results of a questionnaire, 34% of those who applied to participate had “no test automation experience.” I understand that with the scarcity of employees, allocating people for such tasks is difficult. Moreover, I believe that QA engineers will only become busier than ever as we move forward. Hence, I believe that automated testing can be a powerful ally for QA engineers. We are now in an era where tools such as MagicPod make it easy to build an automated testing environment. Thus, I would encourage people to experience it for themselves.
Looking back, there was a time when I was creating test codes by myself while reading documentation on Selenium in English. Now, however, it seems that there are various communities that can be a source of inspiration. If you try MagicPod yourself and it doesn’t work out or you feel uncomfortable, you can just ask someone on MagicPod’s Slack community or Twitter page for guidance. I hope that you can experience the benefits of automated testing firsthand. Some of us are retired and have free time, so if you have any questions, just ask anytime!
