Establishing E2E Test Automation as a Company Standard: How TIS Is Enhancing Development Efficiency and Quality with MagicPod
TIS Inc.
MagicPod's CEO, Ito, spoke with TIS Inc. to learn about the key factors behind their decision to choose MagicPod and how they've been using it since its adoption.
TIS Inc.
As a business partner to over 3,000 companies in diverse sectors such as finance, industry, public services, and distribution, TIS of the TIS INTEC Group addresses a wide range of management challenges for its clients by providing "IT to support growth strategies." Leveraging over 50 years of industry knowledge and IT construction capabilities, the company collaborates with society and clients in Japan and the ASEAN region to deliver IT services, aiming to realize a prosperous society.
KEY POINTS
- Increasing quality assurance burdens and the challenge of effectively utilizing engineering resources.
- Unlimited test executions and the presence of a Slack community were the deciding factors in the selection.
- Visualizing active projects with health scores and sharing success stories horizontally.
- Promoting wider adoption through internal seminars and a Teams generative AI community.
- Shifting from offering standalone tools to standardizing them in the development environment, aiming for a change in development culture.
From left to right:
- Kento Yamaguchi, Section Chief, Development Infrastructure Center, Technology & Innovation Headquarters
- Nozomu Ito, CEO, MagicPod
Mr. Yamaguchi (hereafter, Yamaguchi): I am currently part of the Technology & Innovation (T&I) Headquarters, a division responsible for deploying software and tools to business units and training engineers. This headquarters has about 150 people, and the Development Infrastructure Center where I work has just under 10 employees. I was originally an engineer in a business division and transferred to this headquarters in 2022.
Ito (MagicPod CEO): Was there a specific reason that prompted your transfer from a business division?
Yamaguchi: When I was in charge of developing one of our company's own services in a business division, I wanted to advance our test automation, but there were no tools being offered from a central division, and it was extremely difficult to handle everything ourselves, from securing a budget to navigating internal procedures like security. This wasn't just limited to testing, and the experience led me to request a transfer, feeling that "there must be more we can do to provide support as a headquarters organization."
In fact, the headquarters was also aware of the challenge that "test automation is not progressing," so I was tasked with championing that cause and made the move.
Challenges Before Implementing MagicPod
Ito: As an engineer on the ground, what specific issues were you facing?
Yamaguchi: The biggest issue was "how to effectively utilize our limited engineering resources." At the time, a vast amount of manual testing was consuming a lot of our man-hours, and I really wanted to change this situation.
While such challenges were recognized company-wide, we hadn't been able to come up with a concrete solution. As a first step, we in T&I began with a strategy to "create a success story and then expand on that achievement."
Reasons and Process for Selecting MagicPod
Yamaguchi: After moving to my current department, I conducted a technical evaluation by actually using three different tools, including MagicPod, to move toward the adoption of a test automation tool. In terms of features, all the tools were excellent and it was hard to pick one over the other, but there were several deciding factors that ultimately led us to choose MagicPod.
First was the pricing structure that allowed for a small start. The fact that there was no limit on the number of test executions was a huge attraction for us, as we needed to go through a process of trial and error. It was reassuring that we didn't have to commit to a large contract when we weren't sure if the initiative would succeed.
Another point is that, as I recall, MagicPod was the only one that allowed us to join their Slack community before signing a contract. Seeing the actual interactions among users gave us the confidence that "we would be well-supported after implementation." With our limited human resources, the expectation for a strong support system was also a major deciding factor.
Current Usage of MagicPod
Yamaguchi: The first project we introduced MagicPod to was an internal SaaS for our sales department. Since it's an in-house service, we shared a common awareness of the challenge regarding "how to effectively utilize our engineering resources," which made the decision-making for adoption smooth.
The post-introduction onboarding was as simple as sharing the Help Center, and the dedicated engineering team proceeded proactively. They created the initial test cases independently, without requiring our assistance.
Ito: You mentioned you are currently using it for 10 projects. Is there any knowledge sharing among these projects?
Yamaguchi: I look at the "Health Score" of each project (a feature that measures and numerically evaluates the health of automated testing operations) and use the folder structures and shared step usage of various projects as a reference. For projects that seem to be struggling in the beginning, I might casually reach out and say something like, "The way this other project organizes its folders might be helpful."
In particular, a project within our headquarters called "AIChatLab" accepts requests for view-only access so that other teams can use it as a model, and we guide them to "please use it as a reference."
Ito: Beyond just increasing the number of projects, is there a broader, company-wide goal you are ultimately aiming for through this initiative?
Yamaguchi: Our goal is to increase the company-wide E2E test automation ratio and achieve cost reductions.
The most important thing is to instill a culture of test automation and enhance the maturity of the entire organization's development process. Increasing the test automation ratio is positioned as one part of that effort.
TIS AIChatLab
Ito: Beyond just increasing the number of projects, is there a broader, company-wide goal you are ultimately aiming for through this initiative?
Yamaguchi: Our goal is to increase the company-wide E2E test automation ratio and achieve cost reductions.
The most important thing is to instill a culture of test automation and enhance the maturity of the entire organization's development process. Increasing the test automation ratio is positioned as one part of that effort.
Internal Promotion Activities to Expand Project Adoption
Ito: After the initial introduction, how did you go about increasing the number of projects using the tool?
Yamaguchi: The first two projects were decided on for implementation relatively smoothly. One was the internal SaaS for sales I mentioned earlier. The other project had been using a different automation tool, but it was being discontinued, so they were looking for a replacement and we were able to introduce it at the right time.
However, we struggled to expand from there. So, starting last year, we began holding monthly internal seminars. Since we were finding our way at first, we utilized the premium support from MagicPod's Enterprise Plan and had a MagicPod representative speak at the seminar. We announced it not only within our company but to our entire group, encouraging anyone interested to participate.
Ito: For us, seeing a client like TIS leverage premium support to accelerate internal adoption was something we were very happy about. We aim to not only provide a tool but also to partner with our customers until a culture of automation takes root in their organization, so we see this as an ideal use case.
Yamaguchi: Having them directly answer specialized questions on the spot that I couldn't handle alone increased the level of satisfaction among the participants, I believe.
Through these regular sessions, we began to get a sense of the time it takes for a project to make a decision. For instance, we learned that contract development projects require about three months, including explanations to the client, while even our own services need one to two months. Therefore, we realized the importance of making early contact with people who are interested, even if they don't adopt it immediately. We are still continuing the seminars, as we now have cases where they have directly led to adoption.
Ito: What kind of information do you share internally besides seminars?
Yamaguchi: We use several channels in parallel. We announce T&I Headquarters' initiatives and events on our internal portal and utilize an internal FAQ site for engineers. On the internal portal, we reflect actual questions and answers and make them searchable via our internal AI chat to enable smooth problem-solving.
The most effective channel is the Teams generative AI community, which has about 3,000 members. There, we position MagicPod as "an AI-powered tool" and provide information such as feature introductions and expanded FAQs, creating opportunities for it to be seen by many people.
We also share internal-only case studies. Since many are client projects, we can't write down the details, but the goal is to build credibility by showing that "it's being used in this many projects."
Ito: It sounds like you are making great use of various channels. Are there any new initiatives you are planning to work on in the future?
Yamaguchi: Yes. Until now, we have been offering tools like MagicPod and GitHub Copilot individually in a "please choose" format, but we are planning to change our approach going forward. We will package our recommended tools as a single suite and provide them as a standard feature in new development environments. We want to create a path where the project side doesn't choose the tools, but rather "a good environment is provided as a set from the start."
Company-Wide Technical Challenges and Initiatives for Transformation
Ito: That idea of packaging recommended tools and providing a good environment from the start is truly about culture building itself, isn't it? Does it mean you are aiming to make automation a natural part of the process through the system, rather than trying to change the mindset of individual engineers?
Yamaguchi: Exactly. We have about 3,000 engineers in the company, but the technologies used vary by business division, making it extremely difficult to change the entire company at once with a single measure. To be honest, though, there's a part of me that feels daunted, thinking I might need to become a test expert myself to push this forward.
Ito: I don't think you necessarily have to be a test expert, and if an expert-level person is needed, we would be happy to provide support.
Yamaguchi: Thank you! One of our struggles is the mix of legacy technologies and new tech stacks within the company. With React, you can map out a testing strategy like the "Testing Trophy," but we currently don't have a clear answer on what kind of testing strategy to pursue for legacy technologies. From my perspective, I believe the key will be how we can share test assets between the development and testing environments, with E2E testing as the core.
Ito: I understand completely. Many large companies face the same dilemma where legacy systems and new tech stacks coexist, and migration is slow to progress. Trying to solve the problem head-on requires a tremendous amount of effort, but a counter-intuitive approach might be effective: thoroughly improving the experience of developing with React.
For example, regarding test automation, you could establish an efficient testing process for React projects that includes not just E2E testing but also component testing, and then powerfully broadcast the success story that "developing with React is overwhelmingly better in terms of both quality and efficiency." By doing so, you could potentially create a flow where engineers on the ground naturally want to choose the new technology.
Yamaguchi: So, rather than forcing a change on old things, we prepare new, attractive options to encourage a natural transition. I believe that going forward, we need to seize opportunities for system renewal, such as when server support ends, to proactively propose switching to modern technologies like React.
To achieve that, we first need to increase the number of personnel who can promote modern technologies internally, and we must build a structure where we ourselves can confidently say, "for our clients too, building with modern technology improves both productivity and quality." There may not be a single right way, but I want to tackle this with steady effort.
Final Thoughts
Yamaguchi: The biggest appeal I feel from actually using MagicPod is its "ease of maintenance." As you, Ito-san, mentioned at a MagicPod event, it's far more important for maintenance to be easy than for creating test cases to be easy, and I truly feel that is where MagicPod's real value lies.
The acts of test execution and recording themselves have already become quite easy with tools like Selenium and Playwright, but you can't reach the true value of test automation unless the entire process, including maintenance, becomes easier. In this respect, MagicPod is extremely outstanding, and I definitely want to convey this appeal internally.
In internal promotion efforts, we are often asked about cost-effectiveness and explanations for internal and external stakeholders, but it is clear that test automation is something we absolutely should be doing. Just as we manage repositories and source code as a matter of course, setting up CI/CD and test automation should be given the same status. I want to aim to spread the culture of automated testing at the same time.
