How Test Automation Saved Time and Transformed Mindsets: Inside POLA ORBIS's Strategy to Overcome "We'll Do It Someday"

POLA ORBIS HOLDINGS INC.

Browser testing

We sat down with POLA ORBIS HOLDINGS INC. to hear from them directly. MagicPod's CEO, Ito, asked about why they chose our tool and how their journey with it has unfolded since day one.

POLA ORBIS HOLDINGS INC.

The POLA ORBIS Group is a family of beauty brands centered on cosmetics. With POLA as its founding brand, the group will celebrate its 100th anniversary in 2029. With its core brands—POLA, known for its expertise in anti-aging and skin brightening, and the skincare brand ORBIS—the group develops a range of distinctive brands that respond to the diversifying values of "beauty."

KEY POINTS

Testing that had become a mere formality was creating a vicious cycle of thinking, "Is this even worth doing?"
Transparent pricing and unlimited test runs were the deciding factors in choosing MagicPod.
Manual testing, which had taken up to four hours per sprint, was completely eliminated.
The success of starting small changed the team's entire outlook, transforming testing from a "chore" into a "culture."

From left to right:
- Ms. Miho Hayakawa, Team Leader, IT Product Development Team, Group Digital Solution Center
- Mr. Taizo Kawada, IT Product Development Team, Group Digital Solution Center
- Nozomu Ito, MagicPod CEO

Ms. Miho Hayakawa (Hayakawa): I serve as the Manager and Product Owner for the IT Product Development Team. Our team is part of the Group Digital Solution Center (GDSC), which has about 100 members in total. In the past, each brand had its own information systems department, but they were all integrated into the GDSC in April 2022 to maximize synergy across the group.

Within the GDSC, our IT Product Development Team consists of about 10 people, including both employees and contract members. We used to outsource all of our system development, but to keep up with the rapid pace of the market, we launched an in-house development team in January 2023. We started by bringing the development of Orbis's systems in-house. Because of that history, a lot of our work is still related to Orbis, but we aren't a brand-specific team; we're set up to work across the entire group.

Mr. Taizo Kawada (Kawada): My name is Taizo Kawada, and I'm the Scrum Master for the same IT Product Development Team. I led the effort to research, select, and implement MagicPod. At my previous job, I had seen firsthand the benefits of having automated testing—and the headaches that come from not having it. So, when I joined this company and saw that all testing was being done manually, I knew we had to do something about it, and I began looking into automation.

Ito (MagicPod CEO): Could you tell me a bit more about the specific systems you're working with now?

Kawada: The main system we're handling right now is for internal use, specifically the one that manages products for Orbis. This system's features were originally part of a massive, core legacy system that had been built up at Orbis over many years. The problem was that it was difficult to quickly improve its functionality to keep up with market changes. So, our team carved out the parts that required frequent changes and rebuilt them from the ground up. Our tech stack is Python for the backend and React for the front end.

The Challenges Before MagicPod

Kawada: I joined right after the in-house development team was formed, so the organization was brand new and the structure wasn't really solidified yet.
At the time, we were using Cypress for testing, but tests weren't being run automatically or on a schedule, and we couldn't keep up with maintenance. It was effectively non-functional. As a result, we had to rely on manual testing to verify that anything worked.

One of the biggest struggles was that the product management system had two distinct use cases: one for Orbis employees on PCs, and another for our in-store Beauty Advisors (BAs) on iPads. This meant we had to run tests in a normal browser and run separate, iPad-equivalent tests using Chrome's mobile emulation.

On top of that, we were still in the early stages of building our culture as a development organization. We all recognized the need for automated testing, but we just weren't making any headway. It was obvious that as we added more features, our testing time would only increase, so we started looking for an automation tool.

Ito: Do you have a dedicated QA team for testing?

Hayakawa: We don't have a QA team, so the development team handles everything from implementation to testing.

Kawada: Based on my past experience, I knew from the heart that automated testing is essential for delivering value in an agile way. I explained to Hayakawa, "If we don't tackle this now, it will absolutely become a bottleneck for our development speed down the road," and so we moved forward with automation.

Mr. Taizo Kawada, IT Product Development Team, Group Digital Solution Center / Nozomu Ito, MagicPod CEO

Why and How We Chose MagicPod

Kawada: As I researched various automation tools, I narrowed our list down to three final candidates, including MagicPod. As I mentioned, we had existing test code, but it was broken and we didn't have the resources to maintain it. So, we prioritized ease of use and focused our search on no-code tools.

Ito: When you were narrowing it down to those three, what were your main criteria?

Kawada: I had never used a no-code testing tool before, so at first, I wasn't sure what to look for. While I knew we'd have to try them out to make a final decision, switching tools after you've already committed is a huge pain. I figured that a tool with a solid track record in the industry would be more mature and functionally polished, so I prioritized finding a well-established solution.

Ito: That's a priority we hear from a lot of companies. So, out of those three, what ultimately made you choose MagicPod?

Kawada: I saw a lot of case studies for MagicPod on sites like Qiita and Zenn, and the fact that MagicPod hosts its own seminars and has a user community gave us a lot of confidence. Ultimately, the deciding factors were that there's no limit on the number of test runs and that the monthly cost was reasonable, with single-month contracts available, which let us start small.

Other tools kept their pricing private and required you to contact them, which was a hurdle. But MagicPod had its pricing clearly laid out on the website, so I could immediately see that it was something we could try. That's when I requested a trial.

Ito: And how did you find the trial?

Kawada: What really stood out was how I could use most of the features intuitively, without overthinking it. I got it up and running after just a few questions with the sales representative, which made me think, "If I can use this, so can the rest of the development team." I rarely felt stuck wondering how to do something I wanted to do, and my lasting impression was that it was just incredibly user-friendly.

Ito: Thank you so much!

How We're Using MagicPod Today

Kawada: Currently, we are running two types of tests on our product management system: standard browser tests and iPad-equivalent tests using Chrome's mobile emulation. Specifically, we run a set of essential happy-path tests every hour between 9 a.m. and 6 p.m. on weekdays. The results are sent to a Slack channel, and our process is that if an error persists, the development team investigates.

Looking ahead, we're preparing test cases with the goal of running a full, comprehensive test suite once a day. Eventually, we want to take it even further by integrating it with our CI/CD pipeline to verify our most critical happy paths before every deployment.

Ito: Have you seen any measurable effects or time savings since the introduction?

Kawada: Since the iPad use in our stores is limited to simple read-only functions, we can now cover it completely with MagicPod. Thanks to that, the time we spent on manual verification—which used to be two to four hours per two-week sprint—has been reduced to zero. Overall, we do spend time creating and maintaining tests in MagicPod, so the net time savings are modest for now, but we've definitely felt that our hourly regression tests help us catch impacts on existing features much faster.

Ito: About how many people are using it at the moment?

Kawada: We've made it accessible to everyone on the IT Product Development Team. Not everyone uses it every day, since some of our products don't use MagicPod, but our rule in Scrum is that whoever picks up a task is responsible for making any necessary fixes. In that sense, you could say everyone is an active user.

Ito: What aspects of it do you feel are the best fit for your team?

Kawada: The fact that there are no limits on test runs or the number of users is incredibly convenient. I also love that you can create tests intuitively with no code and that the interface is so simple. There's a rich set of commands, and the ability to create shared steps for reusability and maintainability has been a huge help.

Ito: We're glad to hear it! By the way, have you had a chance to try our AI agent, "MagicPod Autopilot"?

Kawada: Yes, we've been experimenting with it. We're always talking about how we need to use generative AI to become more efficient, and we hope you'll continue adding features that help with test generation and reduce maintenance time. Our company is already moving forward with tools like GitHub Copilot and Devin, so we're hopeful that the MagicPod MCP server (CLI) will be enhanced to create a seamless, one-stop workflow from development all the way through testing.

Ito: That is definitely something we want to work on.

From "Is This Worth It?" to a Culture Where Passing Tests is the Norm

Ito: Beyond the day-to-day operations, have you noticed any changes in your team's culture or development process?

Kawada: I think it's had a really positive impact on the team's mindset. When I first joined, everyone knew that having tests that weren't maintained, weren't run on a schedule, and wouldn't even run when you tried was a bad situation. At the same time, because it would take so much work to get them running correctly, a negative vibe started to creep in—people would say "we'll get to it someday" or even "is there any point in doing this at all?"

Once you get into that state, the priority of fixing automated tests just keeps dropping, and it leads to a vicious cycle: automated tests aren't maintained → the tests fail → no one sees any benefit → motivation to maintain them disappears.

But we could see that as the product grew, the time spent on manual testing would also grow, leading to a clear drop in development speed and output. I knew we had to break that cycle somewhere. That's when I thought that by using an approachable, no-code tool, the team could experience the benefits of test automation for themselves.

And in practice, once we started with just a few test cases and gradually expanded our coverage, getting automated tests running in MagicPod, people started to see the benefits. A new attitude emerged, where they'd say, "This is failing, we need to look into it." Today, I feel our mindset has completely shifted to: "Tests are supposed to pass, and if they're failing, we need to fix or improve something."

Ito: Do you think that kind of vicious cycle can be broken by simply implementing a tool like MagicPod? Or is it more important to start by changing the mindset first?

Kawada: That's a tough question. I think it's very difficult to change a mindset without tangible results. It might be different if it's a top-down mandate to "just do it," but my personal view is that for things like this, if the people actually doing the work don't feel the impact, it's hard to make progress and nothing really changes. I'm sure every team has its own way of doing things, but I feel that you need something you can experience firsthand to get started. And as an option for that, I think MagicPod is very effective.

In terms of a concrete approach, I'm sure that at other companies, too, there are parts of their products where they think, "This area has a lot of bugs," or "We're constantly making fixes over here." I think starting in those places is a great way to see results quickly.

Mr. Taizo Kawada, IT Product Development Team, Group Digital Solution Center

Final Thoughts

Kawada: To maintain quality while responding quickly and flexibly to rapidly changing market needs, I believe streamlining the testing process is absolutely essential. While you can't automate everything, I think MagicPod is a powerful option for front-end and E2E testing. We ourselves started with just one or two test cases and gradually expanded from there. The key to success, I believe, is to not aim for perfection from day one, but to start small.

In our case, bringing in MagicPod changed our perception of testing from a "chore we had to do" to an "important process for protecting quality." For anyone looking to start small, MagicPod's pricing structure makes it very approachable. Plus, the extensive Japanese-language support and community are reassuring, as it's easy to find information when you get stuck.

I imagine the challenges we faced are a common story that many companies can relate to. Of course, for a tech-focused company, testing might be second nature, but I suspect there are many companies out there that can't do it effectively due to a lack of resources. As features grow and services mature, the amount of testing work snowballs. To "automate what can be automated and spend your time on what truly creates value," I believe automation is something you absolutely have to do.