Test Automation For Quality Culture! How Hacobu Built a Product QA Foundation on the Principle That "No-Code ≠ No Design"

Hacobu Co., Ltd.

Mobile app testing Browser testing

We spoke with Hacobu Co., Ltd. about what led them to choose MagicPod and how they have been using it since implementation. The conversation was led by MagicPod CEO Nozomi Ito.

Hacobu Co., Ltd.

Hacobu offers the cloud logistics management solution "MOVO" series, logistics DX consulting under "Hacobu Strategy," and system integration and AI adoption support under "Hacobu Solution Studio."
Their lineup includes the truck appointment scheduling service "MOVO Berth," the fleet tracking service "MOVO Fleet," the dispatch order and management service "MOVO Vista," and the AI-powered ordering and transport optimization service "MOVO PSI" — all delivered as cloud services — as well as the smartphone app "MOVO Driver," designed to transform the working experience for truck drivers.
Hacobu supports the optimization of inter-company logistics as a logistics DX partner.

KEY POINTS

  • Over-reliance on individual knowledge in testing and the lack of a structured regression testing process were the key challenges before implementation
  • MagicPod was selected for its maintainability and Shared Step functionality, and its ability to support structured test creation
  • From the outset, a Data Driven Testing-first design was adopted with long-term operation in mind
  • An automated test case review system was built using MagicPod MCP integrated with Cursor
  • Incorporating the morning Batch run tests into the release approval criteria raised quality awareness across the entire organization
From left: Tetsuya Kawase — Technology Division, Product Development Group, Vista Department Nozomi Ito — MagicPod CEO

From left:
Tetsuya Kawase — Technology Division, Product Development Group, Vista Department
Nozomi Ito — MagicPod CEO

Kawase: I joined Hacobu in August 2023. I am responsible for QA on the dispatch order and management service "MOVO Vista." Hacobu does not have a centralized quality assurance department that spans the entire organization. Instead, we have built a "Product QA" structure in which a dedicated QA professional is embedded in each product team to develop and execute the optimal test strategy for that product.

With this structure, the consensus-building needed to launch new initiatives is handled entirely within the team, which means QA decisions directly shape the test strategy and quality improvement efforts for the whole team. I find it easier to move forward with genuine buy-in, and the agility it enables is a significant advantage. We have nine full-time QA engineers, each operating independently, and we hold a sharing session roughly every two weeks. We also have active informal groups — similar to guilds — where members with shared interests come together in small groups to work on things collaboratively.

I originally started my career as a software engineer, spending about 14 years at an SES company working on system development for other organizations. In the latter part of that period, I increasingly moved into roles overseeing entire teams and projects, and on smaller engagements I had more and more opportunities to handle scenario testing and integration tests.

After that, I joined another company in a business planning and SI vendor liaison role, and became involved in an app development initiative the company was running at the time. Through that experience, I discovered the interest of engaging not only with the builder's perspective of quality — "making sure the system works" — but also with quality from a user's point of view: "what does it take for customers to be genuinely satisfied?" That sparked my interest in the field, and after gaining experience at a third-party verification firm, I joined Hacobu.

Reasons and Background for Selecting MagicPod

Kawase: Product QA is one of Hacobu's strengths, but it also carries the risk of over-reliance on individual knowledge. When I joined, the company was in a phase that prioritized the speed of agile development, and regression testing had not been properly established — the prevailing understanding was that manual testing would cover what it could. The judgment of which tests to run depended on the discretion of each individual, and the situation was such that incidents like "there's a regression somewhere but we can't identify the cause" could easily arise.

Ito: Was it you who drove the adoption of automated testing and the selection of a specific tool?

Kawase: It had already been under consideration before I joined. It seemed to be a concern at the leadership level as well, and when MagicPod was introduced, our CEO responded positively. After I joined, the evaluation came down to a choice between a recording-type tool and MagicPod. In that process, we assessed MagicPod favorably for its straightforward creation flow and high maintainability. Personally, I also appreciated that Shared Steps were available.

Incidentally, I first came across MagicPod at my previous job at a third-party verification firm, when a client company was exploring automation of their regression testing. Another person was leading that effort so my involvement was limited to trying it out, but they ultimately adopted it there as well.

Ito: Many companies appreciate the ability to run tests without limits — was that not a significant factor for you?

Kawase: At the time, I don't think I had a concrete enough picture to fully appreciate it. That said, hearing you mention it now, being able to run tests every day without hesitation is genuinely a major benefit.

Ito: Thank you! Were overseas testing tools or Selenium also among the candidates?

Kawase: In the earlier rounds of evaluation, yes, those were on the list. However, recording-type tools — which are common among overseas products — tend to depend heavily on how each individual builds tests, making it difficult to maintain consistent quality. Coding-based tools, on the other hand, raised concerns about the onboarding learning curve, since programming experience varied across team members, and so they were ruled out. Ultimately, we selected MagicPod for its ability to support structured test creation and its high maintainability.

Ito: With product teams operating separately under a Product QA structure, I imagine it can be difficult to drive adoption across the organization. How did you approach that?

Kawase: Initially there was another person driving the effort alongside me, and that person first introduced MagicPod in the truck appointment scheduling service "MOVO Berth." Through that process, we established a full set of review mechanisms, review criteria, and test creation guidelines, and when rolling out to other products, we applied those frameworks while continuing to refine them.

I believe that in test automation, what matters more than the tool itself is designing the structure through which quality is assured. It is often misunderstood that "because it's no-code, no design is necessary" — but my view is that "no-code ≠ no design."

In practice, when teams jump straight into implementation without sufficiently thinking through the test structure, data handling, and rule design — the "architecture" of automation — changes become difficult to make later, and the result is often what I would call a hollowing-out of the automation: failed tests get left unresolved, and the whole effort falls into disuse.

At the same time, the mechanics of how to use the tool itself can be picked up along the way through hands-on practice. That is precisely why our team prioritized getting the hard-to-change, low-leverage-once-set elements — test structure, data design, and operational rules — sorted out before we began implementation.

Foundation Design in the Early Stages of MagicPod Implementation

Ito: What specifically did that preparation involve?

Kawase: In software development, coding standards and variable naming conventions are typically in place to ensure a baseline of quality. Drawing on my own background as an engineer, I applied a similar approach to test automation and organized the groundwork into four broad layers.

The first was structural design, with post-implementation maintainability and ease of implementation in mind. By clarifying which MagicPod features map to which traditional test structure concepts, the team was able to develop shared understanding and move forward with implementation on a common foundation.

Next, we established guidelines and naming conventions to enable safe Data Driven Testing. Naming rules are differentiated by variable type — for example, shared variables use uppercase snake_case, while variables within test cases use camelCase.

The third layer was operational design. Establishing structure, guidelines, and conventions alone is not enough to ensure they are actually followed, so we documented the implementation flow for test cases, the review flow, and the process from investigating a test failure through to resolution.

Finally, we prepared a "usage guide." In the early days after implementation, test case creation was a process of trial and error, so we set up a page to accumulate observations and anti-patterns from implementation, allowing the team to share good practices and patterns to avoid. Through this preparation, I believe we were able to build a foundation where operations would not become hollow over time and where adjustments would remain manageable.

Ito: That is impressive. It is rare for an organization to have this level of preparation in place before introducing a tool. I felt that the "no-code ≠ no design" philosophy is genuinely reflected in how you operate.

Kawase: Thank you. That said, maintaining this operational foundation over the long term also required keeping the team motivated in the early stages. What proved helpful there was MagicPod's analytics feature. In particular, we set stabilizing the "health score" — which measures the health of operations — in the 90s as our near-term goal. Because the evaluation criteria are broken down into incremental stages with the importance and priority of each item displayed, it was easy to set short-term milestones, and I felt that contributed to sustaining motivation.

How MagicPod Is Being Used

Kawase: Currently, we have automated testing for MOVO Berth, MOVO Vista, and a third — including a smartphone app — bringing the total to three products covered by MagicPod. Accounts have been issued to developers as well, bringing the total to around 50 users. Depending on the team, in the product I am involved in, MagicPod is used as part of the release approval process, and there are cases where developers independently run MagicPod outside the regular release schedule to make their own release judgments.

Scheduled Execution kicks off at 7:30 in the morning and completes in about 45 minutes, with results delivered to a dedicated Slack channel that only receives execution results. In general, when a Batch run test fails, QA investigates — but members who are engaged will proactively look into the error on their own. They first determine whether the issue is transient, and if it turns out to be a defect on the frontend or backend, a fix is applied and the release approval run is executed again.

There was actually a point in the past where production releases were going out before the daily test results had been reviewed, and we had a problem with testing and releases being disconnected. So we established a clear release standard: "all morning tests must have passed." For runs outside that window as well, we use a workflow automation tool — a Slack command triggers a Batch run automatically.

Ito: It sounds like your development engineers are also engaging with MagicPod. Have there been any positive responses since the rollout?

Kawase: We have received positive feedback about the fact that defects that previously slipped through testing are now being caught before release. There is a shared understanding across the entire team that proper verification happens before a release goes out, and I feel that the resulting sense of assurance is a visible effect. Beyond that, incorporating MagicPod into the release approval process has led to engineers proactively asking "Did you run MagicPod?" — and a culture has emerged where the team locks down release content by the day before to make it in time for the morning tests. That improvement in quality awareness across the whole organization has been one of the most significant outcomes.

Ito: It is wonderful to see such a healthy culture taking root across the development team. For that kind of stable daily operation to be sustainable, the maintainability of the test cases themselves also becomes important. Do you have any criteria for granularity when creating Shared Steps, or any tips for improving maintainability?

Kawase: As a baseline, we always implement API calls as Shared Steps — handling everything from the call itself through to result retrieval and return as a single unit. Beyond that, our policy is to actively consolidate anything that is used frequently or corresponds to shared UI components on the frontend.

Another practice we use to improve maintainability is an automated review system leveraging AI. Specifically, we integrate Cursor with MagicPod's MCP. We preload all of our coding rules and variable naming conventions into Cursor's configuration and created a custom command from there.

When you run this command and specify a test case number, the AI automatically performs a review against the conventions and feeds back the points that need to be corrected. In practice, each team member runs this automated review in their local environment, writes the output to Notion, and then uses that as the basis for requesting a final confirmation review from other team members. Once there are no issues, the test case moves into live operation.

Beyond reviews, we also run processing via Cursor through the MCP server to re-execute only the tests that failed in a Batch run, and to fetch data via the API and generate graphs showing trends in execution time. Aggregation and analysis tasks that would take more effort through the GUI can be executed simply by entering a natural language prompt, and I feel that has significantly expanded the freedom we have in managing test automation operations.

Ito: It sounds like quite a sophisticated setup — automated AI-powered reviews, aggregation via MCP, and more. Going forward, how are you thinking about rolling these practices out to other team members and teams within the company?

Kawase: Our organizational culture places a very strong emphasis on individual autonomy and team-level optimization. So rather than managing things top-down, the challenge going forward is how to expand these practices while leveraging the initiative of people on the ground. That said, one of Hacobu's strengths is a culture where proposing to try a new tool gets a "let's go for it" response from the company, and where lively discussion happens across team boundaries on Slack and elsewhere. I think the path forward is to leverage that environment to spread adoption organically.

Toward a QA Organization That Keeps Pace with the Speed of AI-Driven Development

Ito: What challenges are you hoping to address going forward with the help of AI?

Kawase: Proficiency with AI varies across teams and individuals. Some members struggle to effectively translate AI suggestions into test cases, while others find themselves in trial-and-error mode when unexpected side effects arise during fixes. There is also a significant challenge around not being fully aware in advance of which MagicPod test cases will be affected by a given change — even when those tests are part of the release approval criteria, the impact sometimes only becomes clear after the fact. That invisible blast radius is something I want AI to help us detect.

Ito: You mean identifying the scope of impact from a change. One approach that has emerged recently is to pass a GitHub pull request URL to an AI editor like Cursor via MagicPod MCP and instruct it to "list the tests that are likely to be affected by this change" — which allows for a reasonable degree of pre-run impact assessment without actually executing the tests.

Kawase: That sounds very useful — I'll look into it. Our development environment is quite complex, with dev and staging environments running alongside topic environments for each development unit. When resources allow, we can do comprehensive checks in the topic environment, but when things get busy, there are inevitably cases where verification ends up being limited to the dev environment.

Inheriting tests created by someone else is particularly challenging — it is not easy to read back the design thinking behind them. As a result, the tendency is to apply local fixes only to the step where an error occurred, without considering the overall structure, and that is one of our significant operational challenges.

Ito: I see — the more complex the environment, the more important it becomes to triage before running tests. Being able to reduce unnecessary executions through early detection is a meaningful benefit. And beyond just fixing the error location, supporting developers in making corrections that account for the broader context — in a way that is simpler and more accurate — seems like it will become increasingly important going forward.

Kawase: Exactly. On that point of "support for making the fix," I feel that MagicPod Autopilot would become even more powerful if it were extended to support Shared Steps. Even now, I have confirmed that when data patterns are in use, Autopilot appropriately retrieves the relevant variables — and that seems like something we can put to use. For things like calculation logic that I don't work with often, it feels easy to delegate the finer details to it.

Ito: We talked about Cursor and MCP for impact assessment earlier, but there is a step beyond that worth mentioning as well. Depending on how it fits your operational flow — are you currently using the approach of reviewing code in an AI editor and then applying those review results directly to test fixes via MCP?

Kawase: We are not doing that yet.

Ito: Actually, it has recently become possible to call Autopilot from MCP, so that by instructing the editor to "apply these review results to the tests as well," new test creation and modifications can be carried out automatically. At this point in the current specifications, changes are applied directly to the main branch rather than a test branch, so there is one extra step needed — checking the execution history to confirm everything looks correct.

Kawase: For newly created tests, having changes applied directly to the main branch is completely fine. That sounds very useful — I will definitely try it. Thank you.

Recently, AI-powered code generation and similar tools have dramatically accelerated the speed of feature development, and there is a shared sense of urgency within the company that if QA remains reliant on traditional manual testing, it will become a bottleneck for the entire release process.

Ito: How to keep QA pace with development speed as AI continues to accelerate it is a major theme for the industry as a whole. What will be needed going forward is not disposable tests written purely for speed, but tests that are resilient to change and built for stable long-term operation. MagicPod will keep evolving to support that.

Closing

Kawase: In our initial meetings after implementation, you shared common failure patterns and operational tips with us upfront — and that enabled us to design a structure with long-term operation in mind from the very beginning. Looking back on these past two years, I am genuinely grateful that we got that initial foundation right.

I think most engineers today understand the value of automating regression testing. But the true value, I would argue, lies in being able to achieve that "without depending on any one individual's technical ability — with consistent quality regardless of who touches it." My honest assessment is that adopting MagicPod was the right call — as a tool for embedding an automation culture across the organization and eliminating over-reliance on individuals.

Hacobu Co., Ltd.