～Supporting Users in Implementing Best Practices for Test Automation～

MagicPod (Chuo-ku, Tokyo; CEO Nozomi Ito) is excited to announce a new feature of its AI test automation platform, MagicPod, allowing users to check the health of test automation projects created on MagicPod and suggest improvements.

---

Background

Implementing best practices in test automation, such as designing highly maintainable tests and ensuring consistent test execution to prevent obsolescence, poses significant challenges that can impede the success of automation projects.

Despite MagicPod's efforts to disseminate these best practices through diverse resources, events, and the support initiatives led by our customer success team, we have encountered limitations in delivering consistent and quantitative support to all our users.

In response to this challenge, MagicPod has introduced a health score feature designed to empower users to assess the "health" of their automated testing projects within the MagicPod ecosystem. This feature not only provides insights into the current state of projects but also offers tailored suggestions for improvement. Users can conveniently access the contents of the health score through an intuitive in-platform dashboard or by utilizing the Web API offered by MagicPod. This enhancement aims to address the complexities associated with implementing best practices and promote the overall success of test automation projects.

Benefits of the Health Score Feature

The new health score feature offers several benefits:
・User Empowerment: In contrast to common practices in SaaS companies, where user analytics are often collected for the purpose of driving customer success but remain inaccessible to users, MagicPod stands apart by openly disclosing health scores. This transparency empowers users to actively enhance the health of their own projects.
・Industry-Backed Logic: The calculation of the health score is grounded in industry knowledge that MagicPod has amassed through extensive experience in assisting numerous users with their testing automation. This feature translates this wealth of knowledge into tangible insights and recommendations, making it directly accessible to users through numerical scores.
・Clarity in Project Health: While engineers and QA specialists recognize the importance of maintaining the "health" of test automation projects, the criteria for what constitutes a "healthy" project can often be vague. MagicPod addresses this ambiguity by quantifying project health through a concrete "health score." This quantification empowers teams to precisely identify issues and establish a framework for continuous measurement and improvement.

“Health Score” Details

Health Score
The health score of a project is assigned on a 100-point scale, accompanied by a list of actionable items for significant improvement. The calculation of the health score is based on the following key indicators:

・Team Involvement: Is an adequate number of team members actively engaged in the automated testing project?
・Test Coverage: Have a sufficient number of tests been developed to ensure comprehensive coverage?
・Test Length: Are individual tests of a reasonable length, promoting efficiency and manageability?
・Shared Steps: Are "shared steps" implemented to enhance test maintainability and streamline updates?
・Locator Stability: Are "locators" utilized to guarantee stable and consistent test results?
・Consistent Execution: Are tests executed on a daily basis to ensure consistency and ongoing validation?
・Failure Rate Management: Is the test failure rate maintained at an acceptable level?

---

Example of a good health score

---

Example of a bad health score

Improvement Opportunities
Suggestions for improvement, specifying the current and recommended number of points.

---

Web API
The same health scores and suggestions for improvement that are visible on the screen can also be retrieved via MagicPod's Web API, which can be used to notify users of their health scores in their own chat space or embedded in their own dashboards.

---

MagicPod Web API
https://app.magicpod.com/api/v1.0/doc/


Target
The health score feature is available to MagicPod paid plan users, regardless of plan, at no additional charge.

---