Topics Report on the Results of the Questionnaire on Electronic Data Application

Printable PDF

On June 20, 2014, one year after the announcement of the "Health and Medical Care Strategy" in June 2013, the Ministry of Health, Labour and Welfare (MHLW) issued the "Basic Approach to Electronic Data Submission at the Time of Application for Approval (Pharmaceutical Affairs Agency Announcement No. 0620-6, partially revised on January 24, 2019)*1 (hereinafter, "Basic Approach"), which has been in effect since October 2016. Electronic data submission at the time of application for approval" (hereinafter referred to as "electronic data submission") for pharmaceutical products began to be accepted in October 2016. With the final year of the transitional measures period approaching, the Pharmaceutical Manufacturers Association conducted a "Questionnaire on Electronic Data Submission" from April 26 to May 17, 2019, in order to investigate each company's experience with electronic data submission. Some of the results of the survey are reported below.

Introduction

This survey was conducted on 63 member companies of the Data Science Subcommittee of the Drug Evaluation Committee of the Pharmaceutical Manufacturers Association of Japan (PMAJ) to investigate their experience in electronic data submission. The task team found no significant differences between domestic and foreign companies, so this newsletter reports on the overall results, excluding some results. Unless otherwise indicated, the percentages are calculated based on the 56 companies that responded validly to the survey. In addition, the consultation categories with the Pharmaceuticals and Medical Devices Agency (PMDA) were changed in April 2019, and in addition to "Consultation on Confirmation of Electronic Data Submission", "Consultation on Method of Electronic Data Submission" and "Consultation on Exemption from Electronic Data Submission" were newly added. In this survey, we investigated the "Consultation on Confirmation of Submission of Electronic Data for Applications," which has been conducted in the past.

Survey Results (1) Questionnaire Responses

Of the 63 companies surveyed, 56 responded (89% response rate). Of the 63 companies surveyed, 42 (75%) were domestically capitalized (hereafter referred to as "domestic capital") and 14 (25%) were foreign capitalized (hereafter referred to as "foreign capital") ( Figure 1 ). The number of data managers, analysts, and statisticians working in Japan was also surveyed as background information on the companies that responded to this survey. 25 companies (45%) had the largest number of 1 to 10 employees. This was followed by 14 companies (25%) with 21-50 employees, 11 (20%) with 11-20 employees, and 3 (5%) with 51 or more employees. Three companies (5%) had no employees in Japan (0 employees) ( Figure 2 ).

Figure 1: Questionnaire responses (N=56)

Fig. 2 Number of data managers, analysts, and statisticians working in Japan (N=56)

Survey Results (2) Experience of Electronic Data Submission for Applications in Japan

The results show that more than half of the companies have experience in electronic data submission ( Figure 3 ). Questionnaire on the Guidelines for the Description of Confirmation Consultation Materials (Appendix 8) and Electronic Data Submission (63 companies surveyed, 48 companies responded)" showed that 18 companies (37.5%) had experience in submission and 30 companies (62.5%) had no experience in submission, and compared to the results of this survey, it is clear that the transitional measures period was used to gain experience in electronic The results of this survey indicate that the respondents with "no experience of submitting data" have gained experience in electronic data submission by taking advantage of the transitional period.

Fig. 3 Experience of electronic data submission for applications in Japan (N=56)

The 25 companies that responded "no experience" were asked when they plan to make their first electronic data submission. 8 companies (32%) responded "before March 31, 2020" and 17 companies (68%) responded "on or after April 1, 2020. Including companies that have already experienced electronic data submission, we found that approximately 70% of companies will experience data submission before the end of the transitional measures period.

Survey Results (3) Number of Consultations on Confirmation of Electronic Data Submission of Applications

The total of 37 companies (66%) that have already submitted data once was 9 companies (16%), 2 companies (4%) twice, 4 companies (7%) three times, 6 companies (11%) four times, 4 companies (7%) five times, and 12 companies (21%) six or more times ( Figure 4). 19 companies (34%) responded zero, which is a significant increase from the previous survey (2 companies). Many of the companies that responded to this question were those that indicated in survey result (2) that their first electronic data submission would be on or after April 1, 2020. The results indicate that even companies that have no experience in data submission are conducting consultation on confirmation of electronic data submission for applications if they plan to submit data on or before March 31, 2020.

Fig. 4 Number of times of consultation on confirmation of electronic data submission for applications (N=56)

Next, we surveyed the 37 companies that responded that they had experienced consultation on confirmation of application data submission, and asked them about the "content of consultation on confirmation of application data submission" ( Figure 5 ). Looking at the responses other than "Other" in order of frequency, "SDTM (Study Data Tabulation Model) specifications" was the most common (25 companies, 68%), followed by "Explanation of Validation Issue" (24 companies, 65%), "ADaM (Analysis Data Model) specifications" (18 companies, 49%), and "Program submission (49%), "Program Submission" (13 companies, 35%), and "Define Specifications" (6 companies, 16%).

Fig.5 Contents of consultation on confirmation of application data submission (multiple responses allowed, N=37)

Eighteen companies (49%) responded "Other," indicating that about half of the companies consulted on items other than those specified in advance in this survey. The most common "Other" consultations included "Analysis Results Metadata," "Clinical Pharmacology Data Submission," and "Legacy Data Conversion.

The contents other than "Explanation of Validation Issue" are handled in the newly established "Electronic Data Submission Method Consultation" (fee-based), so please be careful about the application categories.

Survey (4) Preparation of Exhibit 8*2

The results were 35 companies (63%) with "Yes (experience of submitting to PMDA)," 4 companies (7%) with "Yes (no experience of submitting to PMDA)," and 17 companies (30%) with "No," which was almost the same as the result for those with electronic data confirmation consultation (37 companies) (see Fig. 6 ).

Fig. 6 Experience in creating Exhibit 8 (N=56)

Survey Result (5) Electronic Data Submission for Applications

The 31 companies that responded that they have experience with electronic data submission were also surveyed on several items related to electronic data submission.

First, we asked whether they submitted data electronically via Gateway ( Figure 7 ). The results showed that 21 companies (68%) "submitted via Gateway using TSV files," 16 companies (52%) "directly entered data into the Gateway system and submitted via Gateway," and 2 companies (6%) "submitted without using the Gateway. The two companies that "submitted without going through the Gateway" all submitted their reports by "submitting them over the counter. Comments from the companies that "submitted via Gateway using TSV files" included: "The regulatory affairs department creates study data information according to TSV file specifications, and the study data creation department checks the TSV file information" and "Simple information, such as controlled terminology, can be submitted via the Gateway pull-down menu. The description for clinical pharmacology studies is entered in the TSV file. We found that some companies are now improving their TSV file operation methods.

Fig. 7 Data submission via Gateway (multiple responses allowed, N=31)

Next, we surveyed "how many weeks prior to the scheduled application date was the data initially submitted. In the case of companies that experienced electronic data submission for multiple themes, the answer is based on the maximum value (the case where data was submitted early). Two companies (6%) answered "less than one week," three companies (10%) answered "one week to two weeks," three companies (10%) answered "two weeks to three weeks," six companies (19%) answered "three weeks to four weeks," 11 companies (36%) answered "four weeks to five weeks," five companies (16%) answered "five weeks," and one company (3%) did not respond ( Figure 8 ). Twenty-two companies (70%) submitted data more than three weeks in advance, indicating that many companies tend to submit data early in anticipation of the time required for PMDA validation after data submission and the risk of correction instructions after submission.

Figure 8 How many weeks before the expected application date did you initially submit data (N=31)

The survey then asked "How many days after the initial submission was the data received acceptable or not acceptable" ( Figure 9 ). In cases where the respondents had experienced multiple electronic data submissions, the maximum value (i.e., the case where more days were required) was used as the answer. The results showed that 15 companies (48%) answered "0 to 3 days," 12 companies (39%) answered "4 to 6 days," 1 company (3%) answered "7 to 9 days," 1 company (3%) answered "10 days or more," and 2 companies (7%) did not respond. 0 to 6 days accounted for the majority, 27 companies (87%), which is the "number of days required to obtain the results of acceptable or non-acceptable" as published by PMDA. The results are consistent with the PMDA's published "number of days it takes to obtain acceptable/unacceptable results" of 5 business days. However, since the survey was conducted during the transitional measures period, it can be inferred that the results include cases where only a portion of studies were submitted. In the case of electronic data submissions that include a large number of studies to be submitted, the PMDA is expected to take more time for validation, so it is desirable to submit electronic data as early as possible.

Figure 9 How many days after initial submission was the data received acceptable or not acceptable (N=31)

Next, we surveyed the number of times that resubmissions of electronic data were made after the initial submission until the data was accepted. If the applicant has submitted electronic data more than once, the maximum number of times (more resubmissions) was used as the answer. Twelve companies (39%) responded "zero (no resubmission)," 10 companies (32%) responded "once," 8 companies (26%) responded "twice," and 1 company (3%) did not respond, indicating that more than half of the respondents had experienced resubmission ( Figure 10 ).

Figure 10 Number of times electronic data was submitted before becoming acceptable after initial submission (N=31)

The 18 companies that experienced resubmission were asked about the length of time it took from initial submission to acceptance. Five companies (28%) responded "less than one week," four companies (22%) responded "one week to two weeks," three companies (17%) responded "two weeks to three weeks," and two companies (11%) responded "three weeks to four weeks," "four weeks to five weeks," and "six weeks or more" respectively ( Figure 11 ).

Figure 11 Time taken from initial submission to acceptance (N=18)

Since resubmission requires time for "correction of the pointed out items," "resubmission of data at Gateway," and "validation on the PMDA side," it is recommended that electronic data submission be conducted well in advance of the application date (electronic data can be submitted 5 weeks prior to the scheduled application date) from the perspective of the risk of resubmission.

Survey Result (6) Tools used for validation

Looking at the tools used individually, "Pinnacle 21 Community Edition v2.1.3 (PMDA Adopted 21E 3.0.5 compliant)" was used by the most 39 companies (70%), followed by "Pinnacle 21 Enterprise Edition" by 19 companies (34%) (Figure 12). Pinnacle 21 Enterprise Edition" was selected by 19 (34%) of the respondents ( Figure 12 ). Looking at the combination of tools used, "Pinnacle 21 Community Edition v2.1.3 (PMDA adopted 21E 3.0.5 compliant) and Pinnacle 21 Enterprise Edition" was the most common combination used by 13 companies (23%). On the other hand, 22 companies (39%) answered that they use only "Pinnacle 21 Community Edition v2.1.3 (PMDA adopted 21E 3.0.5 compliant)" as a validation tool, and from the perspective of checking PMDA validation rules This indicates that the Community Edition (free version) can be used to check PMDA validation rules. However, when we also surveyed whether there were any differences between the PMDA validation results and the company's own results, we found that there were some differences caused by violations that could not be detected by the Community Edition. This information may be shared at the "Workshop for Practitioners on Electronic Data Submission for Applications," so it is necessary to check the latest information along with the materials from past workshops.

Fig. 12 Validation tools used (multiple responses allowed, N=56)

Survey Result (7) Other matters related to electronic data submission

The following are the results of the survey on issues in the operation of CDISC (Clinical Data Interchange Standards Consortium). The top five responses were "securing human resources" (42 companies (75%), "impact on application/approval timelines" (40 companies (71%), "management system (metadata governance, version control, etc.)" and "additional costs incurred" (30 companies (54%) and "training of experts" (28 companies (50%) respectively). ); the top five items were the same as in the same surveys conducted in January 2017 and December 2017 (with a change in ranking) ( Figure 13 ). In particular, with regard to "securing human resources" and "additional costs incurred," it is believed that how to build an efficient process for CDISC implementation and electronic data submission will be the key to solving the issues, and it will be important for each company to continue to consider these issues even after the transitional period ends.

Fig. 13 Challenges in CDISC operation (multiple responses allowed, N=56)

Next, we surveyed the benefits perceived from the experience of electronic data submission ( Figure 14 ). The results showed that 35 companies (63%) "reviewed internal processes," 23 companies (41%) "deepened understanding among departments," 2 companies (4%) "changed inquiries," 3 companies (5%) "reduced opportunities for aggregation in inquiries," and 7 companies (13%) "nothing in particular. It is evident that many companies feel that the system is beneficial from the viewpoint of building an internal system.

Fig. 14 Advantages of the experience with electronic data submission (multiple responses allowed, N=56)

The following questions were collected as free answers. First, the survey results for "Requests to PMDA based on experience with electronic data submission" ( Table 1 ). In this section, we report on the most frequently raised comments after classifying the comments into categories.

Table 1 Requests to PMDA based on the experience of electronic data submission

The most common comments by category were those related to unification of requirements with the FDA. Regarding the Legacy data conversion, some commented that they would like to see a reasonable response based on the position of the study, and some commented that they would like to see a "Reject" when conducting Legacy data conversion. Finally, there was a request for "electronic data conversion after the transitional measures period ends.

Finally, regarding "Concerns about electronic data submission after the transitional measures period," the comments received were wide-ranging, and the editor has summarized the comments that were understandable as follows ( Table 2 ). In addition to resource and cost concerns such as "How far legacy data conversion is required" and "Cost of the new consultation category (fee-based)," concerns were raised regarding the impact on quality and review period.

Table 2 Concerns about electronic data submission after the transitional measures period

Conclusion

In order to support smooth implementation of electronic data submission at the time of application for approval, the Data Science Subcommittee has been discussing with PMDA to establish a better system for electronic data submission through the creation of deliverables related to electronic data submission and the SWG (subworking group) for the establishment of a system for the use of electronic data for applications. In the course of these activities, we have also promoted the effective use of the transitional measures period. This survey was conducted to ascertain the status of electronic data submission by each company during the final year of the transitional measures period, and to provide guidelines for future activities of the Pharmaceutical Manufacturers Association of Japan (PMAJ). The survey showed that more than half of the companies have experienced electronic data submission, indicating that many companies have steadily gained experience in data submission throughout the transitional measures period. In addition, some companies said that they benefited from the experience of data submission by "reviewing internal processes" and "promoting understanding among departments. On the other hand, however, it became clear that many companies are still facing issues related to cost, such as "securing human resources" and "additional costs incurred," as well as "approval and submission timelines. There are also persistent calls to resolve the differences in processes and requirements for electronic data submission between PMDA and FDA. Furthermore, each company will need to consider the operation of the "Electronic Data Submission Method Consultation" and "Electronic Data Submission Exemption Consultation," which were newly established in April 2019.

The Pharmaceutical Manufacturers Association of Japan (PMAJ) will continue to produce deliverables and make recommendations to help resolve these issues. In addition, through the SWG for the establishment of a system for the use of electronic application data, we will continue discussions with PMDA and related stakeholders to explore and propose ways to reduce the gap between the requirements of PMDA and FDA. We would also like to consider the continuation of the "Workshop for Practitioners on Electronic Data Submission for Applications," which was also raised in this survey, including the format for holding the workshop.

Acknowledgments

We would like to express our deepest gratitude to the many companies that cooperated in the survey.

( Taku Sakaue, Sho Hibino, and Keiichi Koizumi, Data Science Subcommittee, Drug Evaluation Committee)

Back to Newsletter Top

TOP