The Value of Randomized Pilot Studies in Demonstrating Patient Safety and Efficacy

Introduction
Medical device manufacturers have at their fingertips many types of clinical studies to demonstrate product safety and efficacy, one of which is a randomized, controlled pivotal trial. While there are many routes to initiate these types of trials, sponsors requiring such studies would be wise to first consider conducting a small randomized pilot study prior to embarking on a full pivotal study. This route is especially beneficial if the safety, effectiveness or operation of a device and/or control are not well understood, or if a sponsor is relatively new to the medical device industry.

Background
Though many medical device manufacturers would prefer to avoid administering a randomized pilot study due to cost and time considerations, total resource savings related to the overall medical product clinical evaluation can be substantial.

In fact, a small randomized (masked) multi-center pilot study should be seen as an opportunity to conduct a “dry run” to gain important insights prior to developing and managing a large – and typically expensive and time-consuming – pivotal trial. This process provides sponsors the benefit of identifying and working out the kinks prior to conducting a pivotal trial, thus avoiding mid-course corrections which can be very difficult and time intensive. It also allows sponsors the ability to terminate a pivotal trial if flaws exist that cannot be corrected.

Benefits of Randomized Pilot Studies
The benefits of a well thought out randomized pilot study are many and are well worth the relatively modest effort, time and cost. As stated above, these types of studies provide an excellent opportunity to observe and resolve potential defects in a pivotal trial prior to launch.

Highlighted below are just some of the advantages of conducting a randomized pilot study prior to a pivotal trial:

• Identify whether the intended patient population has been correctly chosen;
• Evaluate the appropriateness of the study control:
  1. Determine how difficult it is to maintain compliance of unmasked subjects during study visits and treatments, and
  2. Gather information on whether, and how quickly, subjects drop out of the control group;
• Observe and evaluate the masking procedure (if the trial is to be masked);
• Identify the most appropriate endpoint(s) for the pivotal trial; and
• Establish rough estimates of the success rates and/or endpoint means and standard deviations of the treatment and control groups so the trial can be more effectively powered or sized.

While randomized pilot studies offer great benefits, it is important to note that due the small size of these trials, a sponsor should not be too quick to abandon a device because of a poor outcome – or, similarly – become overly confident in what appears to be a very positive result in relation to the study control. In the former case, the pilot study results should be scrutinized to see if improvements can be made to the device and/or study design and conduct. In the latter case, the study should be sized to take into account the possibility of overly optimistic pilot study results.

Additional Recommendations
As a former statistical reviewer and team lead at the Food and Drug Administration’s (FDA) Center for Devices and Radiological Health (CDRH), I always encouraged sponsors to conduct a randomized pilot study. In particular, I would recommend a pilot study of at least ten to 30 subjects depending on the device type and complexity, the understanding of the device, the diversity of the patient population, and the potential risk. I also recommended to sponsors that their studies be randomized in a ratio of 2:1 or 3:1 treatment to control subjects so that additional information on treated subjects can be obtained if the control treatment is well understood. (Another option is to randomize a subgroup of the pilot subjects.)

I would also suggest using at least two – preferably three – investigational centers for the pilot study. This provides invaluable insight into center-to-center variability with respect to adverse events, the speed of enrollment of subjects, and the potential to uncover problems that might be specific to different types of centers. This process also allows an increased number of study centers to be pre-trained and ready to launch a pivotal trial. (It should be kept in mind that the choice of investigational centers and the timely enrollment of subjects is a critical factor in the overall cost of a pivotal trial; any additional insight that can be obtained regarding this aspect of a trial during the pilot study can be extremely valuable.)

Conclusion
The clinical phase of a medical device development program can be the most time consuming and expensive and least predictable– yet, most important part – of any development effort. However, given the relatively minor added cost and low time commitment relative to overall clinical development programs, a randomized pilot study can be worth its weight in gold and provide substantial resource savings to medical device manufacturers.

Obtaining the best clinical trial design, conduct and database collection and management requires knowledgeable support staff and product-specific expertise such as that offered by NAMSA. We have experience with thousands of clinical studies throughout the globe and work diligently to meet the objectives of each client – from first-in-man through post-market to obtaining market approval, improve reimbursement, or to generate publications.

If you would like to speak with one of NAMSA’s clinical trial experts to discuss your clinical trial strategy, please contact us at communications@namsa.com, or visit our Clinical Research Services landing page by clicking here.