Advertisement

A Monte Carlo simulation and sensitivity cost benefit analysis for use of nylon 4N6FLOQSwabs®

Published:September 28, 2022DOI:https://doi.org/10.1016/j.fsigss.2022.09.021

      Abstract

      Forensic genetic laboratories are challenged with implementing innovation even if the benefits to operational performance are well demonstrated often because of internal budget constraints. A prospective cost–benefit analysis (CBA) could support justification for an increased budget by effectively demonstrating in a system-based approach the relatively small cost of increasing a laboratory budget can substantially reduce costs to society (both qualitatively and monetarily). A Monte Carlo simulation and sensitivity CBA was performed using a more expensive swab (i.e., nylon 4N6FLOQSwabs®) compared with a less costly cotton swab. Ranges of input values and tangible and intangible benefits were considered. The outcome is that the relatively small increased cost of using a nylon swab pales compared with the potential tangible and intangible benefits to the overall system. This approach provides a sounder basis for requesting additional funds to support implementation of technologies and better approximates realistic situations while accommodating uncertainty of input values.

      Keywords

      1. Introduction

      Advancements in molecular biology and bioinformatics continue to develop and improve the capabilities of forensic laboratories to acquire more data and achieve greater typing success from high quality to trace level samples. These tools support criminal investigations, assist in identifying the missing, and help exonerate the innocent. Although the benefits to the laboratory are obvious, implementation of these technologies is hampered by resource and budgetary constraints. To increase their budgets laboratories (and their criminal justice partners) would benefit from performing a prospective cost-benefit analysis (CBA) to show that the relatively small cost of the laboratory process(es) can have a large impact on tangible and intangible costs and benefits for the overall system (i.e., individuals, society, and government budgets). Budowle et al. [
      • Budowle B.
      • Ge J.
      • Sajantila A.
      A prospective cost-benefit analysis for nylon 4N6FLOQSwabs®: example of the process and potential benefits.
      ] recently showed through a CBA that the cost of replacing lower performing cotton swabs with higher performing nylon 4N6FLOQSwabs® (COPAN Italia, SpA) was a justifiable investment compared with the cost savings associated with a reduction of future sexual assault victims. While the benefits were large, that analysis was based on a static model (i.e., selected single values) and may not have represented the full range of possible outcomes. Furthermore, although there are some data available to consider reasonable input values, the forensic community does not necessarily collect or process information that could reduce uncertainty in some estimates. Therefore, a Monte Carlo simulation and sensitivity CBA was performed on the potential benefit of the use of nylon swabs which allows input of realistic ranges of possible values to obtain a greater range of probabilistic outcomes. With this approach many inputs can be entertained and varied, key drivers can be identified, and the behavior of real-life systems can be approximated better.

      2. Materials and methods

      The CBA builds upon the premise and logic of [
      • Budowle B.
      • Ge J.
      • Sajantila A.
      A prospective cost-benefit analysis for nylon 4N6FLOQSwabs®: example of the process and potential benefits.
      ], i.e., an increase of DNA profile uploads due to obtaining a greater DNA yield with the use of nylon 4N6FLOQSwabs® will increase identification of recidivists and thus reduces future victims of sexual assault. The sources of swab performance, number of sexual assault cases and total cases with biological evidence that are analyzed per year, CODIS database statistics, tangible and intangible costs associated with rape, and additional investigator costs can be found in [
      • Budowle B.
      • Ge J.
      • Sajantila A.
      A prospective cost-benefit analysis for nylon 4N6FLOQSwabs®: example of the process and potential benefits.
      ]. The ranges and base case values of the various inputs that were sampled via simulation are listed in Table 1.
      Table 1Ranges for input values
      Ranges based on empirical data (see [1] for sources) and judgmental data based on authors’ experience.
      .
      InputMinMaxBase Case
      Tangible Cost$10,000$15,000$13,469
      Intangible Cost$150,000$300,000$182,461
      Increase of Uploads with Nylon Swab10 %30 %14 %
      Total Cases
      Sexual Assault Cases are a constant 13.5% of Total Cases.
      333,000499,500333,000
      Upload Ratio35 %45 %37 %
      Hit Rate (for Uploads)46 %56 %50 %
      Extra Hours for Investigation103020
      Police Hourly Pay Rate
      Pay rates derived from Nashville, TN (for Min) and San Francisco, CA (for Max) police salaries.
      $24$71$46
      a Ranges based on empirical data (see
      • Budowle B.
      • Ge J.
      • Sajantila A.
      A prospective cost-benefit analysis for nylon 4N6FLOQSwabs®: example of the process and potential benefits.
      for sources) and judgmental data based on authors’ experience.
      b Sexual Assault Cases are a constant 13.5% of Total Cases.
      c Pay rates derived from Nashville, TN (for Min) and San Francisco, CA (for Max) police salaries.
      The data simulation was performed with 1000 iterations using Analytic Solver® (FrontlineSolvers®) [

      Analytic Solver®, At: 〈https://www.solver.com/analytic-solver-platform〉, Accessed 8 September 2022.

      ]. Summary statistics were compiled and summarized.

      3. Results and conclusions

      A simulation approach to analyze benefits and risks enables better use of extant data, obtains more realistic outcomes, and provides better conveyance of the uncertainty associated with the input data. This CBA differs from the original CBA performed by Budowle et al. [
      • Budowle B.
      • Ge J.
      • Sajantila A.
      A prospective cost-benefit analysis for nylon 4N6FLOQSwabs®: example of the process and potential benefits.
      ] in that it entertains ranges of values instead of using discrete, deterministic values; considers multiple inputs at one time; stochastically samples across the ranges; and provides probabilistic outcomes. Additionally, based on the premise that as more DNA profiles populate a database the hit rate will increase, the percent database hit rate was allowed to increase with the upload rate instead of using a single, static value. In the initial CBA [
      • Budowle B.
      • Ge J.
      • Sajantila A.
      A prospective cost-benefit analysis for nylon 4N6FLOQSwabs®: example of the process and potential benefits.
      ] the authors relied on CODIS hits to serial sex offenders see [

      Sexual Assault Kit Initiative (SAKI), At: 〈https://www.sakitta.org/metrics/〉, Accessed 8 September 2022.

      ] to determine the possible number of victims that could be reduced by identifying recidivist offenders due to more DNA profiles uploaded and obtaining more hits. A larger category - CODIS hits to serial violent offenders – which is almost four times larger than the number of serial sex offenders identified through SAKI was not included previously. The serial violent offenders were included in the CBA herein as they also are recidivists that can be identified as a person(s) of interest through a database search. Lastly, investigator labor costs associated with sexual assault cases that have a database hit to an offender or arrestee were added. It was believed that identifying a person of interest through a DNA database hit would reduce the effort of a police investigation which in turn would save labor costs and allow police to focus on other cases that do not have the benefit of a DNA profile association. Retrospective analyses have indicated the opposite; investigators tend to spend almost twice as many hours on a database hit case than a no-hit case (see [
      • Budowle B.
      • Ge J.
      • Sajantila A.
      A prospective cost-benefit analysis for nylon 4N6FLOQSwabs®: example of the process and potential benefits.
      ] for information source). At first glance this observation may seem counter intuitive, but the strong lead information may better motivate police to focus on the particular case investigation. Therefore, the additional investigative cost associated with a DNA hit case was included in this CBA.
      The increased cost of replacing cotton swabs with nylon swabs is $1,798,200 (based on 333,000 cases and five swabs per case). As shown in Fig. 1 it only takes on average ∼8 victims (SD = 3.84) to recoup the cost of the swabs considering tangible and intangible costs within a systems approach. Obviously, far more individuals would not become victims if samples in all sexual assault cases were collected with a more effective swab. Based on the minimum and maximum upload increases listed in Table 1, one would predict a minimum of ∼340 victims would be reduced with intangible and tangible cost savings of $76,459,764 and $4,247,743, respectively, and a maximum of ∼1562 victims reduced with intangible and tangible cost savings of $351,481,652 and $19,526,568, respectively. Even with investigator labor costs, the tangible and intangible benefits are much larger than those reported previously [
      • Budowle B.
      • Ge J.
      • Sajantila A.
      A prospective cost-benefit analysis for nylon 4N6FLOQSwabs®: example of the process and potential benefits.
      ], primarily because of the inclusion of serial violent offenders associated with sexual assault identified through DNA database searches.
      Fig. 1
      Fig. 1Distribution for number of prevented sexual assault victims to “break even” with investment cost of using nylon 4N6FLOQSwabs®.
      This simulation and sensitivity analysis is superior to a discretized analysis in that it determines various values of independent variables and how they affect dependent variables as well as may show the degree of effects that uncertainties may have on predicted outcomes. Overall, the conclusions are the same as in [
      • Budowle B.
      • Ge J.
      • Sajantila A.
      A prospective cost-benefit analysis for nylon 4N6FLOQSwabs®: example of the process and potential benefits.
      ], and the cost of use of nylon swabs pales compared with the benefits to victims, society, and government budgets. Laboratories should consider prospective CBAs to justify an increase in resources to implement better technologies, such as massively parallel sequencing and new software solutions, or even just to increase throughput, ranging from swabs, new instrumentation, kits and software.

      Conflict of Interest Statement

      The authors declare that they have no conflict of interest with this work.

      References

        • Budowle B.
        • Ge J.
        • Sajantila A.
        A prospective cost-benefit analysis for nylon 4N6FLOQSwabs®: example of the process and potential benefits.
        Int. J. Leg. Med. 2022;
      1. Analytic Solver®, At: 〈https://www.solver.com/analytic-solver-platform〉, Accessed 8 September 2022.

      2. Sexual Assault Kit Initiative (SAKI), At: 〈https://www.sakitta.org/metrics/〉, Accessed 8 September 2022.