Gatekeeping Out of the Box

1. It is ironic that scientists studying the voting problems raised in the Florida polls say that the old ways of paper ballots and lever machines give more accurate accounts than punch cards or electronic devices.[1] A study analyzing those problems says that part of the difficulty may lie in voters’ lack of familiarity with new technologies. This article addresses a similar issue at the crossroads of law, technology, and science pertaining to standards, reliability, and evidentiary thresholds of proof: digital evidence and the software from which it is generated. Whether or not a vote gets counted or a piece of digital evidence is admitted depends on the standards that are applied to the respective processes. Ultimately it is not the technology itself, but rather the human understanding of its function and capabilities, that paints the picture of truth.

2. Just as transparency in the voting scheme and certification steeped in authority go far in resolving allegations of computer glitches, human error, or mischief at the polls, Open Source software may provide a basis for adjudging the reliability of computer experts and programs that promise to be principal sources of evidence in resolving disputes within our technology-dependent society.

3. Likewise, just as the focus of poll reform is to establish standards that will prevent the reoccurrence of the recent assault on the process, Open Source software can help anticipate legal challenges that will confront digital evidence, and enable a methodology to assess the credibility of attacks against it.

4. Digital evidence has yet to attain widespread smoking gun status. When used to prove claims, it is often a piece or two in the overall puzzle, and questions of its reliability can be quashed by assembling multiple streams of corroborating evidence. Indeed, the implementation of software reliability in industry is dubious at best, and even less clear in the context of the courtroom, where few decisions have squarely addressed more than a cursory evaluation of software reliability. Whether this dearth of judgments can be attributed in part to lack of challenges to the technology that produces digital evidence remains unclear, as there have not been many cases on point to establish precedent. The ubiquity of computer technology that permeates modern society, however, promises to make computer-derived evidence a digital eyewitness. Its reliability must , therefore, be scrutinized in accordance with legal protections.

5. This article examines digital evidence reliability by first identifying and differentiating the two competing categories of software from which this evidence is derived: proprietary and Open Source. The next section explores the standards for software reliability in both the industrial marketplace and the legal arena. Specifically, the current standards are addressed in light of their value to industry and the law, as well as their respective historical origins This sets the stage for a reconciliation of standards for reliability as between industry and the courtroom. An outline of the legal approaches to reconciling digital evidence standards and the ensuing dangers of failing to scrutinize the source of the evidence supports the conclusion that the reliability of some digital evidence is not being properly addressed. Finally, this article will advocate the merits of Open Source software as a solution that facilitates the application of appropriate legal standards to novel evidence and helps bridge the gap between the law and industry in measuring reliability.

II. A Software Primer
A. Software Reliability – An Introduction to the Issue

6. Why should the reliability of software be a concern to industry and the law collectively? There is a growing tension between the need to present probative and visual evidence of digital disputes and the legal standards for the admissibility of scientific and technical evidence.[2] In other words, the ubiquity of computer technology and pervasiveness of data derived therefrom bear a direct relation to software. Since software provides the functional link between man and machine, unreliable software is sure to have an effect on any activity within the realm of everyday communications, transportation, and even survival within a tightly coupled, high-tech society.[3] The industry that both spawns and is aggrieved by symptoms of unreliable software also drives this issue into the legal arena, where the resolution of many disputes ought to be incumbent upon the reliability of the digital evidence derived from such software. Software reliability, therefore, has immediate and profound meaning in both industry and the courtroom.

7. The mention of digital evidence conjures up thoughts of crime scenes, computer cops, and monomaniacal digital bandits – the stuff of “Mission Impossible,” handled by the likes of “techies” named Morpheus, and far removed from the concern of Jane Q. Public. In reality, however, digital evidence is not just a byproduct of computer hacker incidents, but is present in virtually any dispute where there exist data relevant to a civil claim or crime. Like other forms of evidence – hair, blood, eyewitness accounts, or paper documents – its form, prevalence, and existence help clarify competing stories about the reenactment of an event. Furthermore, the reliability of both the physical and digital evidence can be scrutinized by examining the forensic science involved in its identification, collection, preservation, and analysis.

8. Unlike other forms of evidence where the source is not called into question, however, the reliability of digital evidence hinges on the source being somewhat reliable. That source is the software that generates, processes, and stores data. With blood evidence, for example, we do not question whether Jack Doe’s body reliably produces the DNA found in the blood. Either we can DNA-type the blood, or we cannot, and unreliable forensics will not change Jack’s DNA into Jill's DNA. With digital evidence, on the other hand, not only will unreliable forensics change the identifying, correlative, and corroborative properties of such evidence, but unreliable software is also capable of maligning the very data upon which a dispute is based or resolved, regardless of exemplary forensic practices.

9. Thus, the reliability of software within industry has sobering ramifications for a legal system that requires similar assurances from its evidentiary offers of proof. If industry cannot rely on or has no way of determining whether a piece of software does only what it purports to do, how can courts settle conflicting accounts arising from or backed by this software? One logical conclusion is that the challenge of judging fact from falsehood in litigating disputes will take on the characteristics of an historical fiction novel, albeit one in which discerning truth would be more a function of the attorney’s or author’s persuasive style than of reference to irrefutable historical records.

10. Regardless of whether reliable software has a profit margin, the omnipresent evidence created by software, the relevance of this digital data in virtually every type of legal dispute, and the legal principles governing the admissibility of evidence are subtly affecting industry’s need to develop reliability software. How this can be actualized in an industry where software has been and continues to be developed with reliability as an afterthought is where the Open Source model of software development may be useful. This is because the propagation of Open Source software in industry is fueled by the desire for reliable software, which is measured by standards and principles similar to those courts use to determine evidentiary reliability - empirical testing, subjection to peer review and publication, determination of error rate, and general acceptance within the relevant community.

B. Distinguishing Open Source Versus Proprietary Software

11. It is important at the outset to identify and differentiate between the two competing categories of software from which digital evidence is derived: proprietary and Open Source. This process is significant because judicial recognition that any “computer-generated” evidence be a product of a “standard” computer program or system in order to gain admission has migrated to issues surrounding software.[4] Furthermore, the distinction is crucial to understanding the nature of the issues involving the reliability of digital evidence, as well as to appreciating the solution offered herein.

12. “Standard” computer technology, so far as software is concerned, inexorably refers to COTS (Commercial Off-The-Shelf) products, which are publicly available, ready-made software that can be purchased from a manufacturer.[5] Examples include Microsoft Office, Network Solutions' Check Point RealSecure IDS, Lotus Notes, MacIntosh Operating System, Computer Associates' Inoculate IT, and Norton Anti-virus. The pertinent characteristic of this “standard” software is that it is proprietary, meaning that its underlying source code is not freely available to be viewed or changed.

13. In contrast, the features characterizing “nonstandard” software – customizable, able to be manipulated, not necessarily commercially marketed – typify Open Source software. Open access to a computer program's source code is the central defining point for Open Source software, as well as what primarily distinguishes it from proprietary programs. Some prominent examples include RedHat Linux, BIND, and Apache web server software. [6] Open Source is used to describe the conditions under which software source code used by computers is made available to others apart from the developer.[7] Reliability is a significant motivation behind the Open Source movement. This is effectuated by making the code available over the Internet for extensive testing and widespread review to find faults, as well as to catalog responsive code changes, and to maintain concurrent quality control. The software code or some subset can be integrated, packaged, branded, and sold by a developer to customers.[8]

14. Proprietary software is often referred to as a “black box” because, without access to the source code, one can only conjecture as to what happens between the data input and data output stages (except, of course, for the programmer and manufacturer). Without access to these “blueprints ” a computer professional is left to infer, based on his knowledge and experience, the causes of and solutions to software problems. Open source software demystifies many of these problems by exposing the source code to the public arena. In this way, those with programming skills can “see” what the software is really doing, diagnose problems, and substantiate predicted results of future occurrences.

15. The exercise of distinguishing between proprietary and Open Source software for the purposes of evidentiary implications is not intended to paint the former as “bad” and the latter as “good.” Nor does the proposed judicial utility of Open Source software preclude economic entitlement, intellectual property protection or technological advancement.[9] Rather, it is intended to stimulate critical analysis of how the legal challenges confronting digital evidence can be overcome by embracing new concepts that comport with evidentiary jurisprudence.

III. Reliability Standards for Software

A. The Importance of Standards to the Law and Industry

16. Why is it judicially rational to recognize Open Source software as a mechanism to foster reliability standards for digital evidence? To begin, a standard is any set of conditions that describes a desired or ideal state of something and that can be used to describe or evaluate actual examples of this thing.[10] In the context of software and evidence, standards are one response to the difficulties of evaluating reliability and of mitigating the troubles that accompany imperfect information.[11] Our legal system is guided by notions of reasonableness and judged by objective standards representing society’s values. Yet, despite the fact that software is the interface between humans and the machines that permeate modern life, there is no agreement on “specific” processes or standards regarding the reliability of software systems.[12]

17. To be sure, there are numerous technical standards being developed for the world wide web, e-commerce and the Internet that have addressed reliability indirectly by focusing on security.[13] History, however, suggests that whenever a major technology or industry has proliferated sufficiently to affect society at large, some measure of social control has followed.[14] This trend has resulted in public health, safety and environmental needs being addressed by laws that incorporate industry-born technical standards by encouraging reliance on them or by adopting those standards by reference.[15] Nevertheless, there is no software Underwriters Laboratory to denote third party certification of reliability; nor is there a software equivalent to the National Electrical Code to provide a common framework from which reliability determinations can be spawned and infused into local codes. Perhaps the closest attempt to duplicate such standards in the software industry has been declared “dead.”[16]

18. To illustrate this point, one relevant study found 250 different standards applying to the engineering of software, yet observed that these were essentially ineffective, and concluded that software technology was too immature to standardize.[17] Regardless of whether this reasoning holds true at present, the fact remains that software continues to be developed within an alphabet soup of standards. Thus far, many standards that do exist are aimed at promoting interoperability. For example, the well-established “syslog” protocol that allows machines to send event notification messages across Internet Protocol (IP) networks to a central server has become the de facto standard for logging system events.[18] The scalability and flexibility that allow for interoperability between various applications and operating systems, however, are often achieved by sacrificing reliability.[19]

19. The legal implications of this discordance of standards in industry are far-reaching. On one level, the utility derived from having standards is comparable between the judicial and industrial arenas. Standards help simplify the decision-making process for determining reliability. In the context of determining the admissibility of evidence, they act as a measuring stick for judges and juries; whereas in industry, they influence the sale of goods for those who are shopping for reliability. Further, standards assuage decision-making for producers of reliability–whether they be developers and vendors in industry, or computer forensics professionals–by narrowing choices.[20]

20. Related to this decision-making streamlining is the added benefit of third party validation, which is aimed at providing objectivity and independent assurances. Although the software industry is riddled with authoritative groups like the federal government and independent standards-setting organizations (e.g., IETF, NIST, IISP, ANSI, FIPS[21]), the lack of consensus among this amalgamation has not eased decision-making regarding software reliability. Translated into the courtroom, the law finds no clear authority with which to resolve the issue of “circumstantial guarantees of trustworthiness” of evidence derived from such software.

21. Another benefit of accruing standards of reliability for software is grounded in self-perpetuation. The existence of standards invites scrutiny that serves as a check on flaws and accentuates unreliable factors. The underlying rationale is that reliability is enhanced as an end result.[22]

22. Most significantly, standards bear directly on the interplay between industry and the legal system. Insofar as the existence of and adherence to software standards can provide protection from legal liability and facilitate claim settlements, the lack of standards can produce the opposite effect. For instance, a legal system bombarded by and ill-equipped to handle issues involving the disputed validity of software-derived evidence (including creation, identification, collection, preservation, and analysis) could result in the admission of evidence the foundation for which is unreliable; the exclusion of digital data that is vital to a claim at hand; the necessity for government regulation to provide clarity; and/or a disincentive to litigate disputes due to tremendous case backlog or, in the best case scenario, escalating pay-to-prove expenses.

1. Industry Standards for Reliable Software

23. In order to grasp how the Open Source model can facilitate reliability determinations of digital data within legal standards, it is necessary to understand how industry values and measures software reliability under proprietary models. To begin, software’s historical origins, both economic and technical, gave rise to the current disconnect between proprietary software and reliability.

a) History and Origin

24. Drawing on the metaphor that organisms are a product of their environment,[23] software[24] reflects the values of its originating environment and the motivations driving its current distribution. Specifically, reliability was not a consideration shaping early software development in the personal computer (PC) environment. This era was characterized by isolated desktops wherein software failures had no way of propagating to other machines, and data had not yet attained its current lifeblood status. Also, errors and malfunctions were an accepted part of the organizational and programming culture, where notions of computer security and intrusions were embryonic. This acceptance was illustrated by wholesale abdication of responsibility by software developers, represented by shrinkwrap licensing.[25]

25. In addition, the economic breeding ground for software was not conducive for dissatisfied customers to leverage financial influence upon vendors and developers. This climate resulted, in part, because software could be bought separate from the computer, unlike with its mainframe predecessors. Further, when personal computer (PC) popularity ignited, market share became the key to corporate success and personal financial gain. Market share bore a direct relation to market entry and increased features. Thus, reliability of software became the bastard stepchild to the motivations shaping market share – reduced time spent on testing, and features du jour.[26]

b) Software Reliability Standards at Present

26. Against this backdrop of software being developed largely irrespective of reliability, we arrive at the current divergence of standards and dearth of incentives to invest in reliable software. On one hand, lack of accountability in the software industry is a disincentive to producing reliable software. Imagine a society where doctors could use newfangled procedures to treat patients, yet were unencumbered by malpractice liability if someone lost the use of a kidney or reared a child with only one ear. In fact, the computer industry is no stranger to instances of software exposing personal banking records, causing the demise of entire networks, and even facilitating physical harm – oftentimes without matching repercussions upon the developers.[27] Society has become desensitized to the panoply of deficiencies in some software such that customers have become unwitting crash test dummies in the product development cycle.[28]

27. In a sense, reliability can be inferred when a party is held responsible if someone/something is not what it claims to be. A doctor’s professional vows and integrity, backed by insurance and certifying authorities, provide some objective criteria for gauging reliability in the medical arena. Ultimately, litigation ensures that these measurements are not toothless. This is in stark contrast to the software industry, where developers are motivated by market pressures that traditionally have neither embraced reliability nor penalized unreliability.[29]

28. Indeed, the market does not reward reliable software, at least as far as the shortsighted definition of reliability paints it as an impediment to improved functionality, features, and speed. The costs of improving reliability boil down to the production costs of integration and testing.[30] Apparently a piece of software with quick time-to-market, a fancy user interface, and code reminiscent of Swiss cheese is more valuable than one that has undergone significantly more quality control testing to ensure that the hack-of-the-week does not shut down a system.

29. This distills another disincentive to invest in reliable software, one rooted in the lack of a widely recognized and measurable definition of reliability. Despite the fact that software customers (read: everyone) are steered by concerns for functionality, consumer choices are at some level based on reliability. For instance, Jack Consumer generally seeks products that are sold widely or by a familiar vendor. In this way there is a propagation of this perception of reliability. This purchasing and production of reliability, however, is occurring within a context of imperfect information.[31] The lack of both agreed upon standards to test reliability and a recognized body to conduct assessments has led to a distortion in the mass consumer market of what reliability means. This deficiency has provided fertile ground for the proliferation of software that addresses well-known, visible, and publicized problems based on maligned information.

30. Apart from the social misconception of reliable software, what does software reliability mean to computer industry professionals? Accepting that there are various ways to describe these attributes, reliable software can be characterized by most, if not all, of the following attributes: authenticity (that the data came from or is what it purports to be, and that the software performs as described); integrity (that the data is accurate); availability (that the data is accessible, and that the software does not fail or cause other components to fail); and supportability (that the software has dependable, strong, available support--especially in a production environment).[32] This high-level definition is important in assessing how industry criteria square with legal standards for the reliability of evidence, and ultimately, how the Open Source model may facilitate proper analysis of evidence derived from software.

2. Legal Standards for Reliability

31. A basic legal tenet governing the admission of evidence is that it must be relevant, competent, and material to the case at hand. For evidence that is scientific, technical or of a specialized nature, the Federal Rules of Evidence and case law provide standards used by trial courts to determine if such evidence gets admitted.

32. The foremost case establishing guidelines for evidentiary reliability was Daubert v. Merrell Dow Pharmaceuticals in 1993.[33] Daubert involved challenges to the admission of scientific evidence, and aimed to bring clarity to the reliability requirements enunciated in the Federal Rules of Evidence.[34] The guidelines established in Daubert require trial courts to consider the following factors in their role as gatekeepers of admissibility:

· Has the scientific theory or technique been empirically tested; or, is it falsifiable?

· Has the theory or technique been subjected to peer review and publication?

· What is the known or potential error rate?

· Is the theory or technique generally accepted within the relevant scientific community?[35]

33. These criteria are the Court’s attempt to meet the “standard of evidentiary reliability” by ensuring that technical evidence is grounded in knowledge derived from the methods and procedures of science. By tying the validity of the knowledge to the underlying scientific methodology, the Court defines reliability as something that can be validated by testing and supported by more than subjective beliefs or unsupported speculation.[36]

34. Kumho Tire v. Carmichael extended the Daubert guidelines to nonscientific evidence. It gives trial judges the discretion, based on the facts of the case, to determine which Daubert criteria should be applied to determine reliability, as well as whether those criteria are satisfied.[37] The Court interpreted the reliability requirement of Federal Rule of Evidence 702 to apply to the word “knowledge,” not to “scientific, technical or other specialized.”[38] In other words, expert testimony can be based on no scientific knowledge in a particular field in order to meet the standard of evidentiary reliability.

35. At first glance, this ruling appears valuable to elucidating standards of reliability for nonscientific evidence. The Court assumes, however, that the four factors devised to measure the reliability of scientific evidence can be applied just as effectively to evaluate technical or specialized evidence. That is to say, the use of scientific methodology to gauge the reliability of scientific knowledge[39] (validation through testing) is the same criterion courts may use to adjudge, for instance, reliability of technical knowledge. If knowledge is based on experience or subjective interpretations that are not susceptible to validation through testing, those factors do not provide assistance in evaluating the reliability of that knowledge.[40]

36. Indeed, trial judges are not required to use all of the Daubert factors, but may consider them based on the circumstances of a particular case.[41] This flexibility is no less troublesome since it purportedly brings the issue full circle: trial judges are tasked with gatekeeping the reliability of nonscientific expert testimony, but are left alone to choose which keys will open the lock. This has set the stage for continuing controversy and lack of consistency among jurisdictions in developing and applying standards for admitting nonscientific evidence.[42]

37. A judge’s handling of novel scientific evidence, for example, is made easier when the underlying knowledge can be readily mapped against the Daubert criteria, as is often the case with knowledge rooted in academic or laboratory settings. Whereas, technical knowledge that may accompany the introduction of some forms of digital evidence, generally represents complex and uncharted territory for judges. Coupled with the fact that the claimed expertise may have evolved from an experience-based body of knowledge (i.e. computer forensics, intrusion detection, computer security), the discretion to allow such technical evidence to go before a jury is less predictable.

B. Reconciling Industry and Legal Standards for Reliability

38. How does the admissibility of software-derived evidence square with (1) the lack of industry standards for software reliability coupled with (2) unclear legal standards for the reliability of computer-derived evidence?

1. Presumption of Reliability Afforded by Courts?

39. It is not an overstatement to say that digital evidence may carry an aura of infallibility in the public’s eyes, a fact that may facilitate settlements and discourage technical challenges during litigation. Computer technology is afforded a presumption of reliability because there is a common belief that machines are immune to human frailties, desires, and whims that can lead to erroneous information or misinterpretation.[43] The dangers of placing such unbridled faith in automated systems without appropriate checks, however, are a no less serious matter, as illustrated both within and outside legal contexts.[44]

40. Just as the degree of proof necessary to support a chain of custody depends on the nature of the proffered evidence, the mutability of digital evidence makes its digital pathways important in establishing reliability.[45] This approach contrasts with that used to analyze a 19th Century ruby-crested urn that has visibly unique characteristics that are resistant to change, thus making such strict determinations of reliability unnecessary.[46] Similarly, the strictness with which the reliability requirement is applied to software-derived evidence should rest on the importance of the evidence and the extent to which its probative value depends on its accurate and unchanged condition.

41. Indeed, the familiar and neat package in which software can display data and the complexities of examining what happens to digital evidence from creation through end product mean that a more watchful eye and steady gavel are needed. The myriad of possibilities contributing to an undetected error in computer-derived evidence includes: errors introduced at one or more of several processing stages; software programmed with errors, programmed to permit errors to go undetected, or programmed to introduce errors into the data; or data rendered inaccurate or biased.[47]

2. Judicial Approaches to Reconciling Reliability of Digital Evidence

42. In order to answer the aforementioned question of squaring admissibility with reliability, an assessment of the threshold of scrutiny for computer-derived evidence is needed. The majority of relevant evidentiary challenges have addressed: (a) the authenticity of the evidence; (b) hearsay rule violations and satisfaction of the Business Records Exception; and, (c) the propriety of admitting computer-derived evidence as demonstrative rather than substantive evidence. This is the framework within which judicial discretion has been exercised to control the admission of computer-derived evidence and establish the appropriate standard for evaluating computer-derived evidence.

Whether or not these thresholds are appropriate for determining the reliability of the underlying software is an important question to which officers of the court should demand a clear answer. Whereas courts are generally aware of the reliability concerns attendant on digital evidence, there have been no reported decisions that squarely address Daubert challenges to proprietary commercial software. Reliability determinations have thus far been made indirectly by inquiring, most notably, about the mechanical operation of computer hardware, the accuracy of the human involvement in the data handled by the computer, the commercial availability and use of the program, and the proponent’s familiarity with the software. The standards mandated by Daubert, Kumho, and Federal Rule of Evidence 702, however, demand inquiry into the validity of the scientific theory behind both the specific software technique and the application of those techniques encoded in the software. A more direct and less circumstantial satisfaction of proof calls for analysis of the source code, which is the direct blueprint for a particular piece of software. The proprietary nature of commercial software, however, impedes application of this standard by spurning access to the source code. This has impeded courts from pursuing the easy question, “How reliable is the source code,” and facilitated lower thresholds of scrutiny that seek to prove reliability in circuitous ways.

a) Authenticity Analysis

44. Evidence can be admitted by a judge upon the showing of a proper foundation that often entails the establishment of its authenticity and reliability. These preliminary determinations can occur under the auspices of Federal Rule of Evidence 901’s requirement that the matter in question is what it is claimed to be, or via the more demanding showing of reliability addressed in F.R.E. 702. The purpose of this initial screening is to ensure that there is a modicum of reliability upon which a jury can then decide what weight that evidence should carry in resolving the issue at hand. The degree of scrutiny applied to determine whether or not computer-derived evidence goes to a jury is unsettled.

45. To date, computer-derived data have gained admission upon a foundational showing that the computer process or system produces accurate results when used and operated properly and that it was so employed when the evidence was generated. Federal Rule of Evidence 901 affords a presumption of authenticity to evidence such as x-rays, photographs, tape recordings, computer-generated records or scientific surveys produced by an automated process that is shown to render accurate results.[48] This presumption of reliability has been commonly extended to software performing data storage, collection or retrieval functions.[49] Consequently, a majority of the cases considering the admissibility of such evidence have done so in the context of computerized business records that are maintained or prepared by electronic computing equipment.[50]

46. A Texas appellate court, for instance, determined that the computer system that produced and displayed company payroll information, and upon which the expert’s testimony was based, was sufficiently authenticated under a state equivalent to F.R.E. 901(b)(9) and did not require an evidentiary hearing.[51] This decision was based on the prosecution’s introduction of evidence that the IBM System 38 computer and its programs were in proper working order when they produced the display, that they had done so in the past, and that the technology behind computer-generated displays was trustworthy, reliable, and standard to the computer industry. Further, the systems administrator testified about his years of experience in performing monthly tests on the payroll data system, and stated that the system and its monitoring programs were reliable.

47. The irony of this holding underscores the significance of examining the reliability of the software producing the digital evidence. The Defendant was being charged under a violation of the state penal code for harmful access to computer systems. It was alleged that he maliciously modified the source code to his employer’s network software to delete a series of computer files bearing critical company information. The defense challenged the reliability of the computer-generated display, which formed the basis of the systems administrator’s conclusion that the malicious code caused the deletion of about 160,000 records from payroll. By overruling the objection to the payroll display evidence, while simultaneously upholding Defendant’s conviction for manipulating the software that generated such evidence, the Court failed to consider that the software manipulated to produce aberrant results was the same software that was trusted to produce reliable evidence.

48. Authentication standards are meant to ensure that the evidence is what it purports to be, and how rigorous a foundation is needed to make this finding depends on the existence of something that can be tested.[52] This is primarily accomplished through the testimony of a witness who can speak to the identity and accuracy of the computer-derived evidence. The rationale is that the availability of a witness who can be cross-examined about the actual event and its link to the digital exhibit is a sufficient guarantee of authenticity.[53] In the context of photographs, for instance, a witness familiar with the picture need only attest that it is a “fair and accurate portrayal” of the scene.[54] Computer-derived evidence has been extended a similar presumption of authenticity by some courts, as long as a computer operator, who is familiar with the process undertaken by the software, testifies.[55]

49. The threshold for authenticating computer-derived evidence, however, is ambiguous. Some recommend a higher standard than that applied to photographs,[56] whereas others advocate giving judicial notice of the authenticity of computer-derived evidence under F.R.E. 901(b)(9), which governs authentication of evidence describing a process or system. One of the foremost cases in defining a standard demanded that the proponent of the evidence show the competency of the computer operator; the type of computer used and acceptance in the field as standard and efficient equipment; the procedure for input and output of information, including controls, tests and checks for accuracy and reliability; the mechanical operation of the machine; and the meaning of the records themselves.[57] This rigorous authentication requirement for computer-generated evidence, however, has been eschewed by most courts.[58]

b. The Hearsay Rule and the Business Record Exception

(i) An Analysis

50. Another standard to which courts have subjected computer-derived evidence is the evidentiary prohibition against hearsay.[59] It is generally accepted that computer programs that contain out-of-court statements by declarants (computer operators, programmers, data entry personnel) and that are offered to prove the truth of the matter asserted violate the hearsay rule.[60] Nonetheless, federal courts have applied the business records exception (F.R.E. 803(6)) to a wide variety of computer-based information and there is an abundance of case law allowing the introduction of computer-based records under this exception.[61]

51. Alternatively, proponents of computer-derived evidence have bypassed hearsay exception hurdles by convincing the court that such evidence constitutes a product of a device performing pre-programmed tasks on admissible data input, as with a radar gun or a calculator.[62] Computerized printouts of phone traces, for example, were not hearsay in one case because they did not rely on the assistance, observations, or reports of a human declarant; the report of phone traces was contemporaneous with the placement of the calls; and the printouts were “merely the tangible result of the computer's internal operations.”[63]

52. As with authentication and F.R.E. 702 standards, the depth of inquiry and threshold of proof needed to establish computer-derived evidence as a business record are not always clear. One of the earlier cases to address this issue, for example, held computer records inadmissible as business records because of an insufficient foundation. The testimony of a record keeper for the telephone company was insufficient to establish a proper foundation of “trouble recorder cards” at issue because no complete and comprehensive explanation of either their method of preparation or their meaning was provided. This was despite the facts that the witness testified to having direct supervision and control of all the company’s records, and that the cards were business records made in the ordinary course of business at or about the times and dates indicated on the cards.[64]

(ii) The Problem with Hearsay and the Business Record Exception as Applied to Digital Evidence

53. A basic evidentiary tenet governing admissibility determinations is that there be circumstantial guarantees of trustworthiness to ensure a modicum of reliability so that a jury is not unduly confused or prejudiced by a given piece of evidence. Even though hearsay is generally not allowed because it violates this tenet, factors such as routine reliance on records kept within the normal course of business, lack of motives to fabricate those records, and the non-adversarial nature within which they were created converge to create circumstantial guarantees of trustworthiness. Paper-based business records are occasionally found to be inadmissible if the source of information or the method or circumstances of preparation indicate a lack of trustworthiness.[65]

54. Computers may produce different results based on different assumptions of programmers, however, which can only be directly determined by looking at the source code. In other words, these traditional circumstantial guarantees of trustworthiness – that a document or record was made within the normal course of business, at or near the time of the transaction, by someone familiar with the program, etc. – may be irrelevant if the software producing the data is untrustworthy. If the software source code is riddled with bugs[66] vulnerable to exploitation, and/or prone to errors that go undetected by persons relying on that data yet affect the substance of the data deemed to be a business record, the presumption of reliability is faulty. If courts relegate reliability attacks to the coliseum of jurors as an issue affecting the weight of the evidence, is F.R.E. 803(6) being violated, or has the court exposed the factfinder to unfair prejudice or misleading information in violation of F.R.E. 403?

55. Even if business records produced by unreliable software are found to satisfy F.R.E. 803(6), one can envision a scenario where F.R.E. 803(7) is used to force the reliability issue.[67] For example, what if Plaintiff accuses Defendant of intruding into his computer system and stealing data. Further, suppose that as part of Plaintiff's computer security policy, Plaintiff maintains and relies on Intrusion Detection System (IDS) software that produces logs from which Plaintiff monitors unauthorized and suspicious activity. Assume that these IDS logs are found to meet the business records exception, yet they do not contain data to support the allegation against Defendant. Can Defendant use the fact that no entries exist in the logs to counter Plaintiff’s intrusion claim? In other words, does the absence of a log entry indicate that an intrusion did not occur, or does it merely indicate that the IDS software did not capture the relevant data packets that would have revealed an intrusion?

56. Furthermore, in accordance with F.R.E. 803(6), a party against whom the computerized business records are offered must be given the same opportunity to determine its accuracy as is afforded for written business records.[68] Whereas an opponent of a written business record is entitled to inquire about company record-keeping policies and view ledger entries for accuracy, how might that be accomplished when dealing with business records that are produced by software? As with written business records, attempts to alter or delete data can be discerned for the most part, and there is no need to question whether the pen used to mark data entries reliably transferred the information. Computer data manipulations (whether intentional or accidental), however, are easily accomplished without an indication otherwise, and the software creates another dimension between the data input and the digital data compilation sought to be admitted as a business record. This scenario is where examination of the software source code may offer an opponent the chance to determine the record's accuracy if circumstances indicate a lack of trustworthiness.

57. Therefore, F.R.E. 803(6)'s ending provision, “…unless the sources of information or other circumstances indicate lack of trustworthiness,” although perhaps overshadowed by courts’ concern for the other explicit requirements in F.R.E. 803(6), is a reason for divergent rulings on this evidence as well as the evidentiary trump card that harkens a more satisfactory proof of the reliability of computer-derived evidence.

58. In addition, it is dangerous to immunize certain computer records from the hearsay rule by likening them to the product of a mechanical process that cannot produce hearsay. It would be persuasive to argue that computer logs, for example, are merely the “tangible result of the computer’s internal operations” that do not rely on human observations or reports, and are made contemporaneously with the capturing of data.[69] Unlike phone trace records and calculators, however, the software producing the logs is programmed to capture and process data deemed to be relevant to its programmed function from many computers over a network. Questions about how complete the data capture is and how the logging software decides what should be captured and processed can only be done by examining the underlying source. To admit such evidence without uncovering the assumptions that underlie its function would invite the resolution of claims based on less than a modicum of reliable evidence.

c) Digital Evidence as Demonstrative Analysis

59. One final standard used to govern the admission of computer-derived evidence concerns its admission as demonstrative evidence to explain the testimony of a witness.[70] This threshold is wholly deferential to the exercise of judicial discretion, as the standard for review is abuse of discretion. Characterizing evidence as demonstrative can be a strategic attempt to avoid the reliability analysis imposed on substantive evidence so that the evidence can reach a jury. It avoids hearsay attacks and authenticity proofs (because it is not offered for its truth), and dissection under Daubert (because it lacks independent probative value and is only offered to illustrate the testimony of an expert).[71]

60. It is important to determine whether the software is taking the place of a witness or providing the basis for a witness’s testimony, or merely providing a visual portrayal of a witness’s verbal testimony that is subject to cross-examination. As with the former situation, admission under the demonstrative evidence standard ignores the need to test assumptions that may exist in the underlying source code and immunizes the software-derived evidence from reliability determinations. In these cases, judges and factfinders encounter an interface dilemma: computer evidence that is presented in a friendly, GUI-fied (graphical user interface) form can hide any number of encoding errors yet have a subliminal impact on issue resolution equivalent to substantive evidence.

61. This issue was tangentially addressed in Perma Research v. Singer, the seminal case dealing with legal treatment of simulations.[72] Here the Second Circuit overruled the District Court’s ruling denying the defense’s request to examine a computer program, despite assurances of confidentiality to protect claims of proprietary and “private work product.”[73] The defendant argued that the plaintiff’s refusal to provide it with the underlying data and theorems of the simulations impaired the defendant’s ability to adequately cross-examine the plaintiff's expert witness.[74]

62. Could computer logs be deemed a simulation of network activity? An argument could be made that logs are reconstructed images of an event(s) according to input data, which, like any witness testimony, may or may not be credible. Under what standard, for example, should courts admit Microsoft IIS[75] (web server) logs that depict the web activity of any number of users? Is it akin to a chart or graph of a systems administrator’s opinion that Jane Doe defaced the former’s company’s website, and thus subject to a lower threshold? Or are these logs substantive evidence of the alleged incident that should be scrutinized under higher thresholds like Daubert? Does it matter that this particular server software was vulnerable to any number of exploits that could have affected the resulting log data? If this is admitted as demonstrative evidence, is its probative value outweighed by its potential to mislead the jury?[76]

63. Is testimony about a network intrusion analogous to air crash simulation testimony? Surely courts would be hard pressed to admit airline reconstruction simulation without scrutinizing the quality and quantity of data points contained in the black box.[77] Yet, why should the intrusion detection logs derived from computer network software and presented in a common, browser-like interface be treated any differently?

3. Issues Arising From Attempts to Reconcile Standards
a) The Circumvention of Daubert

64. As pointed out in the previous section, judicial treatment of computer-derived evidence is the framework within which judges and litigants will most likely manage reliability challenges to the software producing digital evidence. Courts may opt to take judicial notice of the evidence’s auhenticity, declare it demonstrative evidence that illustrates the testimony of an expert witness, or admit the digital evidence under the Business Records Exception to the hearsay rule. While these standards have been used to justify the admission of digital evidence in the past, they avoid direct scrutiny of the source of the evidence – the software. Whereas the need to present probative and visual evidence of digital disputes is accelerating, it is occurring under the watch of Daubert, Kumho, and Federal Rule of Evidence 702. It is clear that Daubert, Kumho, and Federal Rule of Evidence 702 attach to scientific, technical, or specialized knowledge, so efforts to circumvent software analysis under this standard contravene its purpose. The most obvious solution to facilitate reliability determinations of this type of evidence lies within an already established penumbra of case law surrounding Daubert, Kumho, and Federal Rule of Evidence 702.

65. If computer-derived evidence is admitted under a “fair and accurate portrayal” standard, or is entitled to judicial notice as process or system, are Daubert and Federal Rule of Evidence 702 being thwarted?

66. As stated above, it is unclear whether the reliability of computer-derived evidence necessitates an evidentiary hearing under Daubert. Whether or not this type of evidence is entitled to a presumption of reliability upon such showing or should be scrutinized under the rubric of F.R.E. 702 is unsettled. This issue is beginning to emerge more frequently, undoubtedly fueled by the unclear standards for determining the reliability of technical and specialized knowledge, as well as the case-by-case approach with which F.R.E. 901 has been applied.[78] The case-by-case approach to this issue stems in part from the variety of forms that computer-derived evidence can take (computer-generated business records, computer-stored records, computer logs, animations, re-creations, simulations, etc.). The legal reasoning that has governed admissibility in similar cases is helpful in forecasting how courts may handle future challenges to digital evidence.

67. One federal appeals court postulated in dicta that computer evidence resulting from a process or system would not necessitate an evidentiary hearing because the underlying principles of this technology are well understood. Yet “serious questions of accuracy and reliability arise, if at all, only in connection with their application in a particular instance.”[79]

68. A recent case requiring stricter frontline gatekeeping involved a Washington state court that subjected software used by the state to an evidentiary reliability hearing under Frye.[80] In that case, the defense challenged the admissibility of deleted evidence recovered by this forensic software tool. During the suppression hearing, the court asked: (1) is the underlying scientific theory – that deleted files can be recovered – generally accepted within the relative scientific community; and (2) is the technique used – recovery of deleted files – valid?

69. The court did not rule on the scientific theory directly, since there was no defense challenge to the existence of deleted data; rather the defendant questioned the data’s “completeness.” Thus, the challenge really focused on the accuracy of the technique. For the reasons used to deny the defense's challenge to the use of the software, the court inferred the validity of the technique from the commercial availability of the software; the variety of available software so employing this technique; the wide use of those types of software tools/processes by law enforcement and information technology professionals familiar with the targeted, proprietary operating system; and the familiarity and testing of the software tool upon which the expert based his testimony.

70. Several implications flow from this decision and its underlying rationale. First, the fact that the court subjected the software tool to a reliability hearing shows judicial recognition of software as a “scientific technique,” or, at the very least, as a technique that is suitable and/or prone to the same degree of scrutiny. Further, insofar as this test is utilized to determine the validity of a process used to obtain, enhance, or analyze evidence, it would be difficult to dispute that all software falls under that umbrella. Next, industry usage was decisive in determining admissibility, as the “relevant scientific community” was adjudged to be law enforcement and industry professionals. Finally, the affirmation of findings (deleted files) using other methods and the existence of various programs incorporating this technique (recovery of deleted files) were vital to the software's admissibility. This decision underscores how the judiciary recognizes factors such as repeatability, objectivity, and verifiability in evaluating reliability.

b) Dangers of Presuming Reliability of Proprietary Software

71. What lurks behind this low threshold of scrutiny for computer-derived evidence is a dangerous presumption that necessitates critical re-thinking: if the fact that software has become standard and generally available contributes to the de facto reliability of the evidence it generates, is reliability truly being addressed? If not, how would reliability be proven?

72. Whatever the level of initial scrutiny, the fact that evidence was spawned by “standard computer industry technology” factored into admissibility determinations. Deference to standard computer industry technology may be a symptom of judges’ wide latitude and lack of guidance in handling computer-derived evidence, as well as a foreshadowing of how they will handle challenges to software reliability.

73. There are indications that evidence resulting from proprietary software[81] enjoys a presumption of authenticity, while its customizable Open Source counterpart faces a higher hurdle:

Evidence generated through the use of standard, generally available software is easier to admit than evidence generated with custom software. The reason lies in the fact that the capabilities of commercially marketed software packages are well known and cannot normally be manipulated to produce aberrant results. Custom software, on the other hand, must be carefully analyzed by an expert programmer to insure that the evidence being generated by the computer is in reality what it appears to be. Nonstandard or custom software can be made to do a host of things that would be undetectable to anyone except the most highly trained programmer who can break down the program using source codes and verify that the program operates as represented.[82]

74. There are several misconceptions here that, if adhered to, may lead to evidentiary standards and precedent constructed on a house of cards. For one, reality is sidestepped by assuming that commercial software is more reliable because its capabilities are well known and cannot be manipulated to produce aberrant results. The distinction between “capabilities” and “intentions” is vital to recognizing that what commercial software is capable of and purports to do are often quite different from what results it produces. Arguably, the industry-wide disclaimers and shrinkwrap licenses are admissions by software developers themselves that reliability is not part of the business plan.

75. Another misconceived basis for presuming the authenticity of closed, proprietary software is that it cannot be manipulated. This presumes that the original code harbors no unknown “features” [83] and ensures that deficiencies are not addressed and aberrant results are propagated until and if market pressures force a change. The fact that open software can be made to do a host of things undetectable to none other than a programmer also means that verification that the program fails to operate as represented can be addressed. This is in stark contrast to proprietary software, whose operators proceed on blind faith that the software will perform as advertised despite constant bug fixes, patches, hacks, and vulnerability alerts to correct flaws.[84]

76. The application of Daubert notwithstanding, at some level the aforementioned standards (see Sec III.B.2, supra) involve the exercise of judicial discretion as to whether such evidence is reliable enough to go before a jury. These standards, however, do not simplify decision-making for judges who must arbitrate conflicting accounts of the reliability of software producing the evidence. This is so because when applied comprehensively in the context of technical disputes involving software-derived evidence, these standards reinforce the call for a mechanism to guide their discretionary judgments of trustworthiness. This has resulted in conflicting rulings on computer-derived evidence that will likely become more ambiguous when challenges to software commence.

77. A dangerous precedent is being set if source code for proprietary, commercial software is not required to establish authenticity of computer-processed evidence but such a showing is required for open software. Beside the fact that this distinction is based on faulty presumptions about the nature of proprietary software (as discussed infra), it fails to address the reality that Open Source (custom) software is propagating and becoming a force in the industry. It would be a mistake to assume that several judgments raising the bar for the admission of Open Source-generated evidence will deter the use of custom software. While judge-made law can encourage reliance on technical standards, such standards originate in industry and are influenced by the technical environment. Failure to recognize this course and embrace a mechanism that facilitates reliability determinations regarding evidence that is a guaranteed byproduct of the state of industry will only broaden the disconnect between law and industry.

c) Danger of Inferring Reliability from Market Share

78. The preference for standard commercial software carries with it the inference that circumstantial guarantees of trustworthiness are based on market share, which inference has not encouraged the proliferation of reliable software. As discussed infra, market share for proprietary software has been attained at the expense of testing and reliability.

79. The market share metric infers reliability from acceptance within the user base. Hence, there is a presumption that a particular software program would not be widespread unless it were reliable. The pervasiveness of unreliable software, however, has molded a community of users that has grown increasingly desensitized to unreliable software and its accompanying contraindications as an accepted cost and defining attribute of networked society. This may be attributed to a variety of factors such as lack of accountability demand on vendors, time constraints in the workplace, difficulty collectivizing parties who share the same grievance, and tolerance of vendors’ wholesale disclaimers of software performance and quality. Nevertheless, if courts are using widespread commercial deployment to support presumptions of reliability, they are in effect applying market share to measure the “circumstantial guarantees of trustworthiness.” Whereas business reliance may have been a proper sieve for dubitable physical documents or data not subject to a panoply of algorithms (i.e., calculators, paper records transferred to computers, simple processing), the realities of software in the modern business world suggest that such deference may be dangerous.

80. Furthermore, the traditional business preference for proprietary software is tied back to legal considerations, which are typically as unproven as the software itself. In other words, there is a default preference for proprietary software because it purportedly offers an entity upon which to place blame.[85] Reality proves this to be illusory since commercial vendors are very rarely held accountable when their software does not do what it purports to do. In fact, all commercial proprietary software removes the right to sue the manufacturer.[86] In the absence of objective criteria upon which to hold manufacturers of proprietary software accountable for lack of reliability (i.e., certification, insurance, legal judgments), the presumption that Microsoft’s monopoly on the market, for example, denotes reliable software is unfounded.

d) Problems Assessing the Reliability of Proprietary Software

(i) Proprietary Software is Not Amenable to Measuring Reliability Standards

81. Perhaps the reason that software reliability standards have been so nebulous is that the development model of proprietary software that dominates the market has been incongruent with measures of reliability. Open Source software provides both a mechanism to facilitate the application of judicial standards, as well as arguably a framework for software development that meets legal thresholds for admissibility. One way to evaluate this predictive solution is by extrapolating the proven features of Open Source software to scientific standards, within the context of legal parameters.

82. Verifiability, objectivity and transparency are prominent measurements upon which scientific reliability standards are based. How does one gauge the reliability of closed software? What happens when a machine error obliterates or causes a change in the data being offered as evidence? If there is no trail of crumbs left by human intervention, how does Jack Unsumer dispute machine-generated information produced by software whose code is proprietary and whose effect and outcome cannot be duplicated?

83. One option would be to take the developer at its word. Setting veracity aside, how often have courts accepted the subjective testimony of one person, let alone one who has a partial, vested interest in the success of a product? Combine this with the fact that it would be exceptional to find a developer who did not disclaim virtually all guarantees on the functionality of their software, and this option loses any appeal or practicability.

84. Another attempt to gauge the reliability of closed software is to base opinions on experience using the software. Experience, however, is only as valuable as the breadth of software use and exposure to disparate circumstances. Large software is often too complex to test in every scenario regardless of the amount of time available. How does this account for silent errors that go undetected because all the possible points of failure have not been tested? Open Source presents less of a risk that the underlying software is unreliable because there is a higher level of testing and feedback. This means more flaws can be discovered during development due to a wider scale input into the architecture and design insight, core program code, and documentation.[87] In addition, the features of commercial, closed software only contribute to system and network complexity, which means that specifications for those components are likely to be incomplete.[88]

85. Transparency is another value that characterizes both scientific methodology and the Open Source model. Open Source confers the right to view and modify the workings of a system. Closed software licenses perpetuate any unreliabilities by denying the capacity to make software reliable, such as fixing bugs in code or realizing and correcting integration problems with other packages. Further, proprietary licenses severely impede expectations that those unreliabilities will be alleviated, since there is no guarantee that vendors will fix flaws in a timely manner, if at all.[89] This market force suggestion of which vulnerablities or unreliabilities should be addressed and which ultimately are chosen to be addressed by the developer is reactive. It does not address reliability prior to distribution; the software is still out there and bug fixes are far from uniformly or reliably implemented.[90]

86. Finally, quality assurance mechanisms of proprietary software are primarily based on feedback from customers and publicity spin control tactics of the developer. This invites inconsistent levels of quality in different versions within a single COTS product.[91]

(ii) Impracticality of Proving Proprietary Software's Reliability

87. Open Source tools and experts employing them help resolve some tensions that accompany efforts to prove the reliability of proprietary software. Postive effects include the cost benefit of encouraging the efficient application of justice, operative circumstantial guarantees of trustworthiness, and protection of litigants’ constitutional rights

88. The cost involved in proving the reliability of tools or techniques under Daubert and expert testimony, however, may create a disincentive to litigation in the civil arena and/or set the stage for a criminal arena that is liable to bankrupt itself.

89. Critics argue that the amended F.R.E. 702 will infringe on a litigant's constitutional right to a jury trial by denying parties a fair opportunity to present a complete case or defense, and impose economic barriers to satisfying the standard of proof to the detriment of an economically disadvantaged party.[92]

90. This fear may be realized if Open Source software is not embraced in cases where digital evidence is challenged. For instance, if the reliability of logs generated by a commercial software product (Microsoft IIS, for example) is challenged via a motion in limine, a “poor” party may very well have to expend great resources to establish, through documentation and testimony concerning product development, management, testing, and implementation, reliability sufficient to meet this threshold. This process may include procuring employees from the software company to address these questions, and obtaining the proper source code and software development documentation for the particular version at issue. To analogize the latter point, if the reliability of the brake system on a 1990 Jeep Cherokee Sport was at issue, a court would not allow proof of a model year 2000 Jeep Grand Cherokee to suffice.

91. The preceding example assumes that the source code and its authors are available and willing to make disclosures. All of those assumptions are unlikely, as it is not uncommon to be unable to locate the original programmers, or for the vendor to lack copies of the source code and accompanying documentation. Furthermore, source code disclosure will not likely be surrendered without first having to overcome claims of privilege. Without the source code, hailing an independent expert is futile, so a party is left with the testimony of an employee of the commercial vendor whose motivations to obscure or offer less-than-enlightening reliability testimony are enormous.

92. The bottom line is that a wealthier litigant would have the resources to fund or to raise the bar for such offers of proof. Even if the evidence were to go before a jury, the economic hurdles would not dissipate, but rather, would merely be transumed into the need to convince an entire body of factfinders, rather than a single judge.

93. In an alternative scenario where the reliability of Open Source software is challenged, the economic disincentives may be lowered. In this case, the proponent of the evidence is no longer at the mercy of a corporate juggernaut whose source code is kept under lock and key; nor is the proponent relegated to inferring the reliability of the software in question based on personal usage, which may or may not be effective in exposing all of the potential “unreliabilities” present in the software. The problems of proof – and costs associated with same – are able to be diluted in the case of Open Source. Either the proponent will have the technical skills to review the Open Source software himself and make offers of proof; or he will be able to retain any number of objective technicians to look at the code and its attendant details to render an opinion of reliability. In either case, the costs of hunting down an original software developer to testify that his proprietary code does what it purports to do are alleviated.

94. Open Source software facilitates scrutiny of the reliability of software. Open Source makes it easier to verify whether or not a vulnerability exists that could have allowed the tampering, alteration, corruption, or forgery of information produced by the software. Although outside the scope of a court's duty, from a proactive angle, there is a greater chance that the Open Source software has been reviewed to discover bugs. Open Source software makes it easier for independent, third parties to verify the existence of bugs and the accuracy of the source code itself because it is maintained by disinterested third parties.

95. Another tension that Open Source mollifies is infringement on a litigant’s ability to mount a meaningful challenge to admissibility. Normally, courts will disallow challenges to the authenticity of computer-based evidence absent a specific showing that the computer data in question may not be accurate or genuine; mere speculation and unsupported theories generally will not suffice.[93] As with proprietary software, the question arises of what would constitute a specific showing that would cast doubt on the authenticity of computer data. Certainly there are credible data from disinterested third parties documenting the vulnerabilities of specific software.[94] It is questionable whether a court would find that documentation from such sources verifying that the particular version of software at issue was exploitable (e.g., word processing or server software contained a compiled-in back door account with a known password that would allow any local user or remote user able to access a certain port to manipulate any database object on the system) prior to or contemporaneously with the production of the evidence in question. Would fairness dictate that in order to mount a specific showing, the opponent must have the opportunity to examine the source code responsible for the production of the evidence in question?

96. With evidence derived from Open Source software, such offers of proof are less problematic since they alleviate the tangential conflict involving proprietary objections, while safeguarding the opponent’s ability to substantiate its challenges.

(iii) Changing Technical Environment – Industry and Science Intersect

97. Within industry, the technical and economic environment is shifting to recognize the need for reliable software. This shift to embrace reliability and define it in terms similar to scientific precepts illustrates a common ground between industry and science that is elemental to Open Source software theory.

98. Software in modern industry is no longer sold or implemented in an isolated, stand-alone PC environment. Instead, computer networks connected by interoperable programs define the digital infrastructure. Data storage forms and techniques have changed. Storage capacities on consumer systems average 20 gigabytes (“GBs”), and data in one document often spans multiple disk surfaces via RAID (Redundant Array of Independent Disks) technology. Peripherals have evolved intelligent modems and network routers. Wide area telecommunication methods are more prevalent, and data and voice convergence is a reality that will become standard in due time.

99. Software-created applications have incurred significant changes as well. Software comprising email and client/server applications has not only become ubiquitous, but involves seamless data interfaces overlaying data that are dispersed across a network. Databases are similarly ubiquitous, and the controlling software may assemble a document with one computer from a disbursement among many, which makes its existence quite different than one in a filing cabinet or even on a PC hard drive. Artificial intelligence and computer-directed procedures affect everything, including emergency services, traffic control, manufacturing, sales, and infrastructure management.[95]

100. In this environment, reliability can be improved by identifying, anticipating and targeting vulnerabilities and reducing defects. This involves a breadth of understanding about how software interacts with other elements of a larger system.[96] There is a widening gap, however, between the needs of software developers and their ability to evaluate reliability in networked systems. Controlled scientific experimentation under traditional mechanisms (i.e., empirical testing by building the same system repeatedly under controlled conditions, using different approaches) is not feasible due to the cost of building such systems. Therefore, the bulk of laboratory assessments of software reliability are based on testing with small samples that cannot address issues of scaling new technology.[97]

101. Furthermore, the pace of information technology development is exceeding the ability of users and software developers to adequately document the workings and content of systems.[98] Microsoft is a prominent example of this black box predicament. The size and complexity of these interactions and continual software changes make software reliability more valuable, yet more challenging to obtain. Ultimately, this difficulty has corresponding evidentiary implications, as the ability to generate, send, receive, store, and process potential digital evidence is likewise evolving.[99]

102. As discussed infra (see Sec. III.A.2), judicial scrutiny of scientific or technical evidence is rooted in proving that results produced are repeatable, objective, and verifiable, whereas industry measures the reliability of software in terms of authenticity, integrity, and availability. Open Source software is a mechanism by which both sets of values can be achieved.

103. Within industry, Open Source software is purported to be beneficial where (a) reliability, stability, and scalability are highly valued; (b) accurate software design and implementation is not readily verified by means other than peer review; (c) the software is business-critical; or (d) the software facilitates a common computing and communications infrastructure.[100]

104. Within legal proceedings, Open Source has a similar effect on the determination of reliability for software-derived evidence. As software increases the discriminatory capabilities of digital evidence, the underlying data must be relied upon as accurate and the software producing it should be relatively immune to failure.

105. To be sure, these are probing questions that can be addressed by understanding that what courts are seeking in adjudging evidentiary admissibility is a mechanism by which they can gain circumstantial guarantees of trustworthiness, applied objectively, which mechanism scales to a diversity of cases with minimal potential for error , and that minimizes subjectivity and mitigates error. These standards are why the concept of scientific methodology has been the cornerstone of defining the reliability of evidence. Scientific methodology was founded on principles: the production of objective evidence, the minimization of subjective evidence, and the mitigation of error. Courts have attempted to duplicate these results when dealing with digital evidence, by using the industry standard and/or commercial availability to infer reliability. These measuring sticks,, however, have not been calibrated in the metrics of reliability, but in those of increased features and rapid time-to-market, which work in opposition to reliability.

106. Nevertheless, the outlook is not grim in terms of fashioning solutions to deal with complex disputes over the reliability of digital data. The principles and values underlying industry standards are evolving to resemble those of science, as exemplified by industry efforts to make software tools and systems more reliable.[101] Recognizing the convergence of industry’s embrace of reliability and the law’s requirement of the same, a logical approach would be to embrace the emerging industry standard that is conducive to rendering objective measurements of reliability using scientific principles. The Open Source model of software development embodies the principles of the scientific methodology, and may be the means by which to authenticate or test the reliability of software.[102]

4. Open Source Software As a Solution

107. Should evidence obtained through industry standard software be subject to the scientific analysis outlined in Daubert? If so, how is this possible with proprietary software? Recognizing that the practical resolution of disputes necessitates that lines be drawn somewhere, imputing the reliability of the systems producing the evidence from widespread use and industry acceptance is not faulty in and of itself.

108. Open Source software sits at the crossroads between the need to present probative and visual evidence of digital disputes, and the legal standards beckoned by Daubert, Kumho, and F.R.E. 702. On one hand, digital evidence that does not square with business record protocols, or is produced by pervasive software whose reliability varies depending on the environment, is pervasive and vital to proving legal claims. On the other hand, legal standards call for specific analysis of scientific, technical or specialized knowledge grounded in assurances of reliability before being presented to a factfinder.

109. As precedent indicates, courts have looked to the commercial or standard status of software to draw inferences of reliability. Jurisprudential insistence on measuring reliability via the principles of scientific methodology is skirted when proof of software reliability is based on industry standards that do not embrace those same principles. Where those industry standards are marked as proprietary, closed, and nontransparent, software reliability boils down to successful marketing and distribution, and proof entails a leap of faith that flies in the face of contraindications.[103]

110. How can software that, by its own admission, disclaims all but its price tag qualify as an automated process that produces accurate results under F.R.E. 901?

111. In a similar vein, courts are wise to scrutinize the computer forensic software techniques called upon after legal mechanisms are invoked to ensure that the “original” data was not altered. Yet, there is an underlying assumption that what is collected accurately reflects the transaction. This reflection becomes a house of mirrors, however, when such judgments are inferred from black boxed coding instructions.

112. Courts have been faced with “scientific or technical” evidence generated by computers for some time, such as the printouts from gas chromatographs of chemical substances. Yet analysis has not stopped at determining whether a machine was calibrated and functioning properly before and at the time of testing. Originally, the scientific process underlying drug identification had to be scrutinized using Daubert-like factors. Like gas chromatography, for instance, computer software is scientific in that it is based on mathematical algorithms. Yet these same reliability protocols have not been applied consistently and pervasively to “industry standard” software, undoubtedly because it would be impractical under the proprietary regime that defines “industry standards” today. The rapid changes in software technology development mean that the program (or, at least a particular version of the software) would be outdated by the time peer review, publication, and empirical testing could furnish a reliability verdict. In addition, the sheer quantity of testing variables and situations would preclude implementing traditional reliability measurements, because the expense would be prohibitive and serve as a disincentive to litigation.

113. Open Source software is one mechanism through which these legal principles can be implemented in modern high-tech society, where industry, science and technology are converging.[104]

a) Open Source: The Digital Daubert?

114. A comparison of the values and mechanisms defining Open Source with those enumerated by Daubert reveals striking commonalities that can be drawn upon to determine the reliability of software-derived evidence.

115. Although Open Source is used to describe the conditions under which software source code used by computers is made available to others apart from the developer, its embrace of principles of scientific methodology[105] is what makes it conducive to applying the Daubert standard to digital evidence. As pointed out initially, reliability is a significant motivation behind the Open Source movement. The values defining reliability of Open Source are congruent with the values industry has defined, but not yet realized, for reliable software.[106]

(i) Peer Review and Publication

116. Similar to the scientific process, peer review plays a central role in the Open Source model. Putting source code out in the open makes it difficult to hide design and implementation flaws that may affect the evidentiary end product. This model allows for independent verification and validation of reliability that is buttressed by the quantity and quality of available functional experts. Unlike with proprietary developers of commercial software, for instance, Open Source contributors do not have the same type of vested interest in the success of the system. Moreover, the software is placed under the scrutiny of hundreds and thousands of independent developers who focus on areas of concern or on areas where they have functional expertise.

117. Science and Open Source can both be characterized as adversarial processes. Whereas the former is a search for truth that is played out as competition between ideas using observations and data, the latter is a competition between coders using those same variables to find truth in code and to verify whether the code is accurate (corresponds with what it is intended to do). Peer review is critical to this adversarial process. The Internet and Open Source change control procedures are the hard science journals for software reliability. Opponents might argue that this analogy is unfounded, since software is market-driven, a fact that calls its purity into question. Projects are selected on the basis of there interest to the individuals who participate, however, most of whom do so in their spare time without compensation. The process of selecting Open Source projects is unplanned, and competing implementations and efforts are both welcomed and decentralized. Further, inspiration is not derived from compensation assurances, but from a desire to be part of a movement that embraces the creation of great software or membership in a community with the same values. [107]

118. Even if one were to concede the market theory of Open Source software, which holds that all software is driven by economic motives, it should be noted that the adversarial nature of scientific discovery renders it not without fault. Personal gain – whether it be money or fame – is not beyond the purview of scientists. For instance, scientific articles submitted for publication and for funding proposals are not always the best arbiters between competing valid claims because the reviewer is often a competitor for the same resources. Peer review has also been criticized as functioning to perpetuate the current scientific paradigm.[108]

119. Finally, peer review is essential to promoting objectivity and minimizing subjectivity by utilizing empirical results to determine and promote reliability. This means that access to code underlying software enables observations and experiments leading to rational conclusions that transcend or minimize individual prejudices and self-interests.[109] A more accurate definition of scientific methodology would be hard to find. Further, this objectivity is only enhanced by the fact that Open Source does not labor under any time-to-market or decisional deadlines that might compromise the objective process.

(ii) Error Rate and Falsifiability

120. Open Source provides a mechanism for exposing errors in software. These errors are based on conditions present in the actual use of software. This fact is significant in light of the difficulty of testing software given the increased number and types of systems, features, configurations and implementations; the variety of interactions between different products from different vendors, across different networks; and the divergent interpretations of how software code functions within integrated systems.[110]

121. If error rates are only as valid as the corresponding testing procedures, the quality of testing that can be performed on software whose source code is known is significantly higher and more easily verifiable. In other words, the only way to test proprietary, closed source software is by using “black box” techniques, which entail exercising code over a range of inputs and observing the outputs for correctness.[111] This is far from comprehensive given the near-infinite number of possible test cases needed to represent real-world systems and variables. Error rates may be valid for simple check-balancing programs run in environments where all the variables are known, but the same cannot be said for black box testing of far more complex software, such as Intrusion Detection System programs which have significantly more input/output variables.[112]

122. Another problem with testing proprietary software using black box techniques is that the correct operation of the software may not be a measurable output. It may, for instance, be easy to verify the correct output of a check-balancing program, which is the account balance, by manually doing the calculations. The same cannot be said, however, for determining if data packets collected over a Gigabit Ethernet were captured by an IDS.

123. In addition, black box testing is not comprehensive and is not as “testable.” Without access to the source code, for example, it is impossible to determine if there are portions of the code that have not been executed. If it has not been run, it cannot have been tested. This is likened to having a sleeping bomb in software.[113]

124. Lastly, there is empirical evidence that black box testing does not uncover as many errors as a combination of testing methods.[114] The error rate of software under closed source means that this information is kept proprietary, subject to the same claim of privilege that precludes the source code itself from being disclosed. Even so, there is a presumption that error rates of a particular piece of software are known, which would make for an exceptional case since it is well known that testing is the caboose on the time-to-market train.[115]

125. Code under the Open Source model is subject to falsification, as modifications in software code to produce desired results can be tested to prove whether the results are reliable. Just as rivals attack the scientific theories at their weakest point, so do other developers/coders scrutinize source code and find flaws in the digital blueprint. Closed source developers are often left to wonder if it was indeed the code in one particular piece of software or some other factor that may have caused a mishap, and they are handicapped by the inability to test and verify their hypotheses by tweaking the code. The best way to actually test a hypothesis is to think through the possible consequences and see if those consequences are observed. If software is coded for X, for example, then one would expect the software to produce Y. If Y results, then the hypothesis – that the code will do what it is coded to do and nothing more – is accurate if or until proven otherwise.[116]

(iii) General Acceptance Within the Relevant Community

126. The relevant community in this case would likely refer to those who develop software or have the skill to deal with software at the source code level. Since Open Source is a relatively new concept, measuring its general acceptance would be premature. Those who have an understanding of software programming and the ability to manipulate code to perform desired functions, however, would certainly agree that this feature is preferable to having to contact vendors or draw inferences as to why a program may have crashed, for instance, or how to fix an incompatibility.[117]

127. Nonetheless, if utilization by IT industry professionals evidences general acceptance from a practitioner’s standpoint, then one need only point out the preference for certain Open Source products over their proprietary counterparts. Open Source Apache web server, for example, is the overwhelming dominant choice of webmasters, as it has held over 50% of the web server market.[118] To exemplify further, Open Source GNU and Linux software were determined to be more reliable than commercial software in a study designed to test failure rates of software utilities.[119]

128. Even the software behemoth Microsoft, much maligned for its reputation for monopolizing the software market, has quietly initiated a limited source code sharing program.[120] This endeavor represents a remarkable break from its closed source business model, under which code was only sparingly shared with Micorsoft’s most valued customers under strict NDAs and confidentiality contracts. Even though not truly Open Source, insofar as it extends permission to view rather than modify the source code, it demonstrates recognition of the value of having access to the code. By Microsoft’s own admission, this access will allow troubleshooters to work backwards to solve software-related problems. Further, the President’s Information Technology Advisory Committee (PITAC) has officially recommended that the federal government should encourage the development of Open Source software for development for high end computing.[121]

b. Advantages of Open Source as a Solution

129. The arguments for using the Open Source model for determining reliability are compelling given that (1) the issue of software reliability is being forced by the converging meaning and significance of reliability within industry and courts, collectively; and (2) software is oftentimes the singular basis for expert analysis that will give rise to challenges to the digital evidence at all stages of civil or criminal litigation.

130. Acknowledging the widespread notion that software is a continuous process rather than a static product, Open Source is responsive to the reality of how applications get used in ways and under circumstances that are not anticipated by the manufacturer.[122] Even the most meticulous coding will not alleviate the potential for software failure and error. It would be unreasonable to expect bulletproof software, as it has been estimated that for every single line of code that is written, X number of bugs is created. Such a reality leads to software that fails to do what it purports to do and/or performs functions unexpectedly.

131. The layperson whose Microsoft Excel program crashes, for example, recognizes this truth as “blame-the-computer” grumbling. In such a case, the aggrieved user is left to wonder about the cause of the mishap, feels confounded as to the possibility of future recurrences, and hopes that the work performed has not been irreparably damaged. For the most part, technically skilled professionals have no magic elicir for these problems, since most commercial software conceals its source code – the instructions that interface between the computer user and the computer hardware – under proprietary lock-and-key.

132. Essentially, software operates as follows: commands and data are inputted, the software responds under the direction of the underlying code, and output results. Proprietary software is a “black box,” because what happens between the input and output stages is anyone's guess (except, of course, the programmer and manufacturer). Without access to these blueprints, a computer professional is left to infer, based on his knowledge and experience, the causes of and solutions to software problems. Open source software demystifies many of these problems by exposing the source code in a public arena. In this way, those with programming skills can “see” what the software is really doing, diagnose problems, and substantiate predictions of future recurrences.

(i) Objectifying the Experts

133. Open Source may be valuable to facilitate solutions to issues revolving around experts who will be increasingly hailed to determine the facts of what occurred in complex technical cases, assess critical software application failures or allegations, and render opinions on network intrusions and disruptions.

134. The reliability factors enunciated in Daubert and F.R.E. 702[123] emanated from a tradition of expert testimony being germane primarily to professionals in academic, laboratory, or scientific settings. Experts testifying to the identification, collection, and analysis of digital evidence do not fit the image of prototypical silver-haired scientists with arms-length credentials. Instead, the breadth of computer science applications and the rapid rate of knowledge creation have expanded the pool of professionals who can assist the factfinder to include the likes of non-degreed twenty-somethings with tongue piercings. Unlike experts in hard sciences like physics and chemistry that are steeped in a rich academic tradition, computer science was not institutionalized within academia until relatively recently. Some of the most technically enlightened professionals can wield no more than a bachelor’s degree, yet they possess more knowledge than can be contained in a library of relevant books. Furthermore, the plethora of species of computer science and ever-changing technology make it impossible for any one person to claim expertise in more than a single narrow field. To call someone an expert in computers is analogous to professing someone an expert in medicine.

135. Furthermore, much like clinical medicine, computer science exhibits features of both science and art, and expertise is built almost exclusively on experience. Nevertheless, bestowing the designation of “expert” remains largely within the discretion of the judge and partially a function of challenges or the lack thereof from party opponents. Quite often the types of experts who are or will be called to opine on network and diagnostic issues giving rise to challenges to digital evidence do not operate from textbook procedures that have well-known error rates, as is the case with DNA profilers. When trying to determine if there has been a network intrusion as well as when trying to identify the perpetrator, for example, there is no established methodology the deviation from which would help gauge the reliability of an expert systems administrator’s testimony. The real challenge lies in being able to discern whether the integration of skills that are developed through experience and practiced as a craft should entitle the witness’ testimony to be admitted as that of an expert. A youthful network administrator with a ponytail who works for a dot.com may be as equally qualified to render an opinion as an IT manager who is twice his age and has been employed with General Electric his entire life. By the same token, this awareness creates opportunities for charlatans and those who would have been characterized in the past as “junk scientists,” both of whom are motivated by less than noble incentives.

136. Indeed, the text of F.R.E. 702 expressly states that an expert may be qualified on the basis of experience, and further recognizes that in certain fields this is the predominant if not sole basis for reliable testimony.[124] Kumho maintained that “no one denies that an expert might draw a conclusion from a set of observations based on extensive and specialized experience.”[125]

137. Open Source software is valuable not so much in helping to determine if a given IT professional is “fit” to be an expert as it is in supporting an infrastructure where the linkage of facts and conclusions about cause and effect opinions of digital evidence can be objectively scrutinized. Thus,unlike expert testimony based on analytical techniques taken from proprietary software, Open Source takes some of the “black art” out of the analysis. Insofar as the Open Source model is more peer-to-peer and widely distributed, it more aptly fits with the disintegration of the hierarchical nature of scientific expert admissibility in the courtroom. No longer is reliability proactively made or reactively judged by professionals at the top of the food chain; rather, the pool of potential independent third party validation is broadened.

(ii) Countering the Automating and “De-skilling” of Experts

138. Another reason why Open Source is valuable is because of the emergence and propagation of software that automates the analysis of digital evidence. Such software serves to automate expertise and underscores the importance of having reliable software upon which these analyses rest. Since investigating complex technical cases involves more experience-based testimony, the “facts/data” contemplated in F.R.E. 702 must be scrutinized perhaps more carefully. As discussed, Open Source facilitates the application of Daubert to determine the reliability of the software producing the digital data upon which an expert's testimony is based.

139. A central question throughout this article concerns the threshold of scrutiny for digital evidence. With respect to expert testimony, one is left to inquire whether the threshold should be raised when it provides the sole basis for an expert opinion about an unwitnessed, digital event. Photographic evidence, for example, is often admitted under the “fair and accurate portrayal” standard. Reality, however, can be altered without varying the image. A photo of an accident scene, for example, can be authenticated by a witness, yet not capture the subtleties such as weather and lighting that would affect the reality of whether or not a sign relevant to an accident could be viewed.

140. Likewise, the reality that forms the basis for a technical expert’s testimony, such as logs produced by software, can be altered by variances that affect the story that the corresponding data portrays. While a witness can be cross-examined on perception, memory, bias, etc., regarding an accident scene, how can the reliability of a technician's testimony be tested if it is based on results of proprietary black box software? At least with photographs, factfinders can use reason to explore how, given the facts of a case, a scene may have looked at the time of the accident. Conversely, with digital evidence, jurors are completely at the mercy of an expert who may present evidence that they have no way of discerning from the printout created by a monkey at a keyboard.

141. Open Source software can facilitate reliability determinations of expert testimony in that it assists in determining if an expert’s testimony is based on sufficient facts and data, as well as whether it is the product of reliable principles and methods. This benefit is significant insofar as these digital data, such as computer logs, have become more interpretive and are being used by experts as a basis for their opinions about unwitnessed events, accidents, or crimes.[126] Software tools and systems have become the “digital eyewitnesses” to events that occur over computer networks. The select data points and factual knowledge that they produce, capture and process converge to tell a story about how a digital event may have taken place.

142. In light of this automated analysis being performed by software, the line between the human skills needed to apply and those needed to interpret the results produced by the software is quite blurred. In this way, software is fostering the “de-skilling” of experts, as evidenced by integrated suites of software tools programmed to automate the identification, collection, preservation, and analysis of digital evidence. This has engendered “experts” who ask how they can get the software to answer a problem rather than how to find out what is underlying that problem. This is perhaps as dangerous as drawing conclusions about the reliability of software that are not based on knowledge of the source code. Opinions based on Open Source can help gauge the validity of opinions that are drawn from the use of these tools, because the facts and methodologies collectively chosen, employed, and embedded in the software amount to a significant basis for expert testimony.

143. It is important to distinguish the use of reliable tools from judging reliable conclusions and/or opinions drawn from the use of those tools and techniques. This distinction explains why the crime scene photographer at a murder scene is not hailed into court nearly as often as the blood spatter expert who uses photographs to draw conclusions about the crime. In the digital crime scene or civil dispute, it is increasingly common for the software not only to take the picture, but to draw conclusions as well. Cross-examining the expert whose opinion is derived from such evidence may call for a cross-examination of the software, which is accurately done by having the source code.

144. This fact raises an issue of whether the witness should be qualified for the purposes of establishing a foundation for computer evidence versus opining on technical knowledge under Daubert and F.R.E. 702. The former approach treats the testimony under the lower threshold of scrutiny attendant to authentication under F.R.E. 901, whereas the latter would involve a more detailed analysis of the principles and methodologies. The former approach was taken in People v. Lugashi, which involved a defense challenge to computer-derived evidence presented in the form of a “data dump.”[127] The data dump was a program run on a bank’s computer system that retrieved and organized the daily credit card transactions reported to the bank. The defense argued that the bank's systems administrator, who maintained such records and who admitted he was not a computer expert, was incompetent to authenticate the digital data.[128] The court rejected this argument and ruled that “a person who generally understands the system's operation and possesses sufficient knowledge and skill to properly use the system and explain the resultant data, even if unable to perform every task from initial design and programming to final printout, is a ‘qualified witness’” for purposes of establishing a foundation for the computer evidence.[129] In so doing, the court impliedly rejected the defense position that only a computer expert who is skilled in programming and could inspect and maintain the software and hardware could testify.[130]

145. The issue highlighted by Lugashi emphasizes that in order to determine whether testimony should be characterized as expert opinion or lay witness observation depends on whether evidence obtained from a software process is reliable. In order to justifiably afford a presumption of accuracy to the software in question, the sufficiency of the facts and data as well as a determination of whether the software is the product of reliable principles and methods is necessary. In these situations, Open Source software is advantageous for the reasons discussed infra. Only after this foundation has been established should the testimony of a lay witness be allowed to influence admissibility.

(iii) Curbing the Battle of the Experts

146. Certain attempts to decide between conflicting accounts of technical issues based on closed source software can be likened to competing expert testimony on the cause of a collapsed bridge based on anything but evidence concerning the girders. In such a case, experts can at most only speculate whether it was destroyed by a bomb, buckled under excess weight, or was ravaged by a horde of mutant beavers.

147. Testimony involving Open Source code, however, is insightful. Open Source is valuable in light of Revised F.R.E. 703, which allows experts to opine on inadmissible evidence if it is of a type reasonably relied upon by experts in the particular field. What are the implications if an expert testifies based only on hearsay in logs, and the underlying software producing this log evidence is both proprietary and unreliable? Since various network and system logs are the lifeblood for systems administrators, passing this qualifying portion of F.R.E. 703 would be trivial. An opponent can, however, raise reliability issues of those otherwise inadmissible logs. What such a move does is transplant into the courtroom offers to prove the reliability of the software producing the logs; only this time, the battle of the experts takes center stage in front of the jury. If the source code is in the open, opponents may be able to mount credible and legitimate challenges to its accuracy with more than speculation.[131] In this way, rational decisions that focus on evidentiary weight rather than an expert's appearance and demeanor are more probable.

IV. Conclusion

148. In attempting to resolve the issue of reliability of some digital evidence, this article has advocated asking obvious yet probing questions concerning the reliability of the software producing the digital evidence. An outline of the legal approaches to reconciling digital evidence standards and of the ensuing dangers of failing to scrutinize the source of the evidence has supported the conclusion that the reliability of some digital evidence may typically be overlooked. Finally, Open Source software has been offered as a solution for facilitating the application of appropriate legal standards to novel evidence and helping bridge the gap between the law and industry in measuring reliability.

149. Evidence derived from proprietary software should not be scrutinized just because the source code is made secret, for closed software is quite often a harbinger of probative, reliable evidence. It is inherently incompatible, however, with the tenets embodied in Daubert and F.R.E. 702, and as such creates a dilemma for judges when its reliability is legitimately called into question. Surely, courts can admit digital evidence produced by software of questionable reliability and allow it to go to the weight of the evidence or label it demonstrative evidence. Nevertheless, it is presented before a jury of humans, who cannot delete images from their minds as easily as a computer dumps its memory. As the history of photographic evidence illustrates, the dangers of misleading the jury, creating unfair prejudice, or wasting judicial resources become very real.

150. In circumstances where the importance of the evidence and the extent to which its probative value depends on its accuracy demand more than cursory proof, the risk of ceding decision making to automated programs via adherence to institutionalized advocacy mechanisms looms large. In those cases, the predominant question asks what the legal system compromises by continuing to handle issues of digital evidence heedless of Daubert standards and the existence of Open Source mechanisms.

* Ms. Kenneally is a Forensic Analyst and Attorney with the San Diego Supercomputer Center, Pacific Institute for Computer Security, where she focuses on substantive research, writing and consultation about prevailing issues related to computer forensics law and computer security in both the criminal and civil arenas. These issues include evidentiary, procedural, and policy implications of new technologies and digital evidence. She has lectured and helped coordinate training conferences for state and local judges, law enforcement personnel and professionals concerned with high technology crime. She holds a Master's of Forensic Sciences degree from George Washington University and a Juris Doctor from Cleveland Marshall School of Law.

** The author would like to recognize the incalculable guidance and vision provided by Attorney Fred Chris Smith of Santa Fe, New Mexico. Mr. Smith is nationally recognized for his pioneering work in dealing with high technology legal issues. The author is collaborating with Attorney Smith on a series of projects aimed at actualizing the proposals championed in this article and in forthcoming publications.

[1] Reuters News, Old U.S. Voting Systems May Work Best – Scientists Say, Findlaw Legal News (February 7, 2001), available at http://www.freerepublic.com/forum/a3a8056331ae5.htm (referencing a study conducted at the California Institute of Technology and the Massachusetts Institute of Technology).

[2] See § II.A.2, infra.

[3] The computer industry has a long tradition of not being accountable for software directly responsible for killing people. See THE RISKS DIGEST, Forum on Risks to the Public in Computers and Related Systems ACM Committee on Computers and Public Policy Volume 18: Issue 28 (July 26, 1996), available at http://www.infowar.com/iwftp/risks/risks-18/18_28.txt.

[4] See, e.g., Weisman v. Hopf-Himsel, Inc., 535 N.E. 2d 1222, 1226 (Ind. Ct. App. 1st Dist. 1989); People v. Bovioi; Burleson v. State, 802 S.W. 2d at 441; 71 Am. Jur. Trials 111 & 118 (1999).

[5] Although courts have not explicitly used the term “closed source” to refer to “standard” computer programs, there is an implicit assumption that they are interchangeable. At this point in time, the features describing “commercial” and “standard” software are equivalent to “closed source” software. Also, courts have not been presented with issues involving “Open Source” software, and therefore would have no reason to introduce this terminology into their decisions.

[6] BIND is the software that provides the domain name service (DNS) for the entire Internet.

For a more comprehensive listing, see http://www.ceu.fi.udc.es/opensource.org-mirror/docs/products.html.

[7] Eric Raymond, The Cathedral and the Bazaar, (viewed November, 2000) at 20, available at http://www.tuxedo.org/<diffesr/writings/cathedral-bazaar/cathedralbazaar-1.html. Although access to a computer program's source code is the major defining point for Open Source software, the distribution terms of what is must be applied in a software license, to be referred to as Open Source, must comply with the following criteria: free redistribution of the program; distribution in source code as well as compiled form; allowance of modifications and derived works; the license must not discriminate against any person or group of persons or field of endeavor; the license must be automatic, no signature required; the rights attached to the program must not depend on the program's being part of a particular software distribution; no restrictions on other software that is distributed along with the licensed software. See generally, Bruce Perens, The Open Source Definition, Open Sources: Voices from the Open Source Revolution (1999), available at http://www.dibona.com/writing/os/online/perens.html (for a more thorough analysis of the Open Source definition and model).

[8] Raymond, supra note 7, at 20.

[9] Certainly, the freedoms and rights attendant on a democratic and capitalistic society justify people’s entitlement to protect their intellectual labor and earn its rewards. Moreover, society is driven to develop technology that will automate tasks and “make life more efficient.” For further enlightenment on the economic self-interests and business case for Open Source, see Advocacy: The Open Source Case for Business, at http://www.opensource.org/advocacy/case_for_business.html (last visited March 30, 2001).

[10] Academic Press Dictionary of Science and Technology, available at http://www.harcourt.com/dictionary/def/9/7/5/7/9757500.html (visited March 14, 2001).

[11] Trust in Cyberspace, Committee on Information Systems Trustworthiness, Computer Science and Telecommunications Board, Commission on Physical Sciences, Mathematics, and Applications, National Research Council, Washington, DC (1998) at 154, available at http://www.aci.net/kalliste/tic.htm [hereinafter Trust in Cyberspace].

[12] Id. The development of “systematic” processes is the extent of the recognized standard convergence.

[13] See, e.g., http://www.din.de/ni/sc27/doc7.html.

[14] Fallows, Frontier Days, Industry Standard, (Nov. 14, 1998), available at http://www.thestandard.com/article/display/0,1151,7618,00.html(visited January 7, 2000).

[15] Email from American Bar Association Section of Science and Technology Law Technical Standardization Law Committee (November, 2000).

[16] See Ben Roethke, The Orange Book is Dead, Long Live the Common Criteria, Security Wire Digest/Information Security Magazine, (Jan. 18, 2001), available at http://www.parallaxresearch.com/news/2001/0122/the_orange_book.html. See generally, Institute of Electrical and Electronics Engineers (IEEE), Standards, http://standards.ieee.org/reading/ieee/std/se/12207.0-1996.pdf(visited June 3, 2000). Note that the Department of Defense issued what is referred to as "The Orange Book," which promulgates policy and assigns responsibilities for the security analysis of automatic data processing systems. See, generally, Department Of Defense Trusted Computer System Evaluation Criteria, CSD-STD-001-83, Library No. S225,711 (Aug. 15, 1983), available at http://www.cerberussystems.com/INFOSEC/stds/d520028.htm.The DOD intends these criteria for use in the evaluation and selection of ADP systems as it relates to sensitive and classified information. This is regarded as a DOD standard and does not match the non-classified business model for designing and evaluating software and interconnected systems bearing removable media. The time and resource-intensive evaluation process runs counter to the industry-based business models needed to thrive in the competitive markets. Interview with Tom Perrine, Security Manager at the San Diego Supercomputer Center (Aug. 17, 2000).

[17] S.L. Pfleeger, et al., Evaluating Software Engineering Standards, IEEE Computer, 27(9): 71-79 (1994).

[18] See generally, http://www.ietf.org/Internet-drafts/draft-ietf-syslog-syslog-02.txt.

[19] Id. For example, lack of integrity checking of messages such as required time-stamping and delivery verification, lack of authenticity, and lack of confidentiality.

[20] Trust in Cyberspace, supra note 11 at 154.

[21] IETF (Internet Engineering Task Force), NIST (National Institute for Standards in Technology) , ANSI (American National Standards Institute), IISP (Information Infrastructure Standards Panel),and FIPS (Federal Information Processing Standards Publications).

[22] Trust in Cyberspace, supra note 11 at 199.

[23] Martin Libicki, The Mesh and The Net: Speculations on Armed Conflict in a Time of Free Silicon, INSTITUTE FOR NATIONAL STRATEGIC STUDIES, http://www.ndu.edu/ndu/inss/macnair/mcnair28/m028ch06.html (visited November 2000).

[24] The word “software” is used throughout this paper to refer to “proprietary” software unless specifically referred to as “Open Source” software.

[25] Trust in Cyberspace, supra note 11 at 72.

[26] Id.

[27] See Fallows, supra note 14; see generally, Risks Digest, available at http://catless.ncl.ac.uk/Risks.

[28] See generally, E. Kenneally, The Bytes Stop Here: Liability for Negligent Security, Comp. Security J. (Fall 2000).

[29] Bruce Schneier, Cryptogram (April, 2000), at http://www.counterpane.com/crypto-gram-9904.html.

[30] Trust in Cyberspace, supra note 11 at 153 (stating “[E]mphasis on standards can actually be something of an inhibitor. Standards efforts, in their desire to achieve maximum consensus, have very long cycle times (five or more years), which certainly do not fit well with product development and release cycles.”).

[31] Trust in Cyberspace, supra note at 198.

[32] Interview with Tom Perrine, Security Manager at the San Diego Supercomputer Center (Aug. 17, 2000).

[33] Daubert v. Merrell Dow Phaceuticals, 509 U.S. 572 (1993).

[34] Id. (citing Frye v. United States, 293 F. 1013 (D.C. Cir. 1923)) (holding that Frye (establishing the standard of general acceptance in the scientific community) was superseded by rule 702 of the Federal Rules of Evidence, which governs expert testimony in federal trials). The issue here concerned the admissibility of scientific evidence (expert testimony based on epidemiological evidence) supporting the claim that the drug Bendectin caused birth defects.

[35] Id. at 593-595. Other factors that may be relevant to the consideration include: the relationship of technique to methods established as reliable; the nonjudicial uses of the method; the logical or internal consistency of the hypothesis; the consistency of hypothesis with accepted theories; and, the precision of hypothesis or theory.

[36] Id. at 590.

[37] Kumho Tire v. Carmichael, 526 U.S. 137, 147-49 (1999).

[38] Id.; see also Daubert, 509 U.S. at 589-90.

[39] Observation, statement of proposition to be tested, testing plan devised, selection of methods, testing, interpretation of results to reach hypothesis, generation of predictions based on hypothesis, conclusions based on further testing …all of which can be re-tested for accuracy or reproduced by other experts in the field to arrive at the same result. See John R. Platt, Strong Interference, Science, Oct. 16, 1964 at 347-53; see also Edward J. Imwinkelried, The Next Step After Daubert: Developing a Similarly Epistemological Approach to Ensuring the Reliability of Nonscientific Expert Testimony, 15 Cardozo L. Rev. 2271, 2283-84 (1994).

[40] See generally, K. Issac deVyver, Comment, Opening the Door but Keeping the Lights Off: Kumho Tire Co. v. Carmichael and the Applicability of the Daubert Test to Nonscientific Evidence, 50 Case W. Res. L. Rev. 177 (1999).

[41] Kumho, 526 U.S. at 149-50.

[42] See Kristina L. Needham, Note, Questioning the Admissibility of Nonscientific Testimony After Daubert: The Need for Increased Judicial Gatekeeping to Ensure the Reliability of All Expert Testimony, 25 Fordham Urb. L.J. 541, at 550 (1998); Lynn R. Johnson, et al., Expert Testimony in Federal Court: Frye, Daubert, and Joiner, A.L.I.-A.B.A. Course of Study Materials, SC33, Feb. 12-13, 1998 (reviewing the application of Daubert to nonscientific evidence by appellate courts).

[43] See generally, Leon E. White, Article, Maladjusted Contrivances and Clumsy Automation: A Jurisprudential Investigation, 9 Harv. J.L. & Tech 375 (1996).

[44] See, e.g., Campagna v. Hill, 385 N.Y.S.2d 894, 895 (N.Y. App. Div. 1976). Although this is an antiquated case, the New York State Supreme Court determined that a father was liable for support payments based on computer data that alleged him to be in arrears, despite contrary proof of payment offered in defense. Although the case was reversed, it illustrates the dangers of allowing this aura of infallibility to proceed unabated in judicial decisions.

[45] See United States v. Clonts, 966 F.2d 1366 (10th Cir. 1992).

[46] This example is included merely to demonstrate that its obvious, physical uniqueness distinguishes it from digital evidence in terms of susceptibility to alteration from a chain of custody perspective. The issue of authenticity is a separate one, in which case the degree of proof in an art fraud counterfeit claim would more closely resemble the stricter scrutiny threshold that should be considered for many types of digital evidence.

[47] See generally, Jerome J. Roberts, A Practitioner’s Primer on Computer-Generated Evidence, 41 U. Chi. L. Rev. 254, 256 (1974).

[48] 30B Michael H. Graham, Federal Practice and Procedure § 6830 (1975); see also, People v. Lugashi, 252 Cal. Rptr. 434 (Cal. Ct. App. 1988) (presuming a data collection software program accurate); People v. Mormon, 422 N.E.2d 1065, 1073 (1981) (presuming a data retrieval program accurate).

[49] Lugashi, 205 Cal. Rptr. 454; Mormon, 422 N.E.2d at 1073.

[50] See generally, Donald Zupanec, Annotation, Admissibility of Computerized Private Business Records, 7 A.L.R. 4th (1998) (discussing state and federal cases in which courts have considered whether, and under what circumstances, computerized private business records are admissible as evidence in civil and criminal proceedings).

Computer-derived evidence may be admitted via the stipulation of both sides to a case. Oftentimes stipulation occurs when both sides decide to accept or decline to challenge the evidence, whether it comes from ignorance of the technical challenges or disinclination to make a technical defense over a substantive one. Whatever the motivation, this does nothing to enhance the reliability of the evidence or establish precedent to guide other courts and litigants.

Aside from judicial notice of automated processes, the evidentiary challenges to computer-derived evidence have gone to the weight of the evidence, rather than its admissibility. Thus, the evidence may be allowed to go before a jury, but its reliability is contested by attacking the chain-of-custody (human handling) of the digital data. For instance, the computer forensic practices involved in the identification, collection, preservation, and analysis of the digital data are favorite targets for those seeking to discredit evidence.

[51] Burleson v. State, 802 S.W.2d 429 (Tex. Ct. App. 1991).

[52] Fred Galves, Where the Not So Wild Things Are: Computers in the Courtroom, the Federal Rules of Evidence, and the Need for Institutional Reform and More Judicial Acceptance, 13 Harv. J. L. & Tech. 161, 230 (2000).

[53] See id. at 229.

[54] See id.

[55] See generally, Donal Zupanec, supra note 49.

[56] See, e.g., Edward Hannan, Computer Generated Evidence: Testing the Envelope, 63 Def. Couns. J. 353, 358 (1996)(listing ways to authenticate computer generated evidence).

1. Sources of input data are accurate, reliable, trustworthy in own right (i.e., physical measurements);

2. Assumptions used to quantify non-measured items are reasonable, consistent with laws of nature and bracketed at the upper and lower ends;

3. Commercially recognized hardware is employed;

4. Commercially recognized software used that has capacity of executing applications as intended and subject to appropriate input controls, processing controls, output controls;

5. No relevant data have been overlooked;

6. Data inputted, processed, retrieved by properly trained and supervised technicians.

See also Manual for Complex Litigation (Second) §§ 21.446, at 61-61 (1985) (providing that discovery into the reliability of computerized evidence, "include[ing] inquiry into the accuracy of the underlying source materials, the procedures for storage and processing, and some testing of the reliability of the results obtained,” should be conducted "well in advance of trial” THE MANUAL recommends that the proponent must establish, to the court's satisfaction under Uniform Rule of Evidence 104(a), the reliability of the computer equipment used and the data processing techniques applied. This foundation would include expert testimony that the processing programs accurately process the information in the business record database).

[57] Monarch Fed. Sav. & Loan Ass’n v. Genser, 383 A.2d 475 (N.J. Super. Ct. Ch. Div. 1977).

[58] See, e.g., Lugashi, 252 Cal. Rptr. 434 (noting that there was no need to require testimony on the reliability of hardware and software of particular computer, or internal maintenance and accuracy tests. But, in that case, there was other, non-digital evidence confirming the claims); see generally, Galves, supra note 51 at 231.

[59] Various business records exceptions have been codified in various forms, including: FED R. EVID. 803(6) and A.L.I. Model Code of Evid. 514(1).

The following is not excluded by the hearsay rule:

A memorandum, report, record, or data compilation, in any form, of acts, events, conditions, opinions, or diagnoses, made at or near the time by, or from information transmitted by, a person with knowledge, if kept in the course of a regularly conducted business activity, and if it was the regular practice of that business activity to make the memorandum, report, record or data compilation, all as shown by the testimony of the custodian or other qualified witness … unless the source of information or the method or circumstances of preparation indicate a lack of trustworthiness.

FED R. EVID. 803(6) (emphasis added).

[60] 22 Charles Alan Wright & Kenneth W. Graham, Jr., Federal Practice and Procedure § 5174.1 (Supp. 1998).

[61] “C]omputer data compilations are admissible as business records under Fed. R. Evid. 803(6) if a proper foundation as to the reliability of the records is established." United States v. Briscoe, 896 F.2d 1476, 1494 (7th Cir. 1990) (citing United States v. Croft, 750 F.2d 1354 (7th Cir. 1984)). A survey of federal circuit court decisions reveals that all but three circuits – the Second, Third, and D.C. Circuits – have admitted computer-based documents under Fed. R. Evid. 803(6). See United v. Fendley, 522 F.2d 181 186-187 (5th Cir. 1975); United States v. Russo, 480 F.2d 1228, 1240-41 (6th Cir. 1973), cert. denied, 414 U.S. 1157 (1974); United States v. Vela, 673 F.2d 86, 89-90 (5th Cir. 1982); United States v. Cestnik, 36 F.3d 904, 909-10 (10th Cir. 1994), cert. denied, 513 U.S. 1175 (1995); Ameropan Oil Corp. v. Monarch Air Serv., No. 92 C 3450, 1994 WL 86701, at *3-4, 1994, U.S. Dist. LEXIS 3111 (N.D. Ill. Mar. 14, 1994). See generally, Anthony J. Dreyer, NOTE, When the Postman Beeps Twice: The Admissibility of Electronic Mail Under The Business Records Exception of the Federal Rules Of Evidence, 64 Fordham L. Rev. 2285 (1996).

Advisory committee's note to FRE 803(6) points out that “the expression ‘data compilation’ is used as broadly descriptive of any means of storing information other than conventional words and figures in written or documentary form. It includes, but is by no means limited to, electronic computer storage.” The Senate Committee's report on the Federal Rules of Evidence also implicitly recognized that computer-based records fall under the Rule. FED R. EVID. 803(6).

[62] See, e.g., City of Bellevue v. Lightfoot, 877 P.2d 247 (Wash. Ct. App. 1994).

[63] See, e.g., State v. Dunn, 7 S.W.3d 427(Mo. Ct. App. 1999). See People v. Holowko, 486 N.E.2d 877, 878-79, 109 Ill. 2d 187, 93 Ill. Dec. 344 (1985); see also State v. Armstead, 432 So. 2d 837, 839-41 (La. 1983) (holding that computerized records of phone traces were not hearsay and that such records were computer-generated data to be distinguished from computer-stored declarations).

[64] People v. Gauer, 288 N.E.2d 24, 7 Ill. App. 3d 512 (1972).

[65] Croft, 750 F.2d at 1367.

[66] A “bug” is an error in coding or product design that causes computer hardware or software to behave unexpectedly or crash. See generally http://www.bugnet.com.

[67] “Evidence that a matter is not included in the memorandum, reports, records, or data compilations, in any form, kept in accordance with the provisions of paragraph (6), to prove the nonoccurrence or nonexistence of the matter, if the matter was of a kind of which a memorandum, report, record, or data compilation was regularly made and preserved, unless the sources of information or other circumstances indicate lack of trustworthiness.” FED R. EVID. 803(7).

[68] See, generally, Zupanec, supra note 50 at §§ 3, 7[a]; U.S. v. DeGeorgia, 420 F.2d 889 (1969) (finding that it is immaterial, as far as admissibility is concerned, whether a business record is maintained in a computer rather than company books as long as (1) the opponent is given the same opportunity to inquire into the accuracy of the computer and the input procedures used as he would have to inquire into the accuracy of written business records, and (2) the trial court requires the party offering the computer record to provide a foundation that is sufficient to warrant a finding that the record is trustworthy).

[69] See Howloko, 109 Ill. 2d 187, 191.

[70] Mark A. Dombroff, Dombroff on Demonstrative Evidence § 2.27 at 50 (1983) (stating that demonstrative evidence “[is] used to inform the trier of fact of scenes, places, objects and other pertinent data relative to the issues in the litigation that, for numerous reasons, cannot be described with as much force and effect without the use of those aids. The use of diagrams and other visual aids is justified on the grounds that they represent a pictorial reproduction or communication of the senses that may be used in place of descriptive testimony, or simply to supplement such testimony.”).

[71] Facts or data relied on by experts need not otherwise be admissible into evidence if the information is “of a type reasonably relied upon by experts in the particular field.” FED R. EVID. 703.

[72] Perma Research and Dev. v. Singer, 542 F.2d 111 (2d Cir. 1976), cert. denied, 429 U.S. 987 (1976).

[73] Id. (citing Roberts, A Practitioner’s Primer on Computer-Generated Evidence, 41 U. Chi. L. Rev. 254, 255-256 (1974), (“The possibility of an undetected error in computer-generated evidence is a function of many factors: the underlying data may be hearsay; errors may be introduced in any one of several stages of processing; the computer might be erroneously programmed, programmed to permit an error to go undetected, or programmed to introduce error into the data; and the computer may inaccurately display the data or display it in a biased manner. Because of the complexities of examining the creation of computer-generated evidence and the deceptively neat package in which the computer can display its work product, courts and practitioners must exercise more care with computer-generated evidence than with evidence generated by more traditional means.”)).

[74] Id. at 115.

[75] Internet Information Server (Microsoft's proprietary webserver software).

[76] The Federal Rules of Evidence provide that relevant evidence should be admitted if its probative value is not outweighed by prejudice, potential to mislead the jury, or excessive consumption of time. See, FED. R. EVID. 401-403.

[77] See also Haley v. Pan Am. World Airways, 746 F.2d 311 (5th Cir. 1984) (admitting a videotaped simulation of the last moments of Flight 759, before its crash in Louisiana on July 9, 1982, as evidence of pre-impact fear on behalf of the decedent), reh'g denied, 751 F.2d 1258 (5th Cir. 1984).

[78] See generally Zupanec, supra note 50, at §10[a], 11[a].

[79] U.S. v. Downing, 753 F.2d 1224, 1240 n.21 (3^rd Cir. 1985).

[80] State of Washington v. Leavell (Okanogan County Cause no. 00-1-0026-8, Telephonic Suppression Hearing, October 20, 2000) (calling for an evidentiary hearing under the Frye standard, which was the predecessor to Daubert and still remains in effect in a number of states).

[81] Although courts have not explicitly used the term “closed source” to refer to “standard” computer programs, there is an implicit assumption that they are interchangeable. At this point in time, the features describing “commercial” and “standard” software are equivalent to “closed source” software. Also, courts have not been presented with issues involving “Open Source” software, and therefore would have no reason to introduce this terminology into their decisions.

[82] 71 Am. Jur. Trials 111 & 118 (1999).

[83] See, e.g., Ted Bridis, Microsoft Acknowledges Its Engineers Placed Security Flaw in Some Software, Wall St. J., April 14, 2000, available at http://www.slashdot.org/articles/00/04/14/0619206.shtml (reporting, “Microsoft Corp. acknowledged Thursday that its engineers included in some of its Internet software a secret password -- a phrase deriding their rivals at Netscape as ‘weenies’ -- that could be used to gain illicit access to hundreds of thousands of Internet sites world-wide. [...]”).

[84] See generally infra note 92.

[85] Interview with Thomas Oliver, Security Manager at SAIC (Jan. 15, 2001); series of interviews with Tom Perrine, Manager of Networking and Security at the San Diego Supercomputer (Fall 2000).

[86] See generally Kenneally, supra note 28; Cem Kaner and David Pels, Bad Software (1998).

[87] See, Title, available at http://www.netaction.org/opensrc/(date) (offering a qualitative listing of Open Source commentary and leading projects).

[88] Trust in Cyberspace, supra note 11 at 72.

[89] See Alan Cox, The Risks of Closed Source Computing, available at http://www.osopinion.com/Opinions/AlanCox/AlanCox1.html; interview with Thomas Oliver, Security Manager at SAIC (Jan. 15, 2001).

[90] Trust in Cyberspace, supra note 11 at 72.

[91] Id.

[92] See generally, Impact of the New Federal Rules of Discovery on Evidence Practice, CONTINUING EDUCATION OF THE BAR CALIFORNIA, Business Professional's Network, B-119-154 (January 26, 2001).

[93] See U.S. v. Tank, 200 F.3d 627 (9th Cir. 2000); Wisconsin v. Schroeder 613 N.W.2d 911 (2000).

[94] See generally, CERT/CC- Computer Emergency Response Team / Coordination Center, at http://www.cert.org/; Security Alert for Enterprise Resources, at http://www.safermag.com/; Security Focus, at http://www.securityfocus.com; Bugtraq, at http://www.bugtraq.securepoint.com; SANS Security Digest Services, at http://www.sans.org/newlook/digests/SAC.htm ; Attrition, at http://www.attrition.org; Microsoft Technical Updates, Microsoft Security Bulletins, at http://www.microsoft.com/technet/security.

[95] See, e.g., Peter Sommer, Computer Forensics- An Introduction, http://www.virtualcity.co.uk/vcaforens.htm (last visited March 2, 2001).

[96] Trust in Cyberspace, supra note 11 at 67.

[97] Id. at 65, 67.

[98] Id.

[99] See Sommer, supra note 94.

[100] See Raymond, supra note 7 at 17-18.

[101] See, e.g., IEEE Syslog Project, Institute of Electrical and Electronics Engineers (IEEE) (visited June 3, 2000) http://standards.ieee.org/reading/ieee/std/se/12207.0-1996.pdf; See, generally, http://www.dacs.dtic.mil/databases/url/key.hts?keycode=2:2450&islowerlevel=1 for a list of education, training and conferences aimed improving software reliability. Most notably is ISACC (International Software Assurance Certification Conference), which aims to “bring together the best minds in the nation to address the growing concerns with software quality, reliability, and security” http://www.cigital.com/ISACC99/mission.html.

[102] Note, FRE 901(b) consists of illustrations of ways to authenticate evidence. These are only illustrative and not meant to limit the ways authentication can be accomplished. USCS FRE R. 901 (2000) Article IX. Authentication and Identification.

[103] See § II.B.4.a.ii (discussing black box testing), supra.

[104] See § II.B.3.d.iii, supra.

[105] Raymond, supra note 7 at 20. See generally Bruce Perens, The Open Source Definition, in Open Sources: Voices from the Open Source revolution, O’Reilly and Associates, Inc. (1999) http://eon.law.harvard.edu/property00/alternatives/reading4.htm for a more thorough analysis of the Open Source definition and model).

[106] See § II.A.1.b (discussing interoperability, error recovery (bug avoidance and fixing), efficiency, quality, etc.), supra.

[107] See Jonathan Eunice, Beyond the Cathedral and the Bazaar (May, 1998) http://www.illuminata.com/public/content/cathedral/intro.htm.

[108] See David Goodstein, "How Science Works" REFERENCE MANUAL ON SCIENTIFIC EVIDENCE 2nd Edition, Federal Judicial Center (2000) http://air.fjc.gov/public/fjcweb.nsf/pages/74 (viewed February 23, 2001).

[109] Id.

[110] See, e.g., Bruce Schneier, Are We Ready for a Cyber-UL?, ZDNet News Commentary (January 2, 2001) http://www.zdnet.com/zdnn/stories/comment/0,5859,2669708,00.html.

[111] See Oliver Cole, White-Box Testing, DR. DOBB'S JOURNAL (March 2000) http://www.ddj.com/articles/2000/0003/0003a/0003a.htm; see generally, Boris Beizer and Van Nostrand Reinhold, SOFTWARE TESTING TECHNIQUES, Second Edition, (1990) ISBN 1850328803

[112] Id.

[113] Id.

[114] Id. See also Murray Wood, et al., Comparing and Combining Software Defect Detection Techniques: A Replicated Empirical Study, PROCEEDINGS OF THE 6TH EUROPEAN CONFERENCE HELD JOINTLY WITH THE 5TH ACM SIGSOFT SYMPOSIUM ON SOFTWARE ENGINEERING (1997).

[115] See § II.B.3.c, infra.

[116] Chesterene Cwiklik, Evaluating Significance and Validity When There Are No Hard Numbers, NATIONAL CONFERENCE ON SCIENCE AND THE LAW, San Diego, CA (October 11, 2000) at 5.

[117] The author bases this statement on the cumulative discussions with computer science professionals both at the San Diego Supercomputer Center and within industry.

[118] See http://www.ceu.fi.udc.es/opensource.org-mirror/docs/products.html.

[119] Reliability testing here was as measured by occurrence of hangs (loops indefinitely) or crashes with a core dump of Open Source GNU and Linux software as compared to commercial software. Specifically, the failure rate of public GNU utilities was lowest in study at 6% versus 15-43% for utilities on commercial versions of UNIX from 5 vendors. ftp://grilled.cs.wisc.edu/technical_papers/fuzz-revisited.pdf .

[120] See Todd Weiss, Microsoft to Expand Windows source-code sharing program, COMPUTERWORLD (February 2, 2001) http://www.computerworld.com/cwi/stories/0,1199,NAV47-68-84-88-93_STO57316,00.html.

[121] See Recommendations of the Panel on Open Source Software for High End Computing, PRESIDENT’S INFORMATION TECHNOLOGY ADVISORY COMMITTEE (September, 2000). See, generally, Marcus Maher, Open Source Software: The Success of an Alternative Intellectual Property Incentive Paradigm, 10 FORDHAM I.P., MEDIA & ENT. L.J. 619 (Spring 2000).

[122] See § II.A.1.b, supra.

[123] FED. R. EVID 702: If scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training, or education, may testify thereto in the form of an opinion or otherwise, if (1) the testimony is based upon sufficient facts or data, (2) the testimony is the product of reliable principles and methods, and (3) the witness has applied the principles and methods reliably to the facts of the case.

[124] See, e.g., U.S. v. Jones, 107 F.3d 1147 (6th Cir. 1997) (admitting handwriting examiner testimony based on years of practical experience, training, and his detailed explanation of his methodology); Tassin v. Sears Roebuck, 946 F.Supp. 1241, 1248 (M.D.La. 1996) (admit ting testimony of design engineer when opinion based on facts, reasonable investigation, traditional technical/mechanical expertise, and a reasonable link between the information and conclusions).

[125] Kumho, 526 U.S. at 1178.

[126] Samuel Guiberson, Panel IV. Scientific and Demonstrative Evidence, Is Seeing Believing? PROCEEDINGS- NATIONAL CONFERENCE OF SCIENCE AND THE LAW at 163 (July 2000).

[127] Lugashi, 205 Cal. App. 3d 632.

[128] Id. at 640.

[129] Id.

[130] Id.

[131] See U.S. v. Tank, 200 F.3d 627 (9th Cir. 2000); Wisconsin v. Schroeder, 2000 WL 675942.