Inside The Black Box: Excluding Evidence Generated by Algorithms

[Editor’s Note: John C. Ellis, Jr. is a National Coordinating Discovery Attorney for the Administrative Office of the U.S. Courts, Defender Services Office. In this capacity, he provides litigation support and e-discovery assistance on complex criminal cases to defense teams around the country. Before entering private practice, Mr. Ellis spent 13 years as a trial attorney and supervisory attorney with Federal Defenders of San Diego, Inc. He also serves as a digital forensic consultant and expert.]

For many years, law enforcement officers have used records generated by mobile carriers to place a mobile device in a general area. The records are called Call Detail Records (“CDRs”). CDRs are generated when a mobile device sends or receives calls and text messages. Mobile carriers likewise keep records of when data is used, such as browsing the internet. These records are called Usage Detail Records (“UDRs”). At times, the records generated by mobile carriers include the location of the cell site or cell sites and the direction of antenna that connected with the mobile device.

Cell Site Location Information (“CSLI”) is the practice of creating maps showing the possible coverage area of a cell site at the time a device was being used. For these purposes, it is important to keep in mind that the records only show the location of the cell site and the direction the antenna is facing. Recent technological improvements have resulted in mobile carriers now generating Enhanced Location Records (“ELRs”), which purport to show more precise location data. In AT&T parlance, such records are based on the Network Event Location System (“NELOS”). This location data is derived from proprietary algorithms.

In a recent federal case, the government, through a member of the Federal Bureau of Investigation’s (“FBI”) Cellular Analysis Survey Team (“CAST”), sought to introduce NELOS records in a trial. However, after a Daubert hearing where the CAST agent testified, the district court excluded the records, in part, because of concerns over the reliability of the algorithms used to determine the location data.

This article provides an overview of CSLI and NELOS records, discusses the order excluding NELOS records from trial, and provides practical advice for practitioners.

Overview:

When CDRs include cell site location data, analysts and law enforcement officers use these records to show the location of the cell site and the orientation of the sector. In North America, many cell towers contain three sets of antennas, with each set offering specific coverage area.

Picture 1

To illustrate this point, Picture 1 is an overview picture of a multi-directional cell tower. Each blue arm is a sector. When a mobile device connects to a cell site, the mobile carrier often records the activity (i.e., a sent text message), the time of the activity, and the location of the cell site and sector that was used.

Using these three data points, analysists and law enforcement officers create maps showing the location of the cell site and the orientation of the sector. In Map 1, the arms are used to demonstrate the beamwidth of the sector, which in this case records indicate is 120-degrees. The cone at the base of the triangle is only meant to show the orientation of the sector, not coverage area. Moreover, analysts generally will not testify that the mobile device was within the triangle. The triangle is only meant to represent the location of the cell site and the orientation of the sector.

Map 1

With NELOS records, on the other hand, the ELRs purport to show the location of a device as opposed to the location of the cell site. In the following example, the red pin represents the location of the device. The blue circle represents what AT&T calls the “Location Accuracy.” This accuracy ranges from approximately several meters to 10,000 meters. And some records are marked by “location accuracy unknown.” As discussed below, the Location Accuracy is determined by proprietary algorithms used by AT&T.

Map 2

In Map 2, the ELR indicates that the “[l]ocation accuracy [is] likely better than 300 meters.” In other words, the phone was at the red pin or within the blue circle at a specific date and time. NELOS records, however, contain the following statement: “The results provided are AT&T’s best estimate of the location of the target phone. Please exercise caution in using these records for investigative purposes, as location data is sourced from various databases, which may cause the location results to be less than exact.” DE 156 at 23 (emphasis added).

To put the first two examples into perspective, Map 3 shows both traditional CSLI and the use of NELOS records.

Map 3

The NELOS demonstrative, even taking account of the “Location Accuracy,” still provides a much smaller, and thus more specific, area of where the phone activity took place.

United States v. Smith, et al. (4:19-CR-514-DPM) (EDAR):

Donald Smith and Samuel Sherman were charged in a five-count indictment with various crimes relating to a murder. See Docket Entry (“DE”) 1. The government sought to introduce the testimony of CAST Agent Mark Sedwick “that provider-based location data typically is collected by obtaining historical call detail records for a particular cellular telephone from the service provider, along with a listing of the cell tower locations for that service provider.” DE 102 at 1. According to the government, “[t]his data is then analyzed for the purpose of generally placing a cellular telephone at or near an approximate location or locations on a map at points in time.” Id.

The government sought to have Agent Sedwick testify “regarding the activity and approximate locations of the cellular telephones believed to have been utilized by Donald Bill Smith, Samuel Sherman, Racheal Cooper and Susan Cooper on the approximate dates and times relevant to the charges in the Indictment.” Id. at 1-2. Attached to the government’s motion is the report created by Agent Sedwick. Maps 4 and 5 are examples from Agent Sedwick’s report. Map 4 shows how Agent Sedwick mapped traditional CSLI, and Map 5 shows how he mapped the same time period using NELOS records:

 

Map 4
Map 5

Map 4 shows traditional CSLI mapping with the location of the cell site and the orientation of the sector. With Map 5, each circle represents the area in which the device was used. Here, there are four such events. For comparison, in Map 4, Agent Sedwick’s opinion is limited to testifying about the location of the cell site and the orientation of the sector, whereas with Map 5, the testimony is the mobile device is within the circle.

Prior to trial, defense counsel challenged Agent Sedwick’s potential testimony and the district court conducted a hearing to determine the admissibility of the records pursuant to Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 US 579 (1993). During the hearing, Agent Sedwick explained the reason AT&T created NELOS was to “test the health of the 3G network for planning and troubleshooting. It is a passive system where, while the phone is on the control channel communicating with the network across the control channel, it would passively pull whatever location data it could pull or data to compute location from that device.” DE 156 at 8.

Agent Sedwick further explained: “NELOS also became the generic term for any kind of location data. So depending, there might be other databases that were also pulled into the NELOS report that we receive from AT&T. Just from that report there’s no way to determine what other databases that was pulled from.” DE 156 at 9.

Agent Sedwick also provided information about known issues with NELOS data, specifically based on Temporary Mobile Subscriber Identity (“TMSI”). By way of background, mobile devices are assigned an International Mobile Subscriber Identity (“IMSI”), a unique number used by mobile carriers, which establishes that the mobile device can operate on a specific network. This is the number used by mobile carriers when creating CDRs. At times, however, in order to mask a device’s actual IMSI, networks assign the device a TMSI.[1] This is problematic for NELOS records because as Agent Sedwick explained, “[t]hat TMSI sometimes can get reallocated and then allocated back to a device, so you can have sometimes where the NELOS data will pull from a different device and get put into the records for the device that you’re requesting.” DE 156 at 10.

During cross-examination, Agent Sedwick was questioned about the portion of NELOS records that “caution in using these records for investigative purposes.” Agent Sedwick responded: “I wouldn’t rely on it if all I had was a NELOS point putting someone at a scene and that’s all I had, no, I would not use it. I’m using it—there is a caution with it, but I’m using it in the context of I have call and text to support it, I have other data to support, I have very good precise NELOS data. I feel very, very confident that this is accurate.” DE 156 at 24.

Agent Sedwick’s confidence in the accuracy of NELOS records was based on the proprietary algorithms created by the phone company. See DE 156 at 12 (“Question: Okay. So the device is sending various different events, they’re plugged into that algorithm, and essentially the algorithm will spit out what it computes as accuracy; is that correct? Answer: Yes, ma’am”). But Agent Sedwick acknowledged that he was not privy to the algorithm, nor whether NELOS was tested by AT&T for reliability. Instead, Agent Sewick testified he believed the algorithms are reliable “[b]ecause AT&T relies on that to make multi-million-dollar decisions on how they’re going to design their network.” DE 156 at 32.

In granting the defense’s motion to exclude NELOS data, the district court found:

What particularly concerns me, though, is this mystery algorithm that our—and the proprietary software. We don’t know, I don’t know exactly what is in the algorithm, and the agent gave some testimony at a general level about the kind of information that goes in, but it seems to me that I’m missing a—an important foundational stone there of something with more specificity as to the kinds of things that the algorithm uses and how the algorithm does its work.

We know that there are disturbances from time to time, or anomalies as was called, with the TMSI number. I also—I acknowledge some uncertainty about TMSI numbers and how many devices that might be connected with and how it is that the algorithm might deal with that. So there’s that. Then there is, in my view, almost a—so we’ve got our black box there, which is concerning, and I would say at this point there’s a peer review problem, as well, because I don’t have any scholarly literature or evaluation of the black boxes or the kind of things that could go into this black box and how it would work.

I understand about the corroboration, but I still find myself at sea of understanding how it is the—how things happen in the black box and whether—whether what comes out of the black box is sufficiently reliable that the jury can rely on it.

DE 156 at 85-87 (emphasis added).

Based on this, the district court entered the following order: “Agent Sedwick may testify about call detail records and historical cell-site analysis; but he may not testify about NELOS data and analysis.” DE 154.

Further Consideration:

The district court’s exclusion of NELOS records was based, in part, on the use of data generated by untested algorithms. Other mobile carriers also use ELRs, which generate purported location data that are also based on proprietary algorithms similar to NELOS. In seeking to exclude ELRs, as well as other forms of computer-generated data, counsel should encourage courts to question the reliability of evidence created by algorithms that lack independent validation and verification.

Glossary:

Acronym Full Title
CASTCellular Analysis Survey Team
CDRCall Detail Records
CSLICell Site Location Information
ELREnhanced Location Records
IMSIInternational Mobile Subscriber Identity
NELOSNetwork Event Location System
TMSITemporary Mobile Subscriber Identity
UDRUsage Detail Records

[1] As explained by EFF, “upon first connecting to a network, the network will ask for your IMSI to identify you, and then will assign you a TMSI … to use while on their network. The purpose of the pseudonymous TMSI is to try and make it difficult for anyone eavesdropping on the network to associate data sent over the network with your phone.” See https://www.eff.org/wp/gotta-catch-em-all-understanding-how-imsi-catchers-exploit-cell-networks.

E-Discovery: Mobile Forensic Reports

By Sean Broderick and John C. Ellis, Jr.

[Editor’s Note: Sean Broderick is the National Litigation Support Administrator.  He provides guidance and recommendations to federal courts, federal defender organization staff, and court appointed attorneys on electronic discovery and complex cases, particularly in the areas of evidence organization, document management and trial presentation. Sean is also the co-chair of the Joint Working Group on Electronic Technology in the Criminal Justice System (JETWG), a joint Department of Justice and Administrative Office of the U.S. Courts national working group which examines the use of electronic technology in the federal criminal justice system and suggested practices for the efficient and cost-effective management of post-indictment electronic discovery. 

John C. Ellis, Jr. is a National Coordinating Discovery Attorney for the Administrative Office of the U.S. Courts, Defender Services Office. In this capacity, he provides litigation support and e-discovery assistance on complex criminal cases to defense teams around the country. Before entering private practice, Mr. Ellis spent 13 years as a trial attorney and supervisory attorney with Federal Defenders of San Diego, Inc. He also serves as a digital forensic consultant and expert.]

Most federal criminal cases involve discovery that originally came from a cell phone. CJA panel attorneys and Federal Defenders have now become accustomed to receiving “reports” generated from Cellebrite.[1] In this blog post, we will talk about the valuable information that may be contained in those Cellebrite generated reports and what form of production you can get the reports in. Spoiler alert: we suggest you request that you receive those reports in Cellebrite Reader format and not just default to the PDF format that you know and love.

We are going to cover:

  1. the basic concepts behind the forensic process that law enforcement uses when using Cellebrite UFED to extract information from a phone,
  2. what is a Cellebrite generated mobile forensic report (which Cellebrite calls extraction reports), and
  3. the pros and cons for the potential formats you can receive Cellebrite generated reports in.

Though there are a number of forensic tools that law enforcement may use to extract data from a phone, the most common is Cellebrite. We are going to discuss Cellebrite, but know there are others (e.g. Oxygen, Paraben, etc.). Many of the processes and principles that apply to Cellebrite will apply to other tools.

Basic concepts behind the forensic process

How does a digital forensic examiner get the data from the mobile phone? Extracting data from mobile devices (a.k.a. acquisition) is complex and requires a great amount of skill when done correctly. For purposes of this blog post, we are only going to focus on one concept, which is the type of extraction that was performed. In order to retrieve data from a mobile phone, an examiner attaches the mobile phone to a computer which has the Cellebrite UFED software, follows a series of protocols, and saves a portion of the data on an external storage device. In most cases, examiners will not retrieve all data that was on the mobile phone at the time of the extraction—this is based in part on the phone’s memory architecture. Moreover, the type of extraction that is performed on the device can limit the amount of data that is retrieved.

The following are the most common types of extractions for Android devices: (1) Logical (or Advanced Logical); (2) File System; and (3) Physical. As for Apple, the most common types are Logical (Partial) and Advanced Logical. Generally, physical extractions retrieve the most data. After the iPhone 4, physical extractions are currently no longer available with Cellebrite with an iPhone device.

After a digital forensic examiner does an extraction of a phone (for this example, we will assume that the extraction was done through the Cellebrite UFED4PC), it generates an extraction files/folders, along with a .UFD (text) file that tells Cellebrite Physical Analyzer basic information about the extraction (such as which UFED was used, start and finish time, and hash information). The extraction files can be produced in a number of formats (.zip and .bin are common examples) depending on the type of extraction done. The takeaway here is that the type of extraction impacts the type and volume of data that was retrieved during the extraction process.

What is a Cellebrite generated report?

After extracting the data, the examiner uses Cellebrite Physical Analyzer to review the data retrieved from the mobile phone. The examiner also has the option of generating a report, which allows users without specialized forensic software to view the data retrieved from the mobile phone. As discussed below, the “extraction report” may be produced in multiple formats. Of note, the examiner can apply filters to decide what data types to export (e.g. emails, images, instant messages, searched items, etc.), and can further filter the data by date range. These reports are limited to the data extracted from the original device; the parameters of the forensic program dictated by the forensic examiner. The takeaway here is that a report does not necessarily include all data that was retrieved during the extraction.

Option for the Cellebrite generated report (extraction report)

Cellebrite generated reports, like the extractions described above, contain information from the mobile phone. This may include text messages, emails, call logs, web browsing history, location data, etc. They can be produced in a number of formats, though the most common are .PDF, .HTML, and .UFDR. There are pros and cons for each format of report.

PDF

Report in PDF format

There are several pros to receiving a Cellebrite generated report in PDF. CJA panel attorneys and Federal Defender defense teams are used to working PDFs. It is easy to add Bates stamps to them. They work on Macs. And they can be annotated and highlighted.

But there are also several important cons that make PDF a less desirable file type for Cellebrite generated reports. For instance, because phones have the capacity to contain large volumes of data, the reports generated from extractions can be quite large. A Cellebrite generated PDF report can easily reach 10,000 pages, which can cause a computer to slow down or even crash. Moreover, users cannot sort or filter data, hide data fields, or search within search results. In short, although PDFs are a convenient file type, it is not the most useful or efficient format for reviewing these types of reports.

HTML

Report in HTML format

There are several pros to receiving a Cellebrite generated report in the HTML format. The files load fast and can be viewed in any browser (such as Chrome, Firefox or Safari). In this format, each data type, such as SMS Messages, are hyperlinked and open in a new browser. (Please note that the hyperlinks only work if the file and the data are provided with the HTML file which can easily get overlooked when people move data.) Moreover, it is easy to search within HTML files and they operate on Macs.

But like PDFs, HTML files have several notable cons. First, you cannot sort or filter the data. Nor can you hide data fields. And you cannot easily generate reports for other subsets of information. Although HTML files are easy to use, they have significant limitations when it comes to reviewing reports.

UFDR

Report in UFDR format

The best format for receiving Cellebrite generated reports is the Cellebrite Reader format. The Cellebrite Reader format allows a user to create reports containing all data, or a portion thereof, in multiple formats including PDF, HTML and UFDR. So, if you receive if in UFDR format you can easily convert it to PDF or HTML later on (which is not possible if you receive it in HTML or PDF). Additionally, in this file format, users can sort and filter data, can search within results, can move or reorder data within columns, and can create tags—which is a convenient way to organize large volumes of discovery. And a user can open multiple UFDR files at the time and search across them. This allows a user to, amongst other things, search for keywords across multiple devices simultaneously.

The one downside to UFDR files is that they will not work on a Mac. You also need to have the free Cellebrite Reader program to open and use the UFDR file. Overall, this is the format you should request when speaking to the government about what form you would like reports generated from Cellebrite produced in.

Final note about formats: When deciding about your preferred format to review a Cellebrite generated report, remember that it is easy for an examiner to select all three formats at the same time. Often, an examiner will provide all three to make it easier for people to review the data in the way they want.

Conclusion

Mobile forensic reports are a ubiquitous part of discovery. When reviewing them, it is important to remember that the information in the report is limited by the limitations of retrieving data from mobile devices, the type of extraction performed on the device, and the data the examiner decided to include in the report. And the form of production of the report can affect how you review the data. Attorneys should consider contacting an expert or consultant if they have questions about the contents of a report.

Of note, Troy Schnack, Computer System Administrator for Federal Public Defender Office in Kansas City, Missouri, will be doing a webinar on mobile devices and will go into detail regarding Cellebrite Reader on Tuesday, September 22, 2020. Please register for the program on fd.org – we highly recommend it.


[1] Cellebrite UFED is a mobile forensic software program that allows trained users to extract and analyze phone call history, contact information, audio, photos, and videos and texts from mobile phones or forensic images of mobile devices produced as part of discovery. It has wide coverage for accessing digital devices from Android to Apple, with more than 31,000 device profiles of the most common phones. Cellebrite UFED can come as software only or can include a physical unit with accessories such as tip and cable set to connect to various mobile devices.