Essex Police has not properly considered the potentially discriminatory impacts of its live facial recognition (LFR) use, according to documents obtained by Big Brother Watch and shared with Computer Weekly.
While the force claims in an equality impact assessment (EIA) that “Essex Police has carefully considered issues regarding bias and algorithmic injustice”, privacy campaign group Big Brother Watch said the document – obtained under Freedom of Information (FoI) rules – shows it has likely failed to fulfil its public sector equality duty (PSED) to consider how its policies and practices could be discriminatory.
The campaigners highlighted how the force is relying on false comparisons to other algorithms and “parroting misleading claims” from the supplier about the LFR system’s lack of bias.
For example, Essex Police said that when deploying LFR, it will set the system threshold “at 0.6 or above, as this is the level whereby equitability of the rate of false positive identification across all demographics is achieved”.
However, this figure is based on the National Physical Laboratory’s (NPL) testing of NEC’s Neoface V4 LFR algorithm deployed by the Metropolitan Police and South Wales Police, which Essex Police does not use.
Instead, Essex Police has opted to use an algorithm developed by Israeli biometrics firm Corsight, whose chief privacy officer, Tony Porter, was formerly the UK’s surveillance camera commissioner until January 2021.
Highlighting testing of the Corsight_003 algorithm conducted in June 2022 by the US National Institute of Standards and Technology (NIST), the EIA also claims it has “a bias differential FMR [False Match Rate] of 0.0006 overall, the lowest of any tested within NIST at the time of writing, according to the supplier”.
However, looking at the NIST website, where all of the testing data is publicly shared, there is no information to support the figure cited by Corsight, or its claim to essentially have the least biased algorithm available.
A separate FoI response to Big Brother Watch confirmed that, as of 16 January 2025, Essex Police had not conducted any “formal or detailed” testing of the system itself, or otherwise commissioned a third party to do so.
Essex Police’s lax approach to assessing the dangers of a controversial and dangerous new form of surveillance has put the rights of thousands at risk Jake Hurfurt, Big Brother Watch
“Looking at Essex Police’s EIA, we are concerned about the force’s compliance with its duties under equality law, as the reliance on shaky evidence seriously undermines the force’s claims about how the public will be protected against algorithmic bias,” said Jake Hurfurt, head of research and investigations at Big Brother Watch.
“Essex Police’s lax approach to assessing the dangers of a controversial and dangerous new form of surveillance has put the rights of thousands at risk. This slapdash scrutiny of their intrusive facial recognition system sets a worrying precedent.
“Facial recognition is notorious for misidentifying women and people of colour, and Essex Police’s willingness to deploy the technology without testing it themselves raises serious questions about the force’s compliance with equalities law. Essex Police should immediately stop their use of facial recognition surveillance.”
The need for UK police forces deploying facial recognition to consider how their use of the technology could be discriminatory was highlighted by a legal challenge brought against South Wales Police by Cardiff resident Ed Bridges.
In August 2020, the UK Court of Appeal ruled that the use of LFR by the force was unlawful because the privacy violations it entailed were “not in accordance” with legally permissible restrictions on Bridges’ Article 8 privacy rights; it did not conduct an appropriate data protection impact assessment (DPIA); and it did not comply with its PSED to consider how its policies and practices could be discriminatory.
The judgment specifically found that the PSED is a “duty of process and not outcome”, and requires public bodies to take reasonable steps “to make enquiries about what may not yet be known to a public authority about the potential impact of a proposed decision or policy on people with the relevant characteristics, in particular for present purposes race and sex”.
Big Brother Watch said equality assessments must rely on “sufficient quality evidence” to back up the claims being made and ultimately satisfy the PSED, but that the documents obtained do not demonstrate the force has had “due regard” for equalities.
Academic Karen Yeung, an interdisciplinary professor at Birmingham Law School and School of Computer Science, told Computer Weekly that, in her view, the EIA is “clearly inadequate”.
She also criticised the document for being “incoherent”, failing to look at the systemic equalities impacts of the technology, and relying exclusively on testing of entirely different software algorithms used by other police forces trained on different populations: “This does not, in my view, fulfil the requirements of the public sector equality duty. It is a document produced from a cut-and-paste exercise from the largely irrelevant material produced by others.”
Essex Police responds
Computer Weekly contacted Essex Police about every aspect of the story.
“We take our responsibility to meet our public sector equality duty very seriously, and there is a contractual requirement on our LFR partner to ensure sufficient testing has taken place to ensure the software meets the specification and performance outlined in the tender process,” said a spokesperson.
“There have been more than 50 deployments of our LFR vans, scanning 1.7 million faces, which have led to more than 200 positive alerts, and nearly 70 arrests.
“To date, there has been one false positive, which, when reviewed, was established to be as a result of a low-quality photo uploaded onto the watchlist and not the result of bias issues with the technology. This did not lead to an arrest or any other unlawful action because of the procedures in place to verify all alerts. This issue has been resolved to ensure it does not occur again.”
The spokesperson added that the force is also committed to carrying out further assessment of the software and algorithms, with the evaluation of deployments and results being subject to an independent academic review.
“As part of this, we have carried out, and continue to do so, testing and evaluation activity in conjunction with the University of Cambridge. The NPL have recently agreed to carry out further independent testing, which will take place over the summer. The company have also achieved an ISO 42001 certification,” said the spokesperson. “We are also liaising with other technical specialists regarding further testing and evaluation activity.”
However, the force did not comment on why it was relying on the testing of a completely different algorithm in its EIA, or why it had not conducted or otherwise commissioned its own testing before operationally deploying the technology in the field.
Computer Weekly followed up Essex Police for clarification on when the testing with Cambridge began, as this is not mentioned in the EIA, but received no response by time of publication.
‘Misleading’ testing claims
Although Essex Police and Corsight claim the facial recognition algorithm in use has “a bias differential FMR of 0.0006 overall, the lowest of any tested within NIST at the time of writing”, there is no publicly available data on NIST’s website to support this claim.
Drilling down into the demographic split of false positive rates shows, for example, that there is a factor of 100 more false positives in West African women than for Eastern European men.
While this is an improvement on the previous two algorithms submitted for testing by Corsight, other publicly available data held by NIST undermines Essex Police’s claim in the EIA that the “algorithm is identified by NIST as having the lowest bias variance between demographics”.
Looking at another metric held by NIST – FMR Max/Min, which refers to the ratio between demographic groups that give the most and least false positives – it essentially represents how inequitable the error rates are across different age groups, sexes and ethnicities.
In this instance, smaller values represent better performance, with the ratio being an estimate of how many times more false positives can be expected in one group over another.
According to the NIST webpage for “demographic effects” in facial recognition algorithms, the Corsight algorithm has an FMR Max/Min of 113(22), meaning there are at least 21 algorithms that display less bias. For comparison, the least biased algorithm according to NIST results belongs to a firm called Idemia, which has an FMR Max/Min of 5(1).
However, like Corsight, the highest false match rate for Idemia’s algorithm was for older West African women. Computer Weekly understands this is a common problem with many of the facial recognition algorithms NIST tests because this group is not typically well-represented in the underlying training data of most firms.
Computer Weekly also confirmed with NIST that the FMR metric cited by Corsight relates to one-to-one verification, rather than the one-to-many situation police forces would be using it in.
This is a key distinction, because if 1,000 people are enrolled in a facial recognition system that was built on one-to-one verification, then the false positive rate will be 1,000 times larger than the metrics held by NIST for FMR testing.
“If a developer implements 1:N (one-to-many) search as N 1:1 comparisons, then the likelihood of a false positive from a search is expected to be proportional to the false match for the 1:1 comparison algorithm,” said NIST scientist Patrick Grother. “Some developers do not implement 1:N search that way.”
Commenting on the contrast between this testing methodology and the practical scenarios the tech will be deployed in, Birmingham Law School’s Yeung said one-to-one is for use in stable environments to provide admission to spaces with limited access, such as airport passport gates, where only one person’s biometric data is scrutinised at a time.
“One-to-many is entirely different – it’s an entirely different process, an entirely different technical challenge, and therefore cannot typically achieve equivalent levels of accuracy,” she said.
Computer Weekly contacted Corsight about every aspect of the story related to its algorithmic testing, including where the “0.0006” figure is drawn from and its various claims to have the “least biased” algorithm.
“The facts presented in your article are partial, manipulated and misleading,” said a company spokesperson. “Corsight AI’s algorithms have been tested by numerous entities, including NIST, and have been proven to be the least biased in the industry in terms of gender and ethnicity. This is a major factor for our commercial and government clients.”
However, Corsight was either unable or unwilling to specify which facts are “partial, manipulated or misleading” in response to Computer Weekly’s request for clarification.
Computer Weekly also contacted Corsight about whether it has done any further testing by running N one-to-one comparisons, and whether it has changed the system’s threshold settings for detecting a match to suppress the false positive rate, but received no response on these points.
While most facial recognition developers submit their algorithms to NIST for testing on an annual or bi-annual basis, Corsight last submitted an algorithm in mid-2022. Computer Weekly contacted Corsight about why this was the case, given that most algorithms in NIST testing show continuous improvement with each submission, but again received no response on this point.
Homeland Security testing
The Essex Police EIA also highlights testing of the Corsight algorithm conducted in 2022 by the Department of Homeland Security (DHS), claiming it demonstrated “Corsight’s capability to perform equally across all demographics”.
However, Big Brother Watch’s Hurfurt highlighted that the DHS study focused on bias in the context of true positives, and did not assess the algorithm for inequality in false positives.
This is a key distinction for the testing of LFR systems, as false negatives where the system fails to recognise someone will likely not lead to incorrect stops or other adverse effects, whereas a false positive where the system confuses two people could have more severe consequences for an individual.
The DHS itself also publicly came out against Corsight’s representation of the test results, after the firm claimed in subsequent marketing materials that “no matter how you look at it, Corsight is ranked #1. #1 in overall recognition, #1 in dark skin, #1 in Asian, #1 in female”.
Speaking with IVPM in August 2023, DHS said: “We do not know what this claim, being ‘#1’ is referring to.” The department added that the rules of the testing required companies to get their claims cleared through DHS to ensure they do not misrepresent their performance.
In its breakdown of the test results, IVPM noted that systems of multiple other manufacturers achieved similar results to Corsight. The company did not respond to a request for comment about the DHS testing.
Computer Weekly contacted Essex Police about all the issues raised around Corsight testing, but received no direct response to these points from the force.
Key equality impacts not considered
While Essex Police claimed in its EIA that it “also sought advice from their own independent Data and Digital Ethics Committee in relation to their use of LFR generally”, meeting minutes obtained via FoI rules show that key impacts had not been considered.
For example, when one panel member questioned how LFR deployments could affect community events or protests, and how the force could avoid the technology having a “chilling presence”, the officer present (whose name has been redacted from the document) said “that’s a pretty good point, actually”, adding that he had “made a note” to consider this going forward.
The EIA itself also makes no mention of community events or protests, and does not specify how different groups could be affected by these different deployment scenarios.
Elsewhere in the EIA, Essex Police claims that the system is likely to have minimal impact across age, gender and race, citing the 0.6 threshold setting, as well as NIST and DHS testing, as ways of achieving “equitability” across different demographics. Again, this threshold setting relates to a completely different system used by the Met and South Wales Police.
For each protected characteristic, the EIA has a section on “mitigating” actions that can be taken to reduce adverse impacts.
While the “ethnicity” section again highlights the National Physical Laboratory’s testing of a completely different algorithm, most other sections note that “any watchlist created will be done so as close to the deployment as possible, therefore hoping to ensure the most accurate and up-to-date images of persons being added are uploaded”.
However, Yeung noted that the EIA makes no mention of the specific watchlist creation criteria beyond high-level “categories of images” that can be included, and the claimed equality impacts of that process.
For example, it does not consider how people from certain ethnic minority or religious backgrounds could be disproportionally impacted as a result of their over-representation in police databases, or the issue of unlawful custody image retention whereby the Home Office is continuing to hold millions of custody images illegally in the Police National Database (PND).
While the ethics panel meeting minutes offer greater insight into how Essex Police is approaching watchlist creation, the custody image retention issue was also not mentioned.
Responding to Computer Weekly’s questions about the meeting minutes and the lack of scrutiny of key issues related to UK police LFR deployments, an Essex Police spokesperson said: “Our polices and processes around the use of live facial recognition have been carefully scrutinised through a thorough ethics panel.”
Proportionality and necessity: the Southend ‘intelligence’ case
Instead, the officer present explained how watchlists and deployments are decided based on the “intelligence case”, which then has to be justified as both proportionate and necessary.
On the “Southend intelligence case”, the officer said deploying in the town centre would be permissible because “that’s where the most footfall is, the most opportunity to locate outstanding suspects”.
They added: “The watchlist [then] has to be justified by the key elements, the policing purpose. Everything has to be proportionate and strictly necessary to be able to deploy… If the commander in Southend said, ‘I want to put everyone that’s wanted for shoplifting across Essex on the watchlist for Southend’, the answer would be no, because is it necessary? Probably not. Is it proportionate? I don’t think it is. Would it be proportionate to have individuals who are outstanding for shoplifting from the Southend area? Yes, because it’s local.”
However, the officer also said that, on most occasions, the systems would be deployed to catch “our most serious offenders”, as this would be easier to justify from a public perception point of view. They added that, during the summer, it would be easier to justify deployments because of the seasonal population increase in Southend.
“We know that there is a general increase in violence during those months. So, we don’t need to go down to the weeds to specifically look at grievous bodily harm [GBH] or murder or rape, because they’re not necessarily fuelled by a spike in terms of seasonality, for example,” they said.
“However, we know that because the general population increases significantly, the level of violence increases significantly, which would justify that I could put those serious crimes on that watchlist.”
Commenting on the responses given to the ethics panel, Yeung said they “failed entirely to provide me with confidence that their proposed deployments will have the required legal safeguards in place”.
According to the Court of Appeal judgment against South Wales Police in the Bridges case, the force’s facial recognition policy contained “fundamental deficiencies” in relation to the “who” and “where” question of LFR.
“In relation to both of those questions, too much discretion is currently left to individual police officers,” it said. “It is not clear who can be placed on the watchlist, nor is it clear that there are any criteria for determining where AFR [automated facial recognition] can be deployed.”
Yeung added: “The same applies to these responses of Essex Police force, failing to adequately answer the ‘who’ and ‘where’ questions concerning their proposed facial recognition deployments.
“Worse still, the court stated that a police force’s local policies can only satisfy the requirements that the privacy interventions arising from use of LFR are ‘prescribed by law’ if they are published. The documents were obtained by Big Brother Watch through freedom of information requests, strongly suggesting that these even these basic legal safeguards are not being met.”
Yeung added that South Wales Police’s use of the technology was found to be unlawful in the Bridges case because there was excessive discretion left in the hands of individual police officers, allowing undue opportunities for arbitrary decision-making and abuses of power.
Every decision … must be specified in advance, documented and justified in accordance with the tests of proportionality and necessity. I don’t see any of that happening Karen Yeung, Birmingham Law School
“Every decision – where you will deploy, whose face is placed on the watchlist and why, and the duration of deployment – must be specified in advance, documented and justified in accordance with the tests of proportionality and necessity,” she said.
“I don’t see any of that happening. There are simply vague claims that ‘we’ll make sure we apply the legal test’, but how? They just offer unsubstantiated promises that ‘we will abide by the law’ without specifying how they will do so by meeting specific legal requirements.”
Yeung further added these documents indicate that the police force is not looking for specific people wanted for serious crimes, but setting up dragnets for a wide variety of ‘wanted’ individuals, including those wanted for non-serious crimes such as shoplifting.
“There are many platitudes about being ethical, but there’s nothing concrete indicating how they propose to meet the legal tests of necessity and proportionality,” she said.
“In liberal democratic societies, every single decision about an individual by the police made without their consent must be justified in accordance with law. That means that the police must be able to justify and defend the reasons why every single person whose face is uploaded to the facial recognition watchlist meets the legal test, based on their specific operational purpose.”
Yeung concluded that, assuming they can do this, police must also consider the equality impacts of their actions, and how different groups are likely to be affected by their practical deployments: “I don’t see any of that.”
In response to the concerns raised around watchlist creation, proportionality and necessity, an Essex Police spokesperson said: “The watchlists for each deployment are created to identify specific people wanted for specific crimes and to enforce orders. To date, we have focused on the types of offences which cause the most harm to our communities, including our hardworking businesses.
“This includes violent crime, drugs, sexual offences and thefts from shops. As a result of our deployments, we have arrested people wanted in connection with attempted murder investigations, high-risk domestic abuse cases, GBH, sexual assault, drug supply and aggravated burglary offences. We have also been able to progress investigations and move closer to securing justice for victims.”
Source
