Home Bots in Society Humans Impacted by ‘Killer Computers’

Humans Impacted by ‘Killer Computers’

by Pieter Werner

A new book ‘How to Protect Yourself from Killer Computers’ describes a number of catastrophic computer failures, including a new unreported scandal and new details on the British Horizon IT scandal. The book will also provide advice to the public on how they can protect themselves from ‘killer computers’ alongside explaining how such issues can be prevented.

As part of the investigations in the book, the author Dr Junade Ali CEng FIET, has identified a scandal similar to Horizon is highly likely to be ongoing. After a significant number of cases where TV Licensing direct debits failed, numerous people faced criminal prosecution including a woman with Down’s Syndrome and a woman who now suffers a “PTSD response” following a thwarted prosecution. Now, the author has obtained an admission from the BBC that there have been technical incidents within the TV Licensing direct debit system.

Despite failure states existing in the direct debit system and the Code of Practice for prosecutors in England and Wales requiring that “In conducting an investigation, the investigator should pursue all reasonable lines of inquiry, whether these point towards or away from the suspect”, it was also confirmed there is no “information held by the BBC related to the policy or practices related to the conduct of TV licensing of prosecution in the event a Direct Debit fails”. Statistical analysis in the book estimates that approximately 1,450 people a year could be at risk of criminal prosecution, following doorstep visits from TV Licensing enforcement officers after they were not aware they lost their TV licences due to IT failures.

When asked to disclose the underlying details of technology and software failures in the TV Licensing system under the Freedom of Information Act 2000, the BBC instead issued a statement published in the book acknowledging the existence of the bugs but not disclosing the relevant information, despite not claiming any legal exemption for failure to disclose such information. After an internal review was not completed within the deadline (despite being extended by the BBC), the BBC finally disclosed damning evidence to the author indicating the last time a substantial incident occurred (though the true scale was redacted to protect Capita’s commercial interests), it was only detected after a customer reported not having money debited from their bank account and took 17 days to resolve. In another case, the BBC admitted to breaching direct debit regulations – “Breach of BACS regs by 2 Working Days – Capita to manage the message should the issue arise with BACS”. Whilst public image was a risk considered in the incident report, the risk of miscarriages of justice was not. The BBC did not disclose information on the individual direct debit failures (on which the author based the estimate of 1,450 wrongful investigations), citing cost grounds.

Alongside new technical analysis of the Horizon IT scandal presented in plain English, the book exposes how the architect of the Horizon IT system and the Post Office’s professional witness in prosecution cases, Gareth Jenkins, was able to wrongly convince courts of the integrity of the Horizon IT system despite having no computer science education and there being bugs in the Horizon IT system. The book also reveals how the British Computer Society (BCS), who provided the regulated professional status Gareth Jenkins used to be accepted by the court as an expert, did not intervene even after over more than four years 10 articles had been written in Computer Weekly on the scandal, with details published in the book of how the BCS continue to attempt to conceal their role in the scandal to the current day.

Recently; McDonald’s, Sainsbury’s, Tesco, Greggs and Nationwide Building Society all saw outages affecting payment systems, with it remaining unclear who or what is responsible for these outages. The book sheds new light on how a global payment technology giant, Worldpay, used a gagging clause to workaround public interest disclosure laws concerning a whistleblowing claim, with the Financial Conduct Authority having issued an apology to the book’s author for not correctly responding when notified of these issues. Discussed in the book amongst these outages, on the 20th of March, previously unreported in the media, London police officers took to social media after the Metropolitan Police’s Connect IT system “imploded”. As of August 2023, the cost of the outsourced policing IT system is estimated to reach £214 million.

The book concludes with new analysis of what happened on Qantas flight 72, where an Airbus A330-303 entered a “death dive” as a result of a software bug. Timely, given the software issues that had affected the Boeing 737 Max aircraft and the recent death of a Boeing whistleblower. Providing insight into what happened, not only within the cockpit but also the computers, new advice is provided for how catastrophic computer problems can be averted.

The book is authored by Dr Junade Ali CEng FIET, a British computer scientist who has over a decade of experience in software engineering including engineering critical internet infrastructure in Silicon Valley and working on human transportation systems. Dr Ali is a Chartered Engineer, the terminal regulatory status for engineers in the UK, and was elected the youngest fellow of any professional engineering institution on record in 2023. Having studied for a Masters degree aged 17, Dr Ali holds both an MSc and a PhD in computer science.

Commenting on the book, Dr Junade Ali CEng FIET said: “As we integrate technology more deeply into the fabric of society, the consequences of failure become increasingly severe. This work not only sheds light on past software tragedies but also serves as a call for society to see humans as a fundamental part of ensuring that technical systems are resilient. Moreover, the statistics regarding retaliation against software engineers who report wrongdoing are alarming and represent a systemic issue that must be addressed to ensure technology serves the interests of society.”

Dr Ali went on to say: “In the Post Office Horizon IT Scandal we saw a clear example of what happens when an unquestioning belief in computers collides with serious issues being brushed under the carpet. This case demonstrates that killer computers emerge from toxic cultures. We see disturbing parallels between the TV Licensing direct debit system and the Horizon scandal. The investigation conducted in the book reveals shocking evidence that more people are being prosecuted based on a faulty IT system, including those most vulnerable in society. Not only have we now seen multiple examples which raise serious questions about the integrity of TV Licensing’s direct debit system, but we also now have an admission from the BBC that technical incidents have occurred.”

Finally, Dr Ali said: “The lessons in this book are clear – human beings form a fundamental part of the integrity of computer systems and when they seek to cover up or ignore problems, these scandals happen.”

Misschien vind je deze berichten ook interessant