The rise of fintech services and cryptocurrencies have changed modern banking in a number of ways, and banks face an increasing number of challenges as various third-party payment processors interpose themselves between financial institutions and their traditional customers. The credit scoring systems used broadly in the US and Europe are based on so-called “hard” information — bill payments, pay stubs, and how much of your current credit limit you are tapping.
The researchers point out that so-called “hard” credit scores have two significant problems. First, banks tend to reduce credit availability during a downturn, which is when people most need help. Second, it can be difficult for companies and individuals without credit histories to begin creating one. There’s a bit of a catch-22 in the system, in that what you need to persuade an institution to loan you money is a credit history you don’t have because no one will loan you money.
Having identified two flaws in the existing system, the authors write:
The rise of the internet permits the use of new types of nonfinancial customer data, such as browsing histories and online shopping behavior of individuals, or customer ratings for online vendors.
The literature suggests that such non-financial data are valuable for financial decision making. Berg et al. (2019) show that easy-to-collect information such as the so-called “digital footprint” (email provider, mobile carrier, operating system, etc.) performs as well as traditional credit scores in assessing borrower risk. Moreover, there are complementarities between financial and non-financial data: combining credit scores and digital footprint further improves loan default predictions. Accordingly, the incorporation of non-financial data can lead to significant efficiency gains in financial intermediation.
In a blog post published on the IMF’s website, the authors also write: “Recent research documents that, once powered by artificial intelligence and machine learning, these alternative data sources are often superior than traditional credit assessment methods.”
However much the authors of this paper know about banking systems and finance, they’re clearly not up to date on the latest in AI research. This is a bad idea in general, but it’s a really terrible idea right now.
The first major problem with this proposal is there’s no evidence AI is capable of this task or that it will be any time soon. In an interview with The Guardian earlier this summer, Microsoft AI researcher Kate Crawford had some harsh remarks for the current reality of artificial intelligence, despite working for one of the leaders in the field: “AI is neither artificial nor intelligent. It is made from natural resources and it is people who are performing the tasks to make the systems appear autonomous.”
When asked about the specific problem of bias in AI, Crawford said:
Time and again, we see these systems producing errors – women offered less credit by credit-worthiness algorithms, black faces mislabelled – and the response has been: “We just need more data.” But I’ve tried to look at these deeper logics of classification and you start to see forms of discrimination, not just when systems are applied, but in how they are built and trained to see the world. Training datasets used for machine learning software that casually categorise people into just one of two genders; that label people according to their skin colour into one of five racial categories, and which attempt, based on how people look, to assign moral or ethical character. The idea that you can make these determinations based on appearance has a dark past and unfortunately the politics of classification has become baked into the substrates of AI.
This isn’t just the opinion of a single person. Gartner has previously projected that 85 percent of AI projects through 2022 “will deliver erroneous outcomes due to bias in data, algorithms or the teams responsible for managing them.” A recent Twitter Hackathon found proof that the website’s photo-cropping algorithm was implicitly biased against older people, disabled people, Black people, and Muslims, and it frequently cropped them out of photographs. Twitter has since discontinued using the algorithm because these kinds of bias problems are in no one’s best interest.
While my own research is far removed from fintech, I’ve spent the last 18 months experimenting with AI-powered upscaling tools, as regular ExtremeTech readers know. I’ve used Topaz Video Enhance AI a great deal and I’ve experimented with some other neural nets as well. While these tools are capable of delivering remarkable improvements, it’s a rare video that can simply be chucked into TVEAI with the expectation that gold will arrive out the other side.
This frame isn’t fabulous, but it’s not too bad relative to the original source material, either. If you weren’t paying attention, you might not notice how poorly Dax is rendered in the background (she’s the seated woman at the console in the back).
Here’s frame 8829 from the Star Trek: Deep Space Nine Episode “Defiant.” The quality of the frame is reasonable given the starting point of the source, but we’ve got a glaring error in the face of Jadzia Dax. This is output from a single model and I blend the output of multiple models to improve DS9’s early seasons. In this case, every model I had tried was breaking in this scene in one way or another. I’m showing output from Artemis Medium Quality in this instance.
This is what happens when we zoom. Dax is neither a Navi nor traditionally rendered in the art style of Ancient Egypt.
This specific distortion happens once in the entire episode. Most Topaz models (and every non-Topaz model I tested) had this problem and it proved resistant to repair. There aren’t very many pixels representing her face and the original MPEG-2 quality is low. There is no single AI model that treats an entire episode of S1 – S3 correctly that I’ve found yet, but this is by far the worst distortion in the entire episode. It’s also only on screen for a few seconds before she moves and the situation improves.
The best repair output I’ve managed looks like this, using TVEAI’s Proteus model:
By using a different model, we can semi-repair the damage — but not completely. Too much of AI is currently like this. Capable, but limited, and reliant on human oversight.
There’s a reason why I’m using video editing to talk about problems in fintech: AI is nowhere near perfect yet, in any field of study. The “fix” above is imperfect, yet required hours of careful testing to achieve. Behind the scenes of what various companies smugly call “AI” are a lot of humans performing an awful lot of work. This doesn’t mean there isn’t real progress being made, but these systems are nowhere near as infallible as the hype cycle has made them out to be.
Right now, we’re at a point where applications can produce some amazing results, even to the point of making genuine scientific discoveries. Humans, however, are still deeply involved in every step of the process. Even then, there are mistakes. Fixing this particular mistake requires substituting output from an entirely different model for the duration of this scene. If I hadn’t been watching the episode carefully, I might have missed the problem altogether. AI has a similar problem in general. The companies that have struggled with bias in their AI networks had no intention of putting it there. It was created due to biases in the underlying data sets themselves. And the problem with those data sets is that if you don’t examine them with care, you might wind up thinking your output is composed entirely of frames like the below, as opposed to the damage scene above:
This is more typical of final output; absolute quality is limited by the original source. There are no egregious distortions or other problems. Human oversight of these processes is required because AI tools aren’t good enough yet to always get it right 100 percent of the time. Fintech tools aren’t either.
Even if the AI component of this equation was ready to rely on, privacy issues are another major concern. Companies may be experimenting with tracking various aspects of “soft” consumer behavior, but the idea of tying your credit score to your web history is very similar to the social credit score now assigned to every citizen by China. In that country, saying the wrong things or visiting the wrong websites can result in one’s family members being denied loans or access to certain social events. While the system contemplated is not that draconian, it’s still a step in the wrong direction.
The United States has none of the legal framework that would be required to deploy a credit monitoring system like this. Any bank or financial institution that wishes to use AI to make decisions regarding the creditworthiness of applicants based on their browser and shopping history needs to be regularly audited for bias against any group. The researchers that wrote this document for the IMF talk about hoovering up people’s shopping histories without considering that many people use the internet to shop for things they’re too embarrassed to walk into a store and buy. Who decides which stores and vendors count and which do not? Who watches over the data to make sure intensely embarrassing information is not leaked, either on purpose or by hackers more generally?
The fact that non-bank financial institutions may be jonesing to use some of this data (or already using it) is not a reason to allow it. It’s a reason to stay as far away from said organizations as possible. AI is not ready for this. Our privacy laws are not ready for this. The consistent messaging from reputable, sober researchers working in the field is that we’re nowhere near ready to turn such vital considerations over to a black box. The authors who wrote this paper may be absolute wizards of banking, but their optimism about the near-term state of AI networks is misplaced.
Few things are more important in modern life than one’s credit and financial history, and that’s reason enough to move exceptionally slowly where AI is concerned. Give it a decade or two and check back then, or we’ll spend the next few decades cleaning up injustices inflicted against various individuals literally through no fault of their own.
Now Read:
- AI-Powered Electronic Design Automation Tools Could Redefine Chipmaking
- US Launches Investigation of Tesla Autopilot Following Emergency Vehicle Crashes
- Google Unveils Pixel 6 With Its First Custom Mobile Processor
No comments:
Post a Comment