NLP in Fintech
Introduction
Artificial Intelligence (AI) and Natural Language Processing (NLP) techniques are used in many areas, including medicine, finance, and recently in the Internet of things. NLP has made significant advances in recent years, and its usage has also increased considerably within the financial sector. Read on to learn about the latest advances in NLP and how they can be used in finance to automate text analysis and improve human responsiveness.
What is Artificial Intelligence?
Artificial intelligence, or AI, is a field of computer science that deals with building computers that operate like humans. That means they can do tasks traditionally associated with human intelligence, such as speech recognition and language translation. Most companies in their customer-service applications already use a simplified form of artificial intelligence (known as narrow AI). That type of AI is in production all over and around us these days. However, artificial general intelligence (AGI), an advanced form of AI which can carry out any intellectual task a human being can, is still years away from being developed. Some experts predict that it won’t be invented for another 100 years. Others share the opinion that we are on the right track to start achieving this with self-supervised learning, energy-based models, and manifold-based inference. This movement started and is currently low-hanging fruit with proven successful implementations, such as generative adversarial networks and Transformer based technologies. The speed of reaching upper and upper limits depends heavily on the processing power that we can spend with it, and quantum computing can bring quite an impressive improvement to the table. Unfortunately, this technology is still in the early stages of design and adoption.
Why does it Matter?
There is much promising research and recent breakthroughs into Natural Language Processing (NLP) and artificial intelligence (AI). And these trends are particularly relevant to finance. Over time, text analysis is becoming automated and with constantly increasing accuracy. You could also improve human responsiveness. These days, it can be almost taken for granted that NLP will affect your company, and it has probably already taken its place within, via some products you are already using. However, there’s a chance that increased exposure of the company’s customers to the recent advanced NLP techniques will positively impact how your customers interact with you. In other words, it’s worth keeping an eye on. This post will present insights into some recent advances in NLP that could be applied to financial services used within and served by financial companies and institutions.
How NLP Works, Just an Introductory Insight
In order to be able to extract meaning from language, NLP programs use machine learning methods to find statistical correlations between words and phrases. For example, suppose we have an array of words and associated positive or negative tags. In that case, there’s a high probability that liberty will be tagged as positive because it’s part of a list of things related to our concept of freedom. Likewise, a word like bankruptcy is likely going to be negatively correlated with the overall wellness, accounting liberty as well because bankruptcies inhibit and decrease your personal wellness. In addition to using word vectors (or other methods), NLP programs can understand syntax (how words relate) using more advanced models.
However, with the introduction of Google’s Transformers architecture, the meaning of the words started to be taken into account, having their surroundings in mind. In this way, we can talk about the context of the processed texts for the first time. This is achieved by training huge NLP language models with several billions of parameters over a vast amount of texts, taken from all over the Internet, and using digitalized libraries in a strictly self-supervised manner. In this way, we can produce models capable of capturing the semantics of the processed languages, by themselves, without a need for humanly labeled data — Welcome to the modern NLP :-).
NLP Applications in Finance: Let’s Start with Chatbots
One example of an area where NLP can be used to benefit finance is through customer service. Banks employ thousands of workers to field customer questions. However, it’s certainly not a practical or efficient use of their time. Banks can develop automated agents that instantly respond to customer queries, collect information, and answer questions quickly and accurately using Natural Language Processing. These kinds of applications are called chatbots, and these days, it is hard to find a site without some chatbot presented on the landing page. This technique gained wide acceptance during these Corona days when it is reasonably dangerous to enter rooms with many people inside. Furthermore, it is much more acceptable to contact the appropriate person from home, using the personal computer in any form: desktop, laptop, tablet, or smartphone.
Phone conversation with an actual human person is doable and acceptable. However, chatbots offer 24x7 availability and routing a vast part of the frequent client request with well-defined workflow to the machines.
In short: using artificial intelligence allows organizations like banks to focus on high-level concerns while leaving tasks better suited to machines (like receiving phone calls) free for human workers. Savings and other financial gains are achieved by offering automated services with 24x7 availability, in this way, can be immersive.
Cloud-based Chatbots
Having recognized the need for chatbots, as well as the possibility to implement them reliably with the latest NLP advances, many companies jumped on that bandwagon, initially led by traditionally big cloud players: Google and Microsoft. Google is offering DiagFlow, and Microsoft is providing Power Virtual Agents.
The main advantage of using such cloud-based services is that you don’t need technical knowledge for building them. All the hard work around the intent detection and item extractions is provided automatically; all you need is to define the planned scenario in which the visitor interacts with the virtual agent and provide sources for extracting needed information when the responses are given back.
However, this approach is still not particularly well accepted by the traditionally conservative institutions, like the banks. That is particularly backed by the security concerns, particularly by the law enforcement that regulates the amount and types of information that can leave their premises and be exposed to external parties in a very rigorous manner.
Locally Hosted Chatbots
Having security concerns and law enforcement in place, the banks are left to their own devices: developing and hosting chatbots in their very own premises, under their strictly controlled and secured environment. One of the major and the most mature players in this field is the Rasa chatbot framework.
This framework is relatively advanced, with a greatly established balance between easy-to-use and extensibility: configurable by YAML configuration files and extensible with Python scripts that allow injection of custom business actions where needed. In addition: it is advancing quickly enough, keeping in line with the latest achievements in the NLP area, with the latest versions. And yes: it is hosted completely in the company’s premises, without passing any information collected from the interaction with the customers to external parties. In addition to this: hosting it in-house enables integration with the surrounding software modules without having additional worries related to the possible security issues. This can be covered entirely according to already established security architecture, aligned to the specific company’s needs.
Text Classification and Sentiment Analysis
AI/NLP text classification is used for the automatic classification of text. In financial analysis, text classification methods are mainly used to recognize business-related events (releases) from news articles. That can be done by assigning labels to an entity or object that appears in a text and then use those labels to analyze data.
One situation: you want to determine all the news where the name of your company appears, then classify them to politics, sport, business, celebrities, entertainment, and others of interest.
Another example, suppose you want to determine whether a company is performing well. In that case, you can look at how often it appears on positive or negative news stories and decide whether it’s good to invest money there.
Here, we quickly introduced sentiment analysis as a sub-task of the text classification task. Here we have only two types/classes of texts that we should recognize automatically: positive, negative, and sometimes neutral if found to be influential.
Sentiment Analysis for Fast Trading
General Sentiment Analysis
General sentiment analysis provides mechanisms for text classification to positive and negative (and neutral, when needed) sentiments. Tools for doing this exist a long time ago (ok, several years, but long in AI terms), initially backed by lexicons of positive and negative words with several thousands of words inside, then determining sentiment by simply counting positive and negative sentiment related words. More sophisticated approach: averaging provided weights of the words. This approach is called BOW: Bag of Words. All the words are put in a basket, then processed separately and without taking the ordering into account. Initially adopted by the most popular NLP Python libraries: NLTK and Spacy, many versions ago. This is a very fast, but unfortunately, unreliable technique since the sentiment heavily depends on the meaning of the complete sentence and is dependent on the order in which words appear.
Recent techniques implemented with BERT language models, based on Transformers, and fine-tuned to sentiment analysis, provide much more context-aware analysis by taking into account the meaning of the complete sentence and the ordering of the words inside. This approach covers the main disadvantage of the BOW approaches: not considering ordering the words, but is slow since it is very computationally heavy, as implemented in AI models with hundreds of millions of parameters inside.
Few examples:
- Input: I like you. I love you
label: NEGATIVE, score: 0.0012289267033338547
label: POSITIVE, score: 0.998771071434021
- Input: I don’t know do I like you
label: NEGATIVE, score: 0.999465823173523
label: POSITIVE, score: 0.0005341752548702061
- I don’t love you
label: NEGATIVE, score: 0.0012289267033338547
label: POSITIVE, score: 0.998771071434021
Financial Sentiment Analysis
Semantic and context-aware approaches to building NLP language models for the first time provided mechanisms able to perform NLP tasks that are narrow-downed to some specific business case. That is allowed by using transfer learning technology: first, you train a massive model with several thousand million parameters in a self-supervised way, without a need for slow, unreliable, and expensive human labeling. Then you fine-tune produced modes to particular domain-specific NLP tasks using ten times smaller datasets labeled by the humans. When applying this approach, what you get is a more accurate and ten times cheaper model than the ones provided by the traditional methods.
Applying this technique resulted in a sentiment analyzer distinguishing positive from negative texts from the financial point of view.
Few examples:
Input: “Stocks rallied and the British pound gained.”
Output:
positive: 0.8983616232872009
negative: 0.034473467618227005
neutral: 0.06716492027044296
Fast Trading with Sentiment Analysis
Traders have already worked out what they will do in response to any piece of news before it is released. However, anticipating how that news will affect asset prices still remains a significant challenge these days. Sentiment analysis can improve these predictive skills by providing a quantitative estimate for how people will respond and, therefore, making trades on the crowd rather than simply using historical data and technical analysis. A model released several months ago is capable of doing precisely this: we put in it a piece of news, and what we get out is its analysis whether, and quantitatively how much (in number, 0–1), the provided text is in a positive connotation or a negative one. In this way, the machines could digest a massive amount of incoming news related to some specific financial instrument and calculate the data’s overall sentiment. Valuable insight from this is how the majority of the other traders will react to this, thus having the advantage of acting, i.e., buying/selling upfront. That, in addition, provides valuable and insightful support for advanced trading strategies that the trader can apply by their own choice: whether he will go with the stream, but upfront, or he takes another trading strategy approach: going in the opposite direction, with some lagging and hoping for quick pivoting (which actually happens very frequently, learned in on the hard way, by my own experience). Probably it is worth mentioning that having in mind that all the sentiment analysis results are quantified, they can be additional variables for building more powerful trading bots.
Text Summarization
General Text Summarization
Tackling a long text is a daunting task, particularly if you have to deal with documents that span dozens of pages. The longer a text is, the harder it becomes to read and understand its most essential points; eventually, reading it becomes exhausting and unnecessary — unless your job depends on it. One of NLP’s core functions is summarizing texts — finding and extracting its main ideas in no more than 100–200 words or less. NLP can support you with some proper tools if you’re looking for insight from thousands of pages without feeling like you want to give up before even starting. Text summarization has practical applications across multiple areas: politics, social sciences, tech-journalism, and beyond.
Example:
Input: “The tower is 324 metres (1,063 ft) tall, about the same height as an 81-storey building, and the tallest structure in Paris. Its base is square, measuring 125 metres (410 ft) on each side. During its construction, the Eiffel Tower surpassed the Washington Monument to become the tallest man-made structure in the world, a title it held for 41 years until the Chrysler Building in New York City was finished in 1930. It was the first structure to reach a height of 300 metres. Due to the addition of a broadcasting aerial at the top of the tower in 1957, it is now taller than the Chrysler Building by 5.2 metres (17 ft). Excluding transmitters, the Eiffel Tower is the second tallest free-standing structure in France after the Millau Viaduct.”
Output: “The Eiffel Tower is a landmark in Paris, France.”.
Text Summarization for Fast Trading
Let’s look at a widespread situation — you are a trader, and, of course, you have large volumes of text to summarize. Speed is of the essence, and time is money. You will need to quickly understand what is going on in your market/industry/instrument. That might sound like a job for a human being, simply because it was done till these days by humans only. And yes, they are expensive, complicated, and costly to maintain :-) These days, this task is not performed only by humans. In fact, these days == since February 2021. One of the most exciting news in the AI world at the beginning of this year was the appearance of the Pegasus NLP summarization model. This model was fine-tuned on a novel financial news dataset, consisting of 2K articles from Bloomberg on stock, markets, currencies, rate, and cryptocurrencies, with extreme summarizations done by the humans. Extreme, because the summarizations that it produces are maximum of two-sentence long. Kind of a readable text compression that focuses on the relevant information important from the financial point of view. Of course, this compression is not lossless, and some information is “lost in summarization”. Still, what this model makes extraordinary is the fact that it can recognize the facts with a value from the financial point of view and put them in front of our eyes within 1.5 secs.
Example:
Input: “National Commercial Bank (NCB), Saudi Arabia’s largest lender by assets, agreed to buy rival Samba Financial Group for $15 billion in the biggest banking takeover this year. NCB will pay 28.45 riyals ($7.58) for each Samba share, according to a statement on Sunday, valuing it at about 55.7 billion riyals. NCB will offer 0.739 new shares for each Samba share, at the lower end of the 0.736–0.787 ratio the banks set when they signed an initial framework agreement in June. The offer is a 3.5% premium to Samba’s Oct. 8 closing price of 27.50 riyals and about 24% higher than the level the shares traded at before the talks were made public. Bloomberg News first reported the merger discussions. The new bank will have total assets of more than $220 billion, creating the Gulf region’s third-largest lender. The entity’s $46 billion market capitalization nearly matches that of Qatar National Bank QPSC, which is still the Middle East’s biggest lender with about $268 billion of assets.”
Output: “NCB to pay 28.45 riyals for each Samba share. Deal will create Gulf region’s third-largest lender”.
Automatically Determine the Financial Sentiment of a Long Text
Things are becoming more complicated with more extensive texts that span over 1000 words and several pages of text. We can approach such texts with a hierarchical summarization and sentiment analysis in a recursive manner. So: we have one huge text. We divide it into pages, and then we split pages into paragraphs. Next is automatically mining the sentiments of the paragraphs, then calculating the average sentiment of the pages, in parallel with summarizing the paragraphs and summarizing the summarizations of the paragraphs. Then we reach the final summarization of the complete text as summarization of the summarizations of the pages, and final sentiment as averaged sentiment of the pages. I know, too many hard-to-follow words, but one cool graphical presentation of all of this can do the magic. It’s just up to the innovative and inventive GUI designers to propose the look and feel that would be eye-catching and self-explanatory for the customer at first sight.
The proof-of-concept for the rest is done. Welcome to the magic of the foundation models: the latest AI term just started to float in the AI space on the Internet, just several days ago.
Compliance Analysis of Standard Financial Reports
When you’re trying to speed up your processes, it’s easy to overlook administrative tasks like Am I complying with all of my regulations? Automating a compliance analysis takes much manual work off your plate, leaving you free to focus on more complicated or pressing issues. When you’re ready to roll out such a system, there are several ways to go about it; depending on your regulatory obligations and business model, one will be better than another. Compliance automation is usually not something that comes cheap (or quickly), but once implemented, can save vast amounts of time — and probably put regulators at ease. Get some quotes for systems from big banks and consult with regulators before moving forward on any project; if they don’t say anything within 5 minutes, then chances are they won’t care.
NLP can be used with a processing pipeline with a set of referent documents representing solid and compliant examples. We implement dividing the documents into parts according to regulatory requirements or according to the enforced business rules. Then, we measure the semantic distance between each of the elements of the processing document with the averaged vector over the appropriate document vectors from the referent documents set. We set the threshold for minimal Euclidean distance, and magically, we have a compliance classifier per required part. Then we know if we have all necessary parts adequately filled, with some probability, or not.
We silently introduced one new recently (talking about several months) available feature here: semantic similarity. We measure how close two documents are by the human meaning, not how many words both of the documents share. Another cool thing provided by the so-called foundation models, a term that appeared in the AI circles three weeks ago, gaining in popularity.
Looks like rocket science? Well, it’s not. Want to have it implemented, for some particular use-case, tailored for your own specific needs? Contact me at striki@striki.ai .
Final Words
The techniques of Natural Language Processing are a valuable tool for finding solutions to many complex problems, such as fraud detection, customer service response times, and information retrieval. They can also be used to automate and enhance a company’s digital processes. By understanding these NLP advances, you can decide which will benefit your business most, but most importantly, you will start thinking about how they might be used in your own work or research project.
This text presented several use-cases for NLP, in finance for the particular analysis. But, please bear in mind that, literally speaking, everything that was said above is language and business agnostic. Computers are pretty much dumb creatures, you know. All they understand are numbers. Meaning: absolutely everything can be applied to any other industry in any language other than English, in exactly the same way.
Good luck!. & stay virus-less.
References
https://en.wikipedia.org/wiki/Transformer_(machine_learning_model)
https://towardsdatascience.com/transformers-141e32e69591
https://arxiv.org/abs/1706.03762
https://towardsdatascience.com/bert-explained-state-of-the-art-language-model-for-nlp-f8b21a9b6270
https://www.tensorflow.org/tutorials/keras/text_classification_with_hub
https://towardsdatascience.com/text-summarization-using-deep-neural-networks-e7ee7521d804
#thisisalmostautogeneratedtext
NOTE: all the images are generated by the author, using VQGAN + CLIP AI model.
Author:
Stojancho TudjarskiGraduated software engineer with 29 years of experience in the IT industry.
The last 11 years he spent as an AI practitioner, learning, doing, and consulting. The last 18 months, intensively focused on the latest breakthroughs in the NLP area of AI, which included BERT and GPT-2 language models, and experimenting with them a lot. Trained his own BERT and GPT-2 models for all of the Balkan languages, making them appearing for the first time, publicly available for the first time, at https://huggingface.co/macedonizer, with the support from the Maceonizers team from FINKI.
Experimented extensively with fine-tuning GPT-2 with Shakespeare’s writings, among other things. Result: a neural network capable of writing poetry in Shakespeare’s style, at a scale. In addition to this: passed through the process of mixing styles, which resulted in an automatic text generator about Covid-19, in Shakespeare’s style. All the experiments are documented at https://dmind.ai. The same is done with Blaze Koneski’s writing style, autogenerated texts available on the same site.
Continually monitoring the latest happenings in the AI area on a daily basis and evaluating ideas where the new academic advances can be monetized in the business.