Accuracy of the scoring model based on financial statements
The credit risk database (CRD) was established in March 2001 as a membership organization to collect financial statements, some nonfinancial information, and default information of small and medium-sized enterprises (SMEs) in Japan. CRD members are composed of all credit guarantee corporations (CGCs) in Japan, government-affiliated or private financial institutions, and so on.
Are financial statements such as those collected by the CRD effective for evaluating the creditworthiness of SMEs? The CRD provides members with a stable and accurate scoring model made from a large number of financial statements. Figure 1 compares the scoring model’s estimations with actual results. As you see, we divided the SMEs into 10 groups based on the outcomes of the scoring model, by probability of default (PD), with group1 being the most creditworthy and group10 the least creditworthy. The average PD of each group almost corresponds to the actual default rate in each group. We can confirm that the scoring model’s estimation is accurate and financial statements are effective for evaluating the creditworthiness of SMEs.
Figure 1: Comparison of the Probability of Default with the Actual Default Rate
Note: Actual default rate means the ratio of default small and medium-sized enterprises (SMEs) to the total SMEs (in each group).
Source: Credit Risk Database Association.
The CRD financial statements are effective because
- they are based on a large database,
- they contain rich items from balance sheets and profit and loss statements, and
- data are collected through the scheme for SME finance
Large database
Currently, the database contains data from more than 3 million SMEs (incorporated and sole-proprietor SMEs). Since each SME has consecutive financial statements, the CRD has more than 20 million financial statements (Figure 2). The SMEs’ financial statements might not always be correct and a small part of the SMEs’ data might be sample biased, but the CRD can provide an accurate and stable scoring model based on a large database according to, as it were, the law of large numbers. If we could collect and maintain abundant data, we can extract the common features of SMEs in our analysis without confusing them with incorrect or sample-biased data, and we can make use of such a feature in the scoring model.
Figure 2: Accumulated Data
Source: Credit Risk Database Association.
Maintaining a large number of members and their abundant data will keep the CRD active. Thus, the CRD has been trying to maintain the quality of the scoring models and provide members with other services that conform to their needs.
Since the accuracy of the scoring models generally tends to deteriorate with time, they must be validated to maintain their quality. After creating scoring models, the CRD association has been validating them to ensure they fit the updated database, improving them as necessary and giving helpful information to members.
Although the basic service of the CRD is to provide scoring models to members, it also offers members various services such as statistical information services, sample data service, management consulting support system service, and other consulting services. Providing services that are attractive to members helps to maintain the present membership or gain new members.
Rich items from balance sheets and profit and loss statements
Generally, the accuracy of scoring models would be too high when the size of the database and its contents is large. CRD members submit up to 59 items from the balance sheet (minimum 26) and up to 26 items from the profit and loss statement (minimum 9). We can create a large number of financial indexes from financial statements for purposes of evaluating the ongoing business situation of an SME.
By using the statistical method, we usually grasp what financial indexes are effective and how they are able to detect default. However, there are some financial indexes we can easily identify and can detect possibility of default. For example, “capital-adequacy ratio” shows the stability of capital; and “interest rate for interest-bearing debt” shows the borrowing situation.
We divided the dataset into 10 groups according to the value of those financial indexes. Then we checked the relationship between the value (good and bad) of financial indexes and default rate of each group. The worse the financial indexes, the higher the default rate. If “capital-adequacy ratio” goes down or “interest rate for interest-bearing debt” goes up, the default rate will go up, and vice versa. In general, scoring models for incorporated SMEs use 20~30 financial indexes as explanatory variables. A lot of explanatory variables make scoring models accurate and stable. If we have rich items of financial statements, we have the possibility to create a more superior scoring model. When we build the scoring model, we try to confirm the relationship between various financial indexes and the default, and many combinations of financial indexes (Figure 3).
Figure 3: Relationship between Financial indexes and Defaults
Note: Capital-adequacy ratio = Net Asset/(Liabilities and Net Assets); Interest rate for interest-bearing debt = (Interest Expenses and Discount Interest)/(Liabilities and Discounts on Notes Receivable)
Source: Authors.
Data through the scheme for SME finance
The CRD does not collect financial statements from SMEs directly but collects them through the mechanism of SME finances. When SMEs borrow money from financial institutions, they submit their financial statements to financial institutions. If SMEs are guaranteed by the CGCs, the financial institutions pass the financial statements over to the CGCs. Throughout the loan and guarantee period, financial institutions and CGCs ask SMEs to submit their financial statements so that they can monitor the SMEs’ business situation and confirm the certainty of repayments.
Collecting data through this scheme helps maintain the high quality of the database besides making it easy to collect abundant financial statements. Before putting the SMEs’ financial statements into the CRD, these are screened and improved by financial institutions or CGCs as part of their examining process, enhancing the accuracy of the database and the effectiveness to better evaluate the SMEs’ creditworthiness (Figure 4).
Figure 4: Circulation of Data and Services
B/S = balance sheet, P/L = profit and loss statement, SME = small and medium-sized enterprise.
Source: Authors.
Japan’s experience regarding CRD can help other countries establish the same database of financial statements. If financial institutions and so on can obtain better information on SMEs by using a database like the CRD, finance for SMEs can be provided smoothly. Improving the efficiency of providing financing to SMEs will lead to economic growth in other countries.
_____
References:
Kuwahara, S., N. Yoshino, M. Sagara, and F. Taghizadeh-Hesary. 2015. Role of the Credit Risk Database in Developing SMEs in Japan: Lessons for the Rest of Asia. ADBI Working Paper 547. Tokyo: Asian Development Bank Institute.
Photo: By Danny Choo (Flickr: Good Smile Company Offices) [CC BY-SA 2.0 [1]], via Wikimedia Commons [2]
