Imagine you are at the helm of a small debt collection company, Financial Corporation (FC), with a long and respectable history since 1982. Despite the solid foundation, between 2016 and 2018, the company experienced a worrying decline in financial performance. Revenues generated from fees on recovered receivables fell from 16.2 million to 14.6 million, and the recovery rate (RR), the company’s main performance indicator, plummeted from 10 percent to 7 percent.
How would you intervene? The answer would seem obvious: You would have no choice but to rely on the established tools that, combined with good managerial insight, allow you to assess performance, improving efficiency. But what would you do if you were told that there could be more secure and reliable systems, based on big data, to turn your company’s fortunes around? Perhaps you would do just as FC’s management decided, that is to collaborate with a team of researchers from the University of Bologna to try to improve one of the weaknesses of its Performance Management System: the identification of value drivers.
Performance Management Systems (PMS) are operational mechanisms designed to drive organizational performance toward expected goals. In detail, they deal with the formal and informal mechanisms, processes, systems, and networks used by organizations to convey the key objectives defined by management, support the strategic process and ongoing management through analysis, planning, measurement, control, reward, and, more generally, manage performance by facilitating organizational learning and change.
Yet a growing number of studies have questioned the role of PMSs in today’s business environment of increased organizational dynamism and complexity and the effectiveness of established approaches. This is because they do not always prove capable of handling the complexity in defining and measuring value drivers. Indeed, managing them has typically been left to managerial wisdom and the use of subjective top-down processes, with limited use of quantitative and data-driven approaches. An approach that has been associated with a lack of strategic focus, questioning of selected drivers, misallocation of individual efforts, and uncertainty about the evaluation process. At the same time, recent studies show that interdependencies between drivers and measures are generally neglected in the implementation of PMSs.
From these considerations and following recent requests for the use of business analytics (BA) to support PMS implementation, comes the study “The Quest for Business Value Drivers: applying Machine Learning to Performance Management” conducted by F. Visani, A. Raffoni and E. Costa of the University of Bologna. The study investigates whether methodological and analytical complexities in identifying value drivers can be mitigated through the adoption of Machine Learning (ML) techniques. ML encompasses several algorithms and approaches that can learn from past data without being specifically programmed to do so and has technical features that, from a theoretical point of view, can address the aforementioned sources of complexity. However, the most effective algorithms in providing predictions are often kinds of “black boxes,” which do not offer a representation of the phenomenon being analyzed. This trade-off between predictive ability and interpretability is widely discussed in the literature on ML. Despite this, however, the application of ML to the business context has grown rapidly in recent years, mainly due to the increase in available data and computing power. Several areas of business management, such as marketing and operations, have seen an increase in ML adoption, but the field of performance management and accounting control remains relatively unexplored in this respect.
The research was developed through an “action-research” approach, which allowed researchers to work closely with FC management. This approach made it possible to gain a detailed understanding of organizational practices and emerging challenges, enabling practical and theoretically sound solutions to be developed. The research process began with data collection and cleaning, which were essential to ensure the reliability of subsequent analyses. The original dataset contained 20,218 records regarding both debtors and recovery processes, distributed across 17 variables, including gender, age, geographic region, and debtor creditworthiness, as well as details about the debts themselves such as the amount and duration of recovery. After an initial exploratory analysis of the data, the team proceeded with the selection of the most significant variables using ML algorithms such as logistic regression and decision trees.
The ML models revealed several interesting insights. For example, prior emailing of debtors and quality of operators emerged as key factors influencing the success of debt collection. Logistic regression showed that variables such as debtor’s creditworthiness and geographic region significantly influenced the outcome of recovery procedures. On the other hand, decision trees provided a clear and intuitive representation of the relationships between variables, making it easier for managers to understand and interpret the results. The adoption of ML led to an overhaul of FC’s PMS. The company began using the results of ML models to guide the selection and assignment of cases to caseworkers, improving the overall effectiveness of the recovery process. In addition, a new performance measure, “Return on Effort” (ROE), was introduced, which compares the amount recovered with the total duration of the procedure, providing a more accurate assessment of the effectiveness of the recovery process.
The case study of FC shows that machine learning can reduce methodological and analytical complexity in business performance management by improving the ability to identify and measure value drivers. However, to fully exploit the benefits of ML, a collaborative approach combining technical expertise and managerial knowledge is essential. This research provides new insights into the benefits and challenges of introducing an ML-based PMS, contributing to the literature on business analytics applied to performance management and offering practical insights for other organizations wishing to adopt similar approaches.
