You may be old enough to remember the Talking Heads 1984 concert film, “Stop Making Sense.” I think that phrase is a fitting warning to anyone designing agents for human-centric business processes who assumes that their behavior must always be rational. Despite significant advances in applied analytics, behavior modeling and process intelligence, achieving the right balance between achievable objectivity and perceived subjectivity differs significantly between processes that focus on people versus those that manage “things.” This is especially true for artificial intelligence (AI), agents and agentic systems working within customer relationship management (CRM), customer experience (CX) and human capital management (HCM) software.
ISG’s 2026 AI Value Study finds that a majority of enterprises rely on humans-in-the-loop (HITL) rather than autonomous agents for AI-enabled processes. In particular, for employee
training, coaching and enablement, 44% of participants described their process as humans performing with AI assistance and 29% said that AI performs the process but humans review. For customer engagement and service, 39% have humans performing the process assisted by AI and 30% have humans reviewing what AI performs. Inertia combined with caution is likely to keep HITL as the choice for a majority of processes. As a result, ISG Research asserts that through 2029, enterprises will struggle with aligning agent design and controls around business-focused strategy that defines governance, risk management, process redesign and change management policies. People-centric processes are more intimately entwined with matters that reflect the perceptions and behaviors of individuals or groups and are therefore inherently trickier to design than those that manage accounting, inventory and supply chains. So, “think different” is the order of the day.
In practice this means:
-
- Approach human-centric agent and agentic system design with the understanding that its constraints and consequences are different from those that manage things.
- Rather than using technology solely to perform an analog-to-digital conversion of existing processes to gain efficiency, agentic system design should explore finding ways to transform or even create new processes that are more productive or effective.
- Recognize that some human-centric processes are not usefully codifiable because they are effectively kabuki theater. Sales forecasting is one example, where the process itself is of secondary importance to the posturing, sandbagging and negotiating that takes place around a process that exists in name only.
Understand Constraints and Consequences: Human behavior at the individual level is inherently less deterministic than capturing even the nuances of easier-to-define systems of accounting and supply chain management. Moreover, people are highly sensitive to errors and disparities in outcomes at the individual level. The consequences of unintended results have a greater impact on perceived bias and trust in people-centric systems and may inadvertently run afoul to existing laws and regulations that can result in reputational damage or financial loss. For this reason, although agents are more flexible and can learn and adapt over time, their autonomy in human-centric processes must be strictly governed with approvals, controls and oversight.
Prediction accuracy alone is not enough to judge the usefulness of model driven agent behavior in human-centric processes. The consequences of being wrong matter just as much, and in certain sets of use cases often more. A model can be highly accurate overall and exceed some general expected threshold yet still be defective if its few errors occur in high-impact cases. Conversely, a less accurate model may be preferable if its mistakes are inexpensive or easily corrected. The key distinction is between the probability of error (how often the prediction is wrong) and the cost of the error: the financial, operational, regulatory, safety or reputational damage caused by each type of mistake. Agent and model reviews must routinely assess these parameters.
Attempts at rational modelling are further complicated because false positives and false negatives can carry different consequences of different magnitudes. For example, unnecessarily reviewing a legitimate transaction creates only modest friction, while failing to detect major fraud can cause a substantial loss. In human terms, the asymmetry in compensation errors is stark: underpay an individual just a little and expect outcries and diminished trust. Overpay and…crickets. In principle, decision thresholds must reflect the relative costs of those errors and not simply maximize aggregate accuracy. In practice, the challenge is quantifying situational outcomes and setting confidence thresholds. Past experience suggests that some processes or conditions will defy modeling because the volume of false positives or false negatives is unacceptably high.
Find a Different Approach: Rather than using technology solely to mimic existing processes to gain efficiency, AI and agentic system design should explore finding ways to use technology to enhance, transform or even create new processes that are more productive or effective. Human-centric processes are fertile ground for this approach because the stated process may be an agreed-upon fiction (see below) or because the addition of AI and agents makes it possible to substantially improve the outcomes of existing processes. For example, as my colleague recently pointed out, a more useful role for AI and agents in performance management is providing support that helps managers notice what they are likely to miss, gets them to act in a timelier fashion to address issues and gives them the ability to coach with context to achieve better outcomes. This is a work in progress. ISG Research asserts that through 2029, persistent gaps in performance management innovation will fuel a new generation of AI-enabled performance solutions emphasizing goal-to-work linkage, coaching and decision documentation.
The same applies to sales, where my colleague Barika Pace pointed to the need to use AI and technology to address today’s increasingly more fragmented B2B selling model by creating a digital sales room. Or an example that my colleague Keith Dawson wrote about, where technology enables the fusion of inside-out and outside-in systems to provide software that listens to customers, interprets signals, automates action and coordinates notifications and actions of customer-facing teams.
Recognize Limits: An old adage holds that business would be simple if it weren’t for people. It’s also been true that people overestimate the impact of a new technology in the short run (and underestimate it in the long run). Potential uses of AI and agents are infinite. A large number are readily accessible today to a full range of users. But not all. Some of the potentially most powerful require a substantial foundation for data, architecture and process. The last-named can pose major barriers if the so-called process exists in name only. In other instances, the “process” is really a set of multiple permutations created to address the needs of specific business groups.
Some processes exist in name only and cannot be usefully modeled. In some organizations, sales forecasting, quota setting and territory alignment is part mathematical and wholly political. These are like the Don’t Walk signs in New York City: Merely suggestions. However, AI and agents may be used in these cases to reduce the administrative overhead and perhaps to become more efficient. It’s also the case that processes morph over time in response to business conditions, and some are designed to favor one group or department without regard to their impact on others. Process mining and process analytics are techniques designed to be used to address these use cases but are still a work in progress for many enterprises.
AI and agentic systems have considerable potential to achieve significant increases in the productivity (not just efficiency) of business processes of all kinds. Some are here and now, but many will take years to become feasible and cost-effective. It’s important to recognize that people-centric processes are different from those that do not involve humans and that their design and maintenance have to respect laws, regulations and the possible impacts of unintended consequences.
I strongly recommend that enterprises take an ongoing, pragmatic approach that reasonably tests the limits of what’s achievable, recognizing that this will evolve over time. With a skeptical eye, they should assess their ability to gauge the maturity of their AI and agentic agents because the devil is in the details. As with any assessment, they must test use cases against measurable outcomes. These not only include productivity-based measures such as reduction in cycle time, exception rate and manual effort, but also for possible negative outcomes and how best to deal with them. Issues such as governance capabilities, auditability, integration depth, HITL controls and lifecycle management are especially important.
Regards,
Robert Kugel
Fill out the form to continue reading.