I think a lot about the risk of Human credulity in the decision making process when leveraging AI.
AI did not invent the habit of trusting a confident answer though, you know those expensive consultants with great templates got there years ago. What AI changes is volume. A CPO who once saw three polished forecasts or strategies a year sees thirty before lunch, each one fluent, well formatted and sure of itself. The model has no reason to sound uncertain and the buyer has no time to check anyway. Somewhere between the two, a number nobody verified lands in a supplier negotiation or buying plan.
Additionally AI precision looks like proof..two decimal points usually the least checked part of the slide.
At Kodiact we treat credulity as a design problem and blaming people for believing a confident machine gets you nowhere. It is really important for our customers that when they present a finding and suggest a change in tactic or strategy that they own it.So every output shows where the data came from and how the system reasoned its way to a recommendation. Each figure traces back to it's origin, the contract term, the index price , the reseaach report or news article behind the number.
Users challenge any output directly. If a cocoa butter recommendation looks wrong, the buyer pushes back, the system shows its working, and either the logic holds or the recommendation changes.
We are also exploring a less popular idea: a short check before the answer is shared or adopted. A couple of questions on the assumptions behind the recommendation. If you don’t know why the system says buy forward on sugar, you have no business executing the trade. I expect a mixed reception. Nobody enjoys a pop quiz from their software.
We want buyers who trust the system for the right reasons. If your team adopted an AI tool this year, try one question at your next category review. Who checked the number, and how?