Our AI agent made a settlement offer on a debt collection call

We found this during a routine review of call recordings and it was luck because at the time we only listened to a small share of calls. A customer on a debt collection call said the full amount was too much for them at the moment and the agent replied by offering to settle for a lower figure if they paid within the week. The reply sounded reasonable and was close to what a skilled human agent might say. The trouble was that the agent had made the figure up. We had set approved discount ranges for that type of account and this number was very outside of them. The model produced it because the conversation seemed to call for an offer and it had learned from many similar exchanges how such offers usually sound. since every call is recorded, we now had a recording of our own system making an offer that went beyond anything we had approved. If the customer had paid that amount and later raised a complaint, we would have been arguing against our own recording and hence our legal team took far more interest in this than our engineers did. We then took every decision involving money away from the model. It now chooses from a fixed list of approved offers based on the state of the account while the actual figures come from a lookup table and the same applies to payment dates, waivers and anything else that changes what the customer owes. a tradeoff is that the conversations become stiffer. At the point where money comes up, the agent stops sounding natural and starts sounding as though it is reading from a list, which is exactly what it is doing. Fewer customers who tried to negotiate ended up agreeing to pay so we gave up some results in exchange for the certainty that the agent only says what we have approved. It still feels like the wrong solution since we have more or less built an old phone menu with a friendly voice on top for the part of the call that matters most and the model only gets to use its judgement where the stakes are low. considering our current setup, what can I do to make it even more efficient and what setup do you guys have ?