The inventory-before-the-new-deadline is the bit I recognise. I have watched firms treat a moved date as permission to park the value question, then find the surprise in procurement and HR rather than in the model team. Naming an owner is the first useful move; cheaper models are not. If the FCA lands a year before the EU date, which of those systems would you still fund if someone had to stand behind the output this quarter?
Procurement and HR is exactly right, and I think it's structural rather than bad luck. The Annex III use cases that bite hardest are hiring, credit and access to services. None of them sit with the model team, and most of them arrived as a vendor feature rather than a build decision. That's why the inventory keeps surprising people. Nobody bought an "AI system". They bought an applicant tracking upgrade.
On your test, I'd use three filters rather than a list.
Can you name the individual, not the function. Does a human actually overturn the output, and can you produce cases where they did. Can you reconstruct why a specific person got a specific outcome, six months later, without ringing the vendor.
The third one is where the funding question resolves itself. Systems that fail it are almost always black-boxed scoring bought on a procurement cycle, and they are simultaneously the most expensive to defend and the cheapest to drop.
Thanks for the response Iga, good points, and it will be interesting to watch this shake out in practice when the questions and challenges start arriving.
The inventory-before-the-new-deadline is the bit I recognise. I have watched firms treat a moved date as permission to park the value question, then find the surprise in procurement and HR rather than in the model team. Naming an owner is the first useful move; cheaper models are not. If the FCA lands a year before the EU date, which of those systems would you still fund if someone had to stand behind the output this quarter?
Procurement and HR is exactly right, and I think it's structural rather than bad luck. The Annex III use cases that bite hardest are hiring, credit and access to services. None of them sit with the model team, and most of them arrived as a vendor feature rather than a build decision. That's why the inventory keeps surprising people. Nobody bought an "AI system". They bought an applicant tracking upgrade.
On your test, I'd use three filters rather than a list.
Can you name the individual, not the function. Does a human actually overturn the output, and can you produce cases where they did. Can you reconstruct why a specific person got a specific outcome, six months later, without ringing the vendor.
The third one is where the funding question resolves itself. Systems that fail it are almost always black-boxed scoring bought on a procurement cycle, and they are simultaneously the most expensive to defend and the cheapest to drop.
Agreed on cheaper models.
Thanks for the response Iga, good points, and it will be interesting to watch this shake out in practice when the questions and challenges start arriving.