The philosopher Gilbert Ryle coined the phrase “the ghost in the machine” to critique the fallacy of separating consciousness from the physical body. Today, a new ghost haunts the modern organization: an artificial intelligence that makes consequential decisions without a conscious, accountable mind to explain them. We have embedded a powerful, ethereal force into our HR systems, and it is making decisions with no face we can hold responsible.
This abstraction becomes terrifyingly concrete in moments of crisis. Consider a boardroom in Frankfurt, where a CHRO had just presented a new AI tool designed to identify high-potential leaders. The results appeared compelling until a director asked a simple question: “Why was Maria excluded?”
The system’s response was a digital shrug: “Insufficient data congruence in collaborative leadership markers.”
No one could translate what that meant. Not the engineers. Not the HR team. Not even the algorithm itself. No one knew which facts mattered, how the threshold was set, or what would have flipped the result. Maria, a respected manager with a decade of stellar reviews, had been invisibly deemed unworthy by a system no one fully understood – a modern ghost in the machine.
This scenario represents the reality unfolding in organizations worldwide. AI now routinely shapes careers, determining who gets hired, promoted, or paid fairly. As these systems grow more sophisticated, they also grow more opaque. We are entering an era where life-changing decisions are made by machines that cannot explain themselves, creating a dangerous accountability vacuum where responsibility dissipates into the digital ether. If HR cannot translate algorithmic reasoning into human logic, trust drops, accountability follows it, and leadership takes the heat!
The Three Layers of Unexplainability
The challenge of AI explainability operates at three distinct levels, each compounding the next.
Technical Unexplainability
Some AI models, particularly deep learning systems, are inherently opaque. Their decision-making process involves millions of calculations across layered neural networks. Even their designers cannot always trace how specific inputs lead to specific outputs. A 2023 study by the AI Now Institute found that 78% of HR departments using AI tools could not fully explain how their systems arrived at specific decisions affecting employees. When a promotion algorithm rejects a candidate based on “cultural fit signals,” we must ask: what signals? Who defined them? How are they weighted? This opacity is not merely technical; it is ethical.
Operational Opacity
HR departments frequently purchase AI tools as third-party products. Vendors guard their algorithms as intellectual property, leaving buyers with no visibility into the logic driving outcomes. You cannot explain what you do not own. Consider a real case from my advisory work: A North American tech firm deployed an AI screening tool to prioritize candidates. On the surface, it improved efficiency. But gradually, hiring managers noticed a pattern – the system consistently downgraded applicants from lesser-known universities, even when their experience matched those from elite institutions. The algorithm had not been designed to discriminate. It had simply learned from historical hiring data that favored Ivy League graduates. It optimized for pedigree, not potential. This is the essence of the black box problem. This operational opacity often hides early data filters, cleaning steps, or how vendors chain models together. This is why you must require a Model Card – a vendor factsheet stating intended use, performance across groups, and known limits – for every consequential system.
Institutional Opacity
The deepest layer of the black box is often our own making. Even a transparent model becomes inexplicable when organizational rules are unwritten. If you cannot point to the policy basis for a cut score, meaning the line that separates a yes from a no, the technology is not your primary risk.
The Executive Test: Four Questions for Any AI System
Before any AI-touched decision goes live, it must pass a simple, four-part test. If it fails, it is not ready for real people.
- Can we name the rule? Can we state the policy or business target the system was applying?
- Can we show the pathway? Can we list the inputs and logical checks that produced this specific outcome?
- Can we offer a counterfactual? Can we describe the smallest realistic change that would have flipped the result, so the person knows what to do next?
- Can we guarantee a challenge? Can we point to a human reviewer, a timeline, and the evidence they will consider?
This test transforms an abstract ethical problem into a concrete governance checklist.
The Five-Scene Storyline: A Narrative Path to a Solution
Rather than another checklist, use one coherent storyline for any AI-touched HR decision. This narrative approach turns explainability from a slogan into a defensible habit.
- Scene 1: Set the Scope. State what the system is allowed to decide in plain language before it runs. Example: “Recommend a promotion outcome. Use only delivery against objectives, peer feedback volume, and documented customer impact.”
- Scene 2: Demand a Human-Repeatable Reason. Insist on one short reason code and one paragraph a manager can say out loud. Tools like LIME and SHAP are useful lenses for auditors to see which inputs pushed a prediction, but they support the reason; they do not replace it.
- Scene 3: Make the Counterfactual Visible. State the smallest realistic change that would have flipped the result. This provides a clear, actionable path forward instead of a dead end.
- Scene 4: Guarantee a Real Route to Challenge. Provide a one-click appeal with a time-bound human review outside the original decision chain.
- Scene 5: Keep the Trace. Store the inputs, model version, cut score, and any human overrides. This is how you learn, improve, and defend the decision later.
The Moral Imperative and Legal Floor
Technology does not eliminate ethical dilemmas; it amplifies them. The rise of AI in HR does not absolve us of the responsibility to make fair, transparent decisions. It increases that responsibility.
This is no longer just good practice; it is a legal requirement. The regulatory floor has risen. The EU AI Act classifies HR systems as “high risk,” demanding transparency. Laws like NYC’s Local Law 144 require bias audits and candidate notice. The US EEOC has made clear that civil rights obligations apply to algorithmic tools; if a tool produces a discriminatory outcome, the employer owns the liability.
A vendor that cannot answer fundamental questions, such as which inputs influence the decision most, how they provide counterfactuals, or how thresholds are changed and logged, is not ready for high stakes HR use.
The Path Forward: From Accountability Vacuum to Ethical Assurance
The goal is not to eliminate AI from HR but to master it; to harness its power while ensuring it remains accountable to human values. The leaders who will define the next decade are those who build visible assurance into every automated decision.
This requires a cultural shift from seeing AI as a black box to treating it as a governed asset. It means investing in algorithmic literacy so leaders can ask the right questions. It demands cross functional algorithmic auditing teams to regularly test for bias and alignment with values.
People will accept outcomes they do not like if they can see the rule, understand the path, and exercise their right to challenge. They will reject outcomes they cannot understand. The question is not whether we can understand these systems. It is whether we have the courage to insist that they must be understandable.
Your path is clear. Apply the four part test. Implement the five scene storyline. Demand transparency from vendors. Publish your findings. Do this, and you will not only comply with the new legal floor; you will build trust and lead with decisions you can stand behind.
Critical Considerations for Executive Leaders
- Does our AI explainability strategy pass the four part test covering Rule, Pathway, Counterfactual, and Challenge?
- Are we using Model Cards to demystify vendor systems and create auditable records?
- How are we preparing our managers to articulate the human repeatable reason behind an AI informed decision?
- Is our appeal process for algorithmic decisions a genuine recourse or a token gesture?
- Do our vendor contracts mandate the transparency we need to meet our ethical and legal obligations?
References
- AI Now Institute. (2023). Opacity in Practice: Algorithmic Accountability in HR.
- European Parliament. (2024). Artificial Intelligence Act (AI Act).
- NIST. (2023). AI Risk Management Framework (AI RMF) 1.0.
- New York City Local Law 144. Bias Audit and Notice Requirements for Automated Employment Tools.
- US EEOC. Technical Assistance on Algorithmic Tools under Title VII and the ADA.
- Ribeiro, Singh, Guestrin. “Why Should I Trust You?” (LIME). KDD, 2016.
- Lundberg, Lee. SHAP framework for feature attributions. 2017.
- Mitchell, Wu, Zaldivar, et al. Model Cards for Model Reporting. FAT*, 2019.
- The Alan Turing Institute. Algorithmic Transparency in Organizational Decision Making. 2024.
First published on LinkedIn.