This article follows the first in the series: AI in investment management: Can firms demonstrate good governance?
AI is already being used across HR functions, from drafting letters and summarising meetings to reviewing applications and supporting performance management.
For HR teams facing growing workloads and limited resources, the benefits are clear. But how much visibility do organisations really have over where AI is being used, what decisions it influences and who is accountable when something goes wrong?
These were among the questions explored at a recent City Hive event with Simmons & Simmons, bringing together HR practitioners and ACT Signatories to discuss the practical challenges of governing AI.
The discussion highlighted five areas where organisations are encountering difficulties - and some of the approaches being taken to address them.
1. AI is entering organisations through multiple routes
One of the biggest challenges is knowing where AI is being used.
A specialist recruitment platform will generally go through procurement, due diligence and risk assessment. But AI is also arriving through enterprise-wide tools such as Microsoft Copilot, existing HR systems that introduce new functionality and processes employees build themselves.
An approved tool does not necessarily mean every use of that tool has been assessed.
Using AI to draft an internal email presents very different risks from using it to rank candidates, analyse employee feedback or recommend performance ratings.
The governance consideration is therefore not just which tools are approved, but what employees are using them for.
Organisations need sufficient visibility to distinguish low-risk administrative uses from activities that could materially affect employees.
2. Human oversight needs to be meaningful
HR presents particular legal and regulatory challenges because decisions can directly affect people's careers, pay and livelihoods.
Simmons & Simmons outlined three overlapping areas organisations need to consider: AI-specific regulation, data protection and existing employment law.
Certain recruitment and employment-related AI systems fall within the EU AI Act's high-risk framework. Data protection obligations also apply where employee or candidate information is processed, while existing employment law continues to govern fairness, discrimination and the justification of decisions.
Performance management provided a useful example - AI can help managers gather and summarise feedback, potentially reducing administrative work and making more information available when assessing performance.
But what happens if that information contributes to a performance rating, which is subsequently used in a promotion or redundancy decision?
Can the manager explain the assessment independently of the technology?
The discussion also recognised that human decisions are not automatically fair or unbiased. AI could potentially improve consistency and help identify information a manager might otherwise overlook.
The important distinction is whether someone can scrutinise the evidence, challenge the recommendation and take responsibility for the outcome.
Human oversight should be built into the decision-making process, rather than treated as a final approval stage.
3. Recruitment needs more than an accurate algorithm
Recruitment emerged as an area where AI offers considerable benefits but also creates difficult governance questions.
AI could allow HR teams to assess far more applications against clearly defined criteria, potentially widening the pool of candidates considered.
However, the discussion highlighted the importance of establishing those criteria before introducing technology, testing whether systems are working as intended and ensuring recommendations are subject to appropriate checks.
One example involved an application that had been ranked highly after relevant keywords were hidden in the CV. A human review identified the issue.
This illustrates why screening systems cannot simply be assumed to produce reliable results.
There is also the question of how recruitment changes when candidates are using AI to tailor their applications.
If CVs become increasingly polished and difficult to differentiate, organisations may need to reconsider how they assess applicants' capabilities altogether.
The risk is that recruitment becomes more dependent on existing networks and recommendations, potentially disadvantageous to candidates without those connections.
AI recruitment tools should be assessed not just for accuracy and compliance, but for their impact on fairness, inclusion and access to opportunity. Firms need to test the outcomes, not simply trust the process.
4. Employees need boundaries - and the confidence to experiment
Another challenge discussed at the roundtable was the ease with which employees can now create their own AI-enabled processes.
The ability to build dashboards, automate tasks and generate code can improve efficiency, but it can also create operational risks when processes become widely used without adequate documentation, testing or ownership.
What happens when the employee who built a process leaves? Does anyone understand how it works, what data it uses or how to fix it?
Participants discussed approaches that distinguish between experimentation and activity affecting core business systems, with the latter requiring additional oversight and approval.
Training was another recurring theme, with some organisations requiring employees to complete training before accessing particular AI tools.
The idea of an 'AI driving licence' was also raised - establishing a baseline level of understanding before employees can use more sophisticated systems.
The aim is not to prevent people experimenting, but to give them the knowledge to do so responsibly.
Cross-functional forums offer another practical approach, bringing together employees to share use cases, discuss problems and identify successful applications that could be adopted more widely.
This also gives organisations greater visibility over how AI use is evolving, rather than leaving individual teams to develop their own approaches in isolation.
Firms need clear boundaries for AI experimentation, backed by training, oversight and accountability. Employees should be encouraged to innovate, but not to create processes that nobody else understands or can maintain.
5. Governance must consider the impact on culture and skills
Perhaps the most difficult questions were about how AI could change the way people work together.
Participants discussed the risk of employees becoming overly reliant on AI-generated outputs, potentially weakening critical thinking and the willingness to challenge decisions.
There were also concerns about junior employees - if AI increasingly performs the tasks through which people traditionally gain experience, how will organisations develop the expertise and judgement needed by future managers and leaders?
Interestingly, the discussion also highlighted that AI adoption is not uniform.
Some employees are enthusiastic, while others remain reluctant, whether because of concerns about reliability, limited time to learn or uncertainty about what AI means for their careers.
That suggests governance needs to consider employee confidence and engagement alongside policies and controls.
Organisations need to make clear not only how AI can be used, but why it is being introduced, what employees remain responsible for and how their roles may develop.
AI governance should include a plan for developing people's skills and judgement, not just their ability to use new tools. Firms must create a culture where employees feel confident using AI, but equally confident challenging it.
From governance policy to governance practice
The discussion reinforced that AI governance is not simply a technology or compliance exercise.
It requires organisations to understand how tools are being used, assess the consequences of particular applications, establish clear accountability and equip employees with the skills to make informed decisions.
For HR, this is particularly important because the outcomes affect people directly.
The challenge is to create governance that is sufficiently robust to manage risk without discouraging innovation.
That means moving beyond lists of approved tools towards practical oversight, training, communication and opportunities for employees to share what they are learning.
Ultimately, the quality of AI governance will be reflected not only in the policies an organisation adopts, but in the decisions people make, the challenges they are encouraged to raise and the culture that develops around the technology.
Next steps for ACT Signatories
The roundtable highlighted the value of sharing practical experiences as firms work through the challenges of AI adoption.
While organisations are at different stages, many are grappling with similar questions around oversight, accountability, training and employee confidence.
For ACT Signatories, the next step is to consider how these issues are being addressed within their own organisations, particularly where AI is already influencing HR processes and decisions.
City Hive is making the presentation materials from the event available to ACT Signatories. These include the legal and governance considerations discussed by Simmons & Simmons.
If you are an ACT Signatory and would like access to the presentations, please contact the City Hive team.
We also welcome further conversations with signatories about the challenges they are encountering and the approaches they are taking to manage AI responsibly.