Unless you’ve been hiding beneath the biggest of rocks for the past couple of months, you will have been inundated with news and views around the implications for chatGPT and other Artificial Intelligence (AI) chatbot platforms in managing reputation.

Like me, you’ve probably given it a lot of thought and begun to dabble. You’ve wanted to know how advanced AI has become at sifting for facts, drawing out opinions and doing some of the legwork around compelling copy creation.

It has certainly opened my eyes further to the potential of AI over and above search, just as the stories about experimental use to create content - particularly fake photos - have made toes curl.

But what about the potential for AI to simplify and bring more rigour to ESG reporting and analysis? Will we start to see company stakeholders using the likes of Bard and chatGPT ‘informally’ to understand reputation factors, and investors using them in a structured way as part of their assessment of corporate performance, particularly given the renewed greenwashing warning this week as the number of ESG-labelled funds doubled?

The noise around investors using AI for ESG-related decision-making is also building slowly. This S&P Global piece outlines how investors can use it to collect and analyse far greater volumes of information when assessing risks and opportunities. Meanwhile, the World Economic Forum has said that without AI, we won’t meet ESG goals and address climate change

So I conducted a very rudimentary experiment.

Firstly, I chose three FTSE companies, using chatGPT to understand which currently had the most far-reaching ESG goals in certain areas and had taken broad action to try to achieve them. One company for an E goal, one for a S and one for a G.

Then I asked chatGPT a basic but fundamental question: tell me about <COMPANY X> reputation and sentiment for <ESG goal>. Of course, no-one considering an investment recommendation or wanting to understand a company’s story in one area would stop at that, but it seemed a fair and reasonable parallel for where you might make a start with regular boolean search.

The analysis that chatGPT threw back was surprising - in that it was surprisingly one-dimensional, plucking out what seemed to be a standard list of positives with little or no context, and no balanced assessment of history, sector or the scale of the challenge.

It read like what a student might write for a GCSE exam on a well-worn topic, if they were aiming to just about scrape a pass.

Looking at the E goal company, the response was: “The company has set a target to achieve net-zero carbon emissions across its entire value chain by 2050. .. it has received positive recognition for its commitment to reducing its carbon footprint and addressing climate change. The company has been included in various sustainability indices, such as the Dow Jones Sustainability Index and the FTSE4Good Index.”

And: “..the company's announcement of its net-zero target was positively received by investors and analysts, who saw it as a significant step towards mitigating climate change and reducing business risk. Overall, <COMPANY> has a positive reputation and sentiment for its net-zero commitment and efforts to reduce its environmental impact.”

Looking at the company with a S goal around profoundly improving diversity and inclusion, I was informed that the firm “has received positive recognition for its efforts to promote diversity and inclusion within its workforce and leadership teams. The company has been recognized in various diversity and inclusion rankings, such as the Stonewall Top 100 Employers list and the Financial Times Diversity Leaders list. <COMPANY> has also received positive sentiment from stakeholders for its commitment to diversity and inclusion.

There was pretty much the same type, structure, tone and nature of response on the company with a substantial governance goal. It had “made good governance a key priority and has implemented various policies and procedures to ensure transparency, accountability, and ethical conduct,” while it was felt that “ the company's initiatives, such as implementing a code of conduct, establishing a whistleblowing hotline, and implementing an anti-corruption program, have been positively received by external stakeholders.”

If you can see past the US English, what chatGPT seems to be throwing back is a highly formulaic answer that plucks positives from content in the public domain and pays little if any heed to the negatives.

In other words, something that’s of little real use for understanding and managing reputation. All three companies have had very real issues to grapple with in each of these areas, as a few pointed search strings will quickly serve up.

I also ran each of the ESG dimensions for these three firms through the reputation analysis platform that my own company uses, for a simple comparison, limited to all content published during 2022. The E company had enjoyed positive sentiment for the first half of the year but remained net-negative thereafter. The S company had improved sentiment in that area from slightly negative at the outset of last year to strongly positive. The G company had significantly net-negative sentiment throughout the year.

In conclusion (and no, this piece wasn’t written using AI), chatbots can be useful at this point for sweeping up the main positive attributes and achievements around a company’s ESG dimensions. Throw more at them and they will learn more, and play around with different questions and you will undoubtedly build a more useful picture.

But as a weaponised tool that unmasks reputation truth, they seem to have a long way to go.

The ESG News Review is written by Steve Earl, a Partner at BOLDT.

Subscribe to our weekly ESG related stories by completing the form below.