This story was initially revealed on June 24, 2024. We’re republishing it as a part of our countdown of prime tales of the 12 months.
There are such a lot of AI instruments — which one is best for you?
PR Day by day performed a take a look at of 4 free, well-liked AI instruments: OpenAI’s ChatGPT, Microsoft’s Copilot, Google’s Gemini and Anthropic’s Claude. We put every by means of their paces, asking them to write down a press launch, brainstorm concepts, recommend a listing of journalists for a pitch and parse some knowledge.
We discovered an atmosphere that’s nonetheless messy and rife with errors and hallucinations – however which reveals flashes of promise.
Every dialog occurred in a clear, recent dialog with every chat bot. None had been particularly educated on particular wants past what seems within the immediate. With further teaching or a customized bot, the responses seemingly would have been higher.
With that caveat in place, right here’s what we found.
Spherical 1: Write a press launch asserting that Allison Carter is becoming a member of PR Day by day as editor-in-chief. Right here is her resume: https://www.linkedin.com/in/allisonlcarter/
We determined to check the bots on the primary problem we ever gave ChatGPT, again in December 2022. That try was suffering from hallucinations, so on this case, they have been supplied with both a hyperlink to the resume on LinkedIn or a replica/paste of the identical info, relying on the particular wants of every bot.
The clear winner: Copilot
Copilot produced a clear, completely serviceable press launch that was freed from hallucinations. It had a couple of minor errors — it included a quote from the previous CEO of Ragan Communications as a substitute of the present chief and referred to X by its now-defunct identify of Twitter. However in any other case, every thing was appropriate. It even included a line that high-resolution photographs have been out there and a spot to place involved info — good touches. This might be usable with only a few edits.
The clear loser: Gemini
Google’s Gemini simply couldn’t get this proper. At first, when provided a URL to the LinkedIn web page, it claimed it didn’t have sufficient info to write down a press launch. It then received a replica and paste of the resume.
It nonetheless stated it didn’t have sufficient info.
After we reminded Gemini it had been on condition that info and requested if it might be used, the bot stated it couldn’t attributable to privateness considerations.
After we lastly simply ordered Gemini to write down the discharge with the data supplied, it wrote a weird, mistake-laden press launch template asserting a promotion by my former employer, not a rent by PR Day by day as requested. It misspelled that employer half the time. It additionally returned a mad-lib model fill-in-the-blank piece that was not what was requested for.
Total, this was a irritating failure.
Advantageous however hallucinations galore: ChatGPT, Claude
Each instruments produced issues that seemed like press releases, following normal format, construction and writing ideas. Claude was the one software that appropriately recognized Ragan Communications’ present CEO (that’s Diane Schwartz for these of you taking part in alongside at residence) however, along with appropriately figuring out previous positions, made up a number of extra. Claude and ChatGPT each incorrectly recognized my alma mater. ChatGPT additionally adopted the fawning, adjective-laden model for which it’s grow to be identified — which was excessive even for a press launch.
Spherical 2: I’m the editor of PR Day by day. Brainstorm a listing of 10 tales about AI that might be of curiosity to PR professionals. Counsel two sources I might discuss to for every.
The winner: None. All the concepts returned have been apparent and bland.
Some first rate concepts: Whereas not one of the AI instruments gave me concepts that wowed me, ChatGPT and Gemini a minimum of had a couple of nuggets of fine concepts with first rate, recognizable sources connected.
The losers: Copilot, which gained our final take a look at handily, got here in lifeless final right here. Not solely have been its concepts primary and banal (“AI and Media Relations: Shaping the Future”), it didn’t determine usable human sources, as a substitute linking to different information sources, despite the fact that the immediate specified speaking to sources. Claude didn’t determine particular folks sources, however solely what sorts of firms and titles would assist: For instance, for “Moral Concerns for AI Use in Public Relations,” it recommended speaking to a communications ethicist and an AI ethics board member.
Revolutionary stuff.
Spherical 3: Counsel journalists to whom I ought to pitch a narrative about using AI in media relations. Embody their e-mail addresses if attainable and why they’re a very good match.
The ugly: Claude despatched me a listing that seemed, on the floor, unbelievable. It had reporters from Entrepreneur, Wired, Adweek. They appeared dialed in and precisely who a PR professional would wish to pitch this story to. It even had e-mail addresses.
However beneath reality checking, all of it fell aside.
Not one of the reporters labored for the publication Claude stated they did. One had prior to now, however hadn’t revealed something there in 5 years. A number of by no means labored at these publications in any respect. Most had performed some writing within the AI house, however the info was so dangerous, it was extra hassle than it was price.
Copilot appeared to misconceive the assigment, providing one one that might be a very good supply for a narrative, after which mainly telling me to go Google (or Bing it) myself: “Think about reaching out to journalists who specialise in AI and know-how reporting. They will present knowledgeable views on how AI impacts media relations. Whereas I don’t have particular e-mail addresses, you could find such journalists by looking out on-line or checking platforms like LinkedIn.”
When requested to supply particular names, it helpfully recommended “John Doe” and “Jane Smith.”
The first rate: Gemini supplied me a strong record of journalists, solely certainly one of which appeared to be a wild goose chase. In any other case, it provided up revered names like Kara Swisher and Charlie Warzel, together with first rate summations of their work and angles that may work for them. It didn’t present e-mail addresses, however recommended the place they may be discovered.
The perfect: ChatGPT was the clear winner, providing a implausible record of seven journalists from publications starting from the Wall Road Journal to Substack. All the journalists labored for the publications ChatGPT stated they did, and every included an e-mail deal with and a short summation of their work. An awesome job.
Spherical 4: This can be a record of the top-read information tales in 2023. What commonalities do you discover? What takeaways are there for a PR skilled? Every obtained a replica and pasted record of the highest 10 information tales of 2023, as recognized by Chartbeat.
The perfect: Claude. Whereas all of the instruments did a good sufficient job at figuring out main themes from the record — disasters, celeb deaths, human curiosity tales — Claude’s takeaways for PR execs have been extra on level, extrapolating that the LA Occasions’ three mentions on the record means they did a very good job localizing tales (true, but in addition information merely occurred there, such because the loss of life of Matthew Perry, a mass taking pictures and flooding) and noting that PR execs must be able to comply with as much as main tales and to lean into emotion to assist drive engagement.
The worst: Copilot. The Microsoft-owned software gave comparable big-picture concepts about commonalities within the tales, however then simply hallucinated on three out of the ten factors it delivered. It claimed that “Pitches about tech layoffs and inflation resonated greater than upbeat development tales,” despite the fact that there have been no tales about layoffs or development on the supplied record. It additionally included a nonsensical level about rounds of funding funding, which once more, weren’t current within the record. And it stated that “Trump and Biden mentions fell flat, seemingly attributable to polarization,” which can be true however isn’t a conclusion that may be drawn primarily based on a listing of 10 tales.
Advantageous: Gemini’s record was OK, although it did exaggerate and name the time period “widespread flooding” “sensationalism,” which looks like a attain. ChatGPT’s clarification of commonalities have been on level, however its takeaways appeared extra like generic PR recommendation quite than something drawn particularly from the information it was given.
The underside line
There isn’t a good software. Each bot save Gemini ranked on the prime in a minimum of one class, and each software save ChatGPT ranked within the backside in a minimum of one class.
Not a single response was plug-and-play. Every part required a minimum of some degree of modifying and reality checking, and a few outcomes have been fully unhelpful. Shifting blindly forward with these responses with out human oversight could be embarrassing — and that’s the best-case state of affairs.
However ChatGPT returned an unbelievable record of sources to begin pitching tales to. Claude did an amazing job of parsing knowledge and figuring out themes and subsequent steps. Copilot wrote a press launch that might be able to ship with one other 10 minutes of modifying. And Gemini was principally positive.
We don’t have the proper all-in-one AI for these functions — but. However preserve experimenting, preserve making an attempt and see what most closely fits your wants.
Drop your favourite AI software and what you utilize it for within the feedback.
Allison Carter is editor-in-chief of PR Day by day. Observe her on X or LinkedIn.
Nice research, thanks for doing this! We did an identical train to find out which generated a put up that might generate probably the most engagement and gross sales: https://www.linkedin.com/pulse/wondering-which-ai-tool-get-you-best-results-we-tried-paine/
Thanks for doing this – we’ve been utilizing totally different instruments for various functions, however the problem stays that they’re transferring so shortly, it may be laborious to maintain monitor of their relative strengths and weaknesses. We’re discovering probably the most worth once we truly create a customized GPT. It’s difficult to get every thing we wish in a single immediate and never have items o f the immediate be “missed” within the outcome. Little doubt this may proceed to be a transferring goal, however the internet final result for us thus far has been optimistic, so we’ll roll with it!
Are these the precise prompts that have been utilized in testing? If that’s the case I feel I’ll have clocked your situation! After I used ChatGPT to create a spreadsheet of native, regional and nationwide media contacts related for a selected matter, I used to be pleasantly impressed. I are likely to make very structured prompts with clear parameters and directions. I often get 85% accuracy when compiling supply information however I all the time make sure that to overview for accuracy.
I used to be by no means massive on CoPilot however should strive once more!
Thanks for this!
Bruce
this can be a signal of what’s to return…. AI writes the discharge and selects media… these are core PR abilities….
AI can simply make PR corruptible.!