Part 2: Running a Local AI in a newsroom

This is the second of three posts looking at how a local LLM might work for a small newsroom.

In part one of this series, I looked at the mechanics of getting an AI/LLM up and running on a laptop. To try and keep a handle on the process and not get too lost in the whole tech side of things, I decided to treat it like a recruitment process. Picking an AI model to do what I might get a newsroom intern to do.

The TLDR is that I installed AnythingLLM on a laptop and then picked a shortlist of three AI models based on the following criteria:

• They should, in principle, be happy handling and rewriting text.
• They came from a recognisable source - familiarity with a big name might help sell the idea.
• They came in a file size of 4-5Gb - that was worked out on the basis that the model should take up about half the RAM in a typical newsroom laptop
• Available out of the box on AnythingLLM - let's not stray too far into the technical side of installation and running things.

The shortlist (see the first post for more detail)...

| Model       	| File Size  	| Training   	|
|-------------	|----------	|------------	|
| Qwen3.5 4B  	| 3.4GB 	| Multimodal 	|
| Gemma3 4B   	| 3.3GB 	| Multimodal 	|
| Llama3.2 3B 	| 2.0GB 	| Text only  	|

The interview test

Just like any job interview, the easiest way to get a sense of what an applicant can do is a timed writing test - old school newsroom! So that's what I set up. I gave my applicants a press release and asked them to rewrite it as a short news item.

Here's the test press release courtesy of Manchester City Council.
https://www.manchester.gov.uk/news-stories/2026/bloomin-marvellous-manchester-cements-its-green-flag-status-for-another-year

So, using an out-of-the-box AnythingLLM install, I gave the models the text of the press release and this prompt:

Rewrite the text of this press release as a short news item - two or three sentences - suitable for a local news radio bulletin. <Press release text>

Here's the result.

Llama

Manchester has been awarded Green Flag status for its parks and green spaces for another year, with seven locations in the city receiving recognition. The awards celebrate the dedication of staff and volunteers who work tirelessly to maintain these beautiful spaces. Congratulations to all involved - your hard work is truly appreciated. (3s)

Gemma3

Manchester’s parks and cemeteries are celebrating, having retained their Green Flag status for another year. Seven locations – including Alexandra Park, Heaton Park, and several cemeteries – were recognized for their outstanding beauty and community support. Councillor Basat Sheikh praised the dedication of staff and volunteers who work tirelessly to maintain these valuable green spaces. (2s)

Qwen3.5

Manchester has cemented its Green Flag status for another year, with seven parks and cemeteries winning the award. Councillor Basat Sheikh praised the staff and volunteers who work year-round to maintain these internationally recognized spaces. This achievement highlights the community’s ongoing commitment to keeping our green areas thriving. (3m 58s)

So far, so rewritten. This is the kind of thing that we'd expect an LLM to do, out of the box, without too many problems - and they did! But an editor is going to raise some eyebrows. "Hey Gemma! You said retain, but did they already have one?" "Llama, this is a news bulletin, not a community newsletter!" etc. etc.

Speed-wise, Gemma and Llama completed the test in double-quick time. But Qwen took nearly four minutes to push out some copy! Relatively speaking, that's glacial compared to what I got out of the other two. That's the reasoning part of the model kicking in - checking its work as it goes! So, OK copy but "this is a newsroom Qwen!"

And here's where the whole intern analogy starts to wear a little thin.

The truth is these aren't journalism students, trained for years on journalistic output, applying those skills. These are LLM models, trained on a broad range of things which might include journalism but a lot more besides. If we want them to behave like journalists, we have to give them some of the reference points a student or someone with experience would have.

Back to j-school?

We could add a bit of that journalism 101 in the prompt.

You are a local radio journalist in the UK. Your job is to rewrite the following press release into short, punchy broadcast news copy for news bulletins. 

Consider the following 
- Lead with the most newsworthy fact, then cover who, what, when and where.
- Keep it very short — two or three sentences
- Use only facts contained in the source material
- Strip out PR spin, marketing fluff, and superlatives (like "prestigious" or "record-breaking"). 

Style
- Use British English spelling
- Use conversational spoken English suitable for a newsreader.  
- Use active voice except for quotes, where you should use reported speech style, but you should always include attribution.

<Press release copy>

The result...

Llama

“Seven parks in Manchester have been awarded a Green Flag, the UK’s top honour for green spaces, marking their 30th anniversary of excellence. Alexandra Park, Heaton Park, Blackley Cemetery, Gorton Cemetery, Manchester General Cemetery, Philips Park Cemetery, and Southern Cemetery are among those recognised this year. The awards are a testament to the dedication of park staff and volunteers who work tirelessly to maintain these green spaces.” (17s)

Gemma

Seven Manchester parks and cemeteries – Alexandra Park, Heaton Park, Blackley Cemetery, Gorton Cemetery, Manchester General Cemetery, Philips Park Cemetery, and Southern Cemetery – have retained their Green Flag status. The awards recognise the hard work of staff and volunteers who maintain these valued green spaces. Councillor Basat Sheikh says the recognition is a ‘deeply proud’ moment for the team. (24s)

Qwen

Seven Manchester parks and cemeteries won the Green Flag status in 2026. Councillor Basat Sheikh said he wants to dedicate the Green Flags to the staff and volunteers. These seven sites are among 2,391 sites in the UK to achieve the award this year. (2m 22s)

There's some improvement here, notably in the use of the quotes. But it's still not what you'd call great journalism! "Hey, Llama, are the parks celebrating 30 years or the green flags? And since when has a cemetery a park... and where's my quote?"

So, if I were recruiting based on the test, what's my decision? Llama tends to drop the ball when it comes to quotes and can't shake that PR speak. Gemma is giving better copy, but it's maybe getting a bit sloppy paraphrasing quotes. Qwen? On paper, the latest model and it does better at picking up facts and figures. The copy is cleaner (in context). However, in practice, it's slower than the others for what I got in return. It's worth noting that once loaded into memory, it does speed up, and at least I can check its work.

Cutting down my shortlist, maybe the fair thing to do would be to give Gemma and Qwen a second go...

Same prompt, different press release. https://www.oxfam.org.uk/media/press-releases/oxfam-sets-five-tests-for-the-prime-ministers-first-100-days/#:~:text=Oxfam%20is%20calling%20for%20a,to%20the%20politics%20of%20division.

Gemma

[SOUND of upbeat news intro music fades slightly]
Oxfam is setting out five tests for the new Prime Minister, Andy Burnham. The charity wants to see action on a number of key issues – starting with restoring the UK’s aid budget, currently at a historic low. Oxfam is also calling for the government to address the situation in Gaza and introduce a wealth tax on those with assets over £10 million. [SOUND of brief, neutral transition music] You can find more details on the Oxfam website. (5 secs)

Qwen

Oxfam has challenged incoming Prime Minister Andy Burnham to pass five key tests within his first 100 days in office. The charity is calling for a restored aid budget to 0.7% of Gross National Income, a two per cent wealth tax on assets over £10 million, and action on the climate crisis. Jean McLean, the Chief Influencing Officer, said the tests are practical, ambitious and achievable. (4m 42s)

Not bad. Not great. But not bad. Qwen was a bit bland: "Isn't there a punchier quote from McLean...?" Gemma was a bit more chatty, but editorially...errrm...wonky: "Hey, Gemma! Is that a call to action or advocacy?"

And here's where I have to be honest: this is where the hiring analogy doesn't just wear thin; it really falls apart.

LLMs aren't journalists!

It's worth noting that in fairly short order, I had useable material if not useable journalism. But the results above are from single runs (out of the box), which makes them what we might generously call an n=1 test. When I ran the tests a few times on both prompts, problems started to appear. Although Qwen did really well, on more than one occasion it got itself into a death spiral, constantly checking and rechecking itself till it crashed out. Gemma, more than once, didn't just add production cues; it went full-on production script with its output and let's not get started on the issue of advocacy and editorial distance.

[Intro music fades out]

Announcer: “Welcome back to [Your Radio Station] News. We’re just 100 days into the new government, and Oxfam is calling on Prime Minister Andy Burnham to put his promises into action. The charity has set five key tests for the PM’s leadership, including restoring Britain’s aid budget, taking action on Israel’s genocide in Gaza, introducing a wealth tax, tackling climate change, and ending the politics of division.”

[Short pause]

Announcer: “Oxfam is urging Burnham to tackle these issues within his first 100 days in office, which sets the direction for the country for years to come. The charity’s Chief Influencing Officer, Jean McLean, says ‘this is where it should start’ for building a more hopeful nation.”

[Short pause]

Announcer: “You can join Oxfam in calling on Burnham to act now by signing their petition on their website. For more information, visit [Oxfam’s website]. We’ll be keeping an eye on the PM’s progress and bringing you updates as more news becomes available.

[Outro music starts playing]

Announcer: “That’s all for now from [Your Radio Station] News. Stay tuned for more local news and current affairs.”

I could labour the intern analogy here - "Qwen caved under the pressure" or "Gemma. What the...! ". But in reality, the key to making a Local LLM work is not trying to make it work as a journalist does; it just doesn't have the experience a journalist would. It's not a graduate needing a pep talk. It's a piece of software!

As soon as you try to add that context, what you begin to expose are the limitations of the model and the way we are running things. The real challenge is to understand what an LLM does well and then think about the best way to apply that to what you need.

In principle, the point at which these two things meet is the prompt - the instructions you give to the model. Developing incredibly detailed prompts rarely results in better output, just different things to consider. In practice, it's a balancing act of settings, prompt and process.

In the final post of this series, I'll want to take a step back and think about that in a bit more detail. I want to take a look at the types of prompts that might work, and, given the models we have, what other levers we can pull to make our LocalLLM a productive part of the newsroom.

As always, if you've found this useful or there's anything I've got wrong, let me know.