The new front: Britain's fight for hearts and minds

The barren and unforgiving killing fields of Helmand are a world away from the green highlands of Africa. But it is here, on the foothills of Mount Kenya, that Britain's new military strategy in Afghanistan, the blueprint for a long war, is being put together. As President Barack Obama unveils his fundamental review of American policy in Afghanistan to combat an "increasingly perilous" situation, the UK, too, is seeking to define its role in a rapidly shifting political and military landscape.

The man carrying out the dry run is Brigadier James Cowan, who first came to prominence when leading his regiment, the Black Watch, in Iraq's "Triangle of Death" four years ago. In a few months' time, he will be involved in a new chapter in the "war on terror" as British commander in Helmand in what is expected to be a particularly turbulent time.

There is a resonance to this. Iraq was then the all-consuming focus of American and British foreign policy with Afghanistan filed away as a relatively easy victory. But as the allies moved on to dethrone Saddam Hussein, the Taliban returned to a security vacuum to launch the real war.

There are other echoes from the past. Sending the Black Watch from Basra to support the American offensive in Fallujah was a highly emotive and contentious issue in a foreign adventure which had become deeply unpopular. While the engagement in Afghanistan has not so far aroused the passions as Iraq did, there is acute awareness that questions will be increasingly asked about the mission as the death toll climbs.

Iraq and Afghanistan have both been painful learning curves for the UK military, but the latter is likely to prove more costly in the long run. A total of 179 military and MoD civilian staff died in Iraq, and numbers have fallen drastically in recent months in the final preparations for withdrawal. In Afghanistan, on the other hand, all but five of the 152 deaths have been since 2006 when the deployment to Helmand took place – and the then defence secretary John Reid declared that he hoped that "not a shot would be fired in anger".

There is now a sense of momentum on the ground in Afghanistan with the American military and diplomatic offensive taking place against a backdrop of impending national elections in which the future of President Hamid Karzai – who has fallen out of favour with his Western sponsors – is in doubt.

The test for the UK is how to play a role of influence in Afghanistan with a force that may be the second biggest international contingent, but is still small compared to the expanding American presence, and an aid budget dwarfed by that of the US.

Brigadier Cowan's approach , "war among the people" seeks broadly to position British forces as the conduit for winning hearts and minds. It moves the emphasis away from concentrating on fighting towards winning the consent of the population, hastening reconstruction and laying the groundwork for reconciliation. There is focus on avoiding "collateral damage", casualties among civilians which has caused much anger within Afghanistan and led to President Karzai repeatedly protesting to the Western powers. Soldiers are being taught to engage in shuras and loya jirgas – modelled on meetings with local people in Iraq and Afghanistan – and engage in development projects, building schools and hospitals.

The plan is being put into practice in one of the biggest-ever exercises carried out by British forces abroad. The manoeuvres in Kenya, near Nanyuki, by Brigadier Cowan's 11 Light Brigade have been observed and approved in a visit by some of the most senior officers in the British military including General Sir Richard Dannatt, the head of the Army, Lieutenant-General Graeme Lamb, a former SAS chief and deputy to the American General David Petraeus in Iraq, Major-General Nick Carter, who is going to be taking over command in Kandahar, and Major-General Kennett, the Army's director of training. The strategy also has the support of General Sir David Richards, who will be taking over from General Dannatt.

Some of the most highly regarded young commanders, all with recent battlefield experience, as well as senior diplomats working the region, believe that old policies need reviewing. They stress the need for more open-mindedness and say that negotiations with elements of the insurgency is inevitable in the future.

Brigadier Cowan used muse about the demands of a new type of war with a small group of journalists, including me, who accompanied him to Camp Dogwood. This week, in Kenya, with his plans coming to fruition, he reflected: "We have to adjust. War between nations has been replaced by war among the people where it is absolutely vital that it is not just about defeating the enemy, but, most importantly, winning the consent of the people.

"This is a much more complex form of warfare and the training we are carrying out is tailored to that. We are absolutely trying out things here which have not been tried out before. We believe that this will apply not only to Afghanistan but other such conflicts in the future."

Tony Blair's government, fearing a public backlash, attempted to delay the announcement of the Black Watch heading for Fallujah as long as possible. Brigadier Cowan recalled: "We started to hear rumours about three weeks before the deployment, at the end we were on six days' notice to move. One learns from one's experience and I was determined to be as well prepared as possible this time. I have been to Helmand four times. We started preparing from October 2007. We have had fairly significant counter-insurgency seminars held at Sandhurst. We have had a lot of cultural training, lessons in Dari and Pashtu. This is a fairly sophisticated approach."

Other senior officers heading for Afghanistan were surprisingly frank about the need to embrace fresh ideas. Lieutenant-Colonel Rowly Walker, commanding officer of the 1st Battalion, Grenadier Guards battlegroup, who has served in Iraq and Afghanistan, said: "What is being said, I suppose, is 'it's the people, stupid'. Simply, we have to win over the community. It could only be a good thing to talk, to find some common ground. Did you ever meet any Taliban? I bet they think they are right. I think I am right. The answer probably lies somewhere in the middle, with the proviso the Afghan people have to agree to agree to that answer.

"Did we win in Iraq? It is too early to tell. But we all have moral responsibilities and so, of course, we must learn the lessons from Iraq. But, we must learn from the positive things in Iraq as well."

For Major James Bowder, also of the Grenadier Guards, "it's a no brainer. The fact that James Cowan is being sent on this mission and what we are trying here shows the British Army is prepared to adapt".

The arrival of the American reinforcements, with their formidable arsenal, will help this process of adaption. The future role for its soldiers may lie in combing military duties with establishing secure communities which allows the growth of civic society and also allows scope for coherence in what have often been confusing messages from London.

John Reid's "no shots" forecast seems risible after the discharge of six million bullets by British forces. Yet at the time of the deployment the Government was indeed insisting that the mission statement was primarily one of reconstruction. In Kabul, General Richards, who had become the first non-American commander to have US troops placed under him since the Second World War, had his own "inkspot" strategy under which main towns like the provincial capital, Lashkar Gar, would be secured and the process gradually spreading out elsewhere.

Instead the first wave of British combat troops, the 16 Air Assault Brigade under Brigadier Ed Butler charged off to fight the Taliban in remote outposts like Sangin and Musa Qala, setting up isolated platoon houses which immediately began to draw waves of Taliban attacks. Fifteen British soldiers died and "inkspot" and other theories went out of the window.

The size of the UK force has doubled since 2006 to the current strength of just over 8,000. But it was never going to be enough to hold ground seized from the Taliban and the troops faced the frustrating sight of seeing gains slip back. At the same time the violence sweeping through Helmand meant that development projects were being stymied. The handicap became acute during the long summer of bitter fighting in 2006 with military commanders demanding that reconstruction projects should take place on the ground to show the Pashtun population that they would have some compensation for their land being turned into a battleground, but the Department for International Development was adamant that this was simply too dangerous in the middle of a war.

Garmsir, on the other hand, is an example of what can be achieved with adequate numbers of boots on the ground. The town and its hinterland in the deep south of Helmand provides the supply route which keeps the insurgency watered and armed, and a "blooding ground" where young jihadists cut their teeth. I was there in 2006 when the main urban centre, called "the snake's head" because of its topography, had been taken back and lost in almost constant fighting. One attempt to wrest control was undertaken by just 17 UK troops alongside 10 Estonians and 200 Afghans. Last year a force of 2,000 US Marines backed by massive firepower captured Garmsir. Returning a month ago I found it now had a thriving market and a British-led stabilisation team under a UK official, Ian Purves, successfully implementing civil projects.

Rahimtullah Khan, a shopkeeper in a nearby village, was typical of many Afghans who have learned to hedge their bets during 30 years of war. He told British soldiers how glad he was they were there to protect him and his community from the Taliban. He later said to me – non-white, non-soldier – that the insurgents were not far away, the police were corrupt, and he remained unsure about the future. He had stayed on during Taliban times and no doubt said the appropriate things at the time to ensure his survival.

As we were leaving, there was an urgency in his voice. "No one wants all this fighting. We are tired. We are no different from anyone else, we just want to live and look after our families. I look at these foreigners and think 'You say fine things, will you stay to see we get the new clinic and more things for the school? I will vote in the election, but will these foreigners stay to protect me if the Taliban comes to kill me because I voted? These are the things we need to know."