There were reports of minor skirmishes between some of the workers and police, as the demonstrators marched on Blackfriars and the site of the Shard at London Bridge.
However, no arrests have made and organisers said they expected the protest to be peaceful.
Electricians, plumbers and engineers from across the country joined the rally at the Blackfriars Station construction site, and will later lobby MPs in Parliament.
Unions claim that de-skilling and pay cuts are being imposed by a group of construction firms and have been staging protests at building sites across the UK in recent months, including Sellafield in Cumbria, Grangemouth and Ratcliffe power stations and Blackfriars and King's Cross railway in London.
Unite is due to give details of a strike ballot among its members who work for Balfour Beatty.
General secretary Len McCluskey said: "Highly skilled workers in construction are facing an unprecedented attack on their skills.
"If Balfour Beatty and their fellow conspirators get their way, eight out of nine workers will lose one third of their pay and will lose any voice they have over their working lives. These employers have threatened to sack any worker who does not sign up to their employers' charter."
The Heating and Ventilating Contractors Association said it was working with seven major British construction firms to introduce a new single national agreement next March to replace five separate deals drawn up 40 years ago in the building engineering services sector.
Chief executive Blane Judd said: "The claims by the Unite union that the new proposed agreement will lead to what it calls massive pay cuts and workers heading for the sack are total fiction. No-one will take a pay cut - in fact 30% will see an increase in their pay packets, and no-one will lose their jobs."
Unite announced that its strike ballot will start on November 16 and will close on the 29th. It warned that any action could hit some of the UK's key infrastructure projects, including power stations and London's Crossrail.