The search for new premises in Peterborough has begun, with the aim of completing a move from Bircham Newton by January 2020.
The move remains controversial, particularly among staff, and the Unite union that represents them has been rallying signatories to a petition against it.
However CITB chief executive Sarah Beale said: "This is a necessary step for CITB. British construction firms pay their levy and rightly expect high-quality skills as a result. The creation of a single head office will bring us closer to our customers and create the right environment to ensure that CITB delivers the 'levy in, skills out' focus that industry and government have demanded.
"As a GB-wide organisation, it is vital that we are better connected to our customers so that we can reach important partners – including construction firms – more easily providing a flexible workspace which can accommodate a highly mobile workforce. Our new head office will also be designed to suit the size and mission of the organisation that CITB will become.”
She continued: "The Vision 2020, Future CITB programme is designed to secure the long-term future of CITB; which was by no means certain before the last consensus vote. However we are mindful of the impact these reforms has on colleagues and this decision will not be taken lightly.
"For a number of us who work at CITB, this change will be a challenge as well as an opportunity. Peterborough was selected as the preferred location with colleagues, as well as customers, in mind as it offers the greatest potential to reduce redundancies, therefore retaining some talented colleagues, whilst meeting our future needs. We will be working with Unions and elected employee representatives to ensure their views are heard throughout the change process."
CITB chair Peter Lauener said: "CITB has embarked on a radical transformation programme which will deliver the skills British construction needs. To do this, we will need to become a more flexible, dynamic organisation, working in partnership with employers in all parts of Great Britain and setting the standards for future skills.
"Part of that change will mean establishing a new head office in a more central location such as Peterborough where access to other parts of GB will be easier.
"Meanwhile Sarah Beale and I are committed to doing everything we can to maximise the development of the CITB's site in Bircham Newton and thereby leave a lasting legacy for the local community and the economy of north Norfolk."
Unite regional co-ordinating officer Mark Robinson said: "The CITB's plans will lead to far more than 133 job losses. While there may be a job for some in Peterborough it is simply too far away for it be financially viable for the overwhelming majority of staff.
"The board will be paying millions in redundancy payments and spending huge sums on recruiting new staff.
"In addition, the board's plan to outsource services and divest from direct training later this year is likely to have a catastrophic impact on employment in the area. Unite believes the head office consultation is a smokescreen to hide the actual number of people who will lose their jobs.
"Unite is calling on the CITB's new chairman Peter Lauener to suspend the whole process, undertake a full and independent review and consult with industry to find out what it wants."