Thanks for the question, and welcome to the new forums.
'Height” - references to height (for example, the heights of the eaves on a house extension) is the height measured from ground level. (Note, ground level is the surface of the ground immediately adjacent to the building in question, and would not include any addition laid on top of the ground such as decking. Where ground level is not uniform (for example if the ground is sloping), then the ground level is the highest part of the surface of the ground next to the building.)
Source of definition
In other words wherever you put the outbuilding, the wall on the lowest part of the land, may well be higher than the maximum allowed under Class E of Part 1 as you measure from the highest point.
https://PlanningGeek.co.uk/outbuildings