A man believed to have been skylarking on a crane at a northern Sydney construction site fell 45 metres to his death this morning, police say.

Three men scaled the site's perimeter fence in Bower Street, Manly, then climbed the crane and walked along its jib just before 12.30am, police said.

Do you know more? Message 0424 SMS SMH (+61 424 767 764) or email us with information.

But one man, believed to have been 29, fell from the jib, which was horizontal, and died.

Police said investigators were speaking to witnesses, but did not think the man's death was suspicious.

Detective Inspector Luke Arthurs of Manly Local Area Command said the man was understood to have spent Australia Day in Manly, "visiting a number of people and at the beach".

A resident at the house at which the man was believed to have been before he climbed the crane would not comment. He and another man at the house were seen clearing the property of beer bottles and glasses.

The house is owned by the trustees of the Roman Catholic Church for the Archdiocese of Sydney and its tenants are Snowsport Australia, a sporting goods retailer. A spokeswoman for the archdiocese would not comment while a police investigation was under way.

A neighbour on Bower Street, who asked not to be named, said the residents had started partying from about 5.30pm yesterday.

A police spokesman said investigators would be looking into whether alcohol may have been a factor in the victim's death.

A spokesman for Kell & Rigby, the construction company that manages the site, would not comment.

A row of seven houses was being constructed at the site, a neighbour at nearby Montpelier Place said.

The man's family was informed and a report will be prepared for the coroner.