The more immigration that an area had, the more likely it was to vote Remain. There were other things at play in Scotland and Northern Ireland, but in England and Wales, which were decisive of the result, the Leave vote was carried by those who had suffered most as a result of the economic policies of the previous 39 years, since Callaghan's turn to monetarism in 1977, but especially from the 1980s onwards. Most of those areas had very little immigration, and many of them had practically none. That was simply not what this was about.
If your complaint against the EU is that "it was only ever supposed to be a free trade area", then you cannot complain about the Customs Union, which is the free trade area. If you think that things only went to the bad after the fall of Margaret Thatcher, or in the last days of her Premiership, then you cannot complain about the Single Market, which she devised, implemented and championed, branding as "Loony Left" the people who presciently opposed it at the time.
But those who had the most profound complaints both against the Customs Union and against Thatcher's Single Market were those whose votes delivered the result for Brexit. And those complaints were precisely why they voted as they did.