When Ashley was just a boy, his family moved to Preston, and it was there he lived and played junior B hockey, moving up to junior A with the Toronto Marlies and Guelph Biltmores. He had a tryout with the Maple Leafs in 1950, but ended up on the farm team, playing for Pittsburgh and Syracuse in the AHL. When he was sold to Eddie Shore's Springfield team three years later, he retired, believing his pro career had gone as far as it would ever go. He sat out a year, applied for reinstatement as an amatuer, and played another three years before he started refereeing minor hockey games in his hometown of Kitchener.
Ashley moved his way up through the OHA chain quickly, and after just two seasons was signed to an NHL contract, in 1959. He worked mostly in the AHL and USHL that first year, though he did earn a few spot assignments as both a referee and linesman. For the next dozen years after that, he was a fixture in the NHL.
All told, Ashley worked 605 games as a ref and another 17 on the line, 58 more playoff games and 350 minor pro assignments. From 1964 to '72 he was assigned every game seven of the playoffs, true testament to his reputation among both the NHL and the players. In 1971, he worked game seven of the quarter-finals, semi-finals, and finals, an unprecedented series of assignments under the greatest pressure imaginable for an official.
After hanging up his whistle in 1972, Ashley remained active in the game, working as a talent scout for the NHL for young officials across the country after which he retired to Kitchener to work in sales of recreational vehicles.