While Ofcom said there was no evidence that Mr Murdoch knew of wrongdoing at the News of the World or that he was complicit in a cover-up, it hit out at his failure to uncover the problems earlier.
It said: "We consider James Murdoch's conduct, including his failure to initiate action on his own account on a number of occasions, to be both difficult to comprehend and ill-judged."
Media regulator Ofcom carried out a review of Sky after James Murdoch and News Corporation, which owns 39% of its shares, were engulfed in the phone hacking scandal which led to the closure of News Corp's News of the World. Ofcom found no evidence that Sky was directly or indirectly involved in any of the wrongdoing either admitted or alleged.
Mr Murdoch - son of media mogul Rupert Murdoch, who controls News Corp - has since resigned as chairman at Sky in a bid to distance the broadcaster from the scandal, claiming he had become a "lightning rod" for bad publicity. He remains a non-executive director of BSkyB and an executive director at News Corp.
Mr Murdoch apologised for his conduct in a letter to the Culture, Media and Sport Select Committee in March, when he said he should have "asked more questions, requested more documents and taken a more challenging and sceptical view of what I was told".
He had no involvement with News International until the end of 2007, which was almost a year after the sentencing of the News of the World's royal editor Clive Goodman and private investigator Glenn Mulcaire for phone hacking. But after he took over as chief executive, he received an email suggesting that criminal activities were more widespread, although he claimed he failed to read the correspondence fully.
Ofcom added: "We consider that the events set out above raise questions regarding James Murdoch's competence in the handling of these matters and his attitude towards the possibility of wrongdoing in the companies for which he was responsible."
Ofcom's ruling lifts a cloud from Sky, which welcomed the decision and said the regulator was right to conclude that it was a fit and proper broadcaster. It said in a statement: "As a company, we are committed to high standards of governance and we take our regulatory obligations extremely seriously. As Ofcom acknowledges, our track record of compliance in broadcasting is good."
Labour MP Tom Watson told BBC Radio 4's Today programme he was "disappointed" by the finding. Asked if it was a finding he accepted, the MP for West Bromwich East said: "Yes, of course. I am disappointed by it obviously, but not surprised, but I do think it shows the flaws in Ofcom's methodology, they can't do deep investigations in the way that other organisations can and they also do hedge their bets."