Jeffrey Blue told the court Newcastle United's owner reneged on a promise to pay him a multimillion-pound sum if he increased the firm's share price.
Blue said he had only been paid £1 million of the full amount promised to him.
That claim was rejected by Mr Justice Leggatt, who held that no reasonable person would have thought the conversation in a pub was serious and was meant to be a legally binding agreement and, even if the parties had meant to make an agreement that such an agreement was not sufficiently certain to be enforceable and, still further, than even if that were wrong that Mr Blue had not proved that he was the cause of the rise in the share price.
Ashley also told Mr Justice Leggatt that he could not remember many details of the evening's conversation, because he downed approximately five pints in the first hour.
Mr Blue told the judge that Mr Ashley was a "serious businessman".
The court had heard that Ashley frequently conducted company business in the pub while consuming large volumes of alcohol, and encouraging others to do the same.
In the aftermath of Sports Direct's IPO, which was followed quite swiftly by a profit warning, Mr Ashley pointed the finger at Merrill Lynch for not properly explaining its business model to the City.
Despite hearing this, the judge agreed with Mr Ashley.
So if you thought management meetings were made decadent by the presence of coffee and a plate of generic custard biscuits, you might want to have a rethink.
Ashley was taken to court by investment banker Jeff Blue who alleged that the billionaire promised to pay him £15 million if he brought Sports Direct's share prices up to £8.
Mr Justice Leggett ruled the "jocular" remark made by Mr Ashley was not a contract.
"They all thought it was a joke".
"The fact that Jeffrey Blue deemed that it was genuine only proves the human capacity for wishful thinking has few boundaries".
He called Mr Blue a "lightweight" when it came to drinking and said he would never have been able to keep up.
He said the work ethic at Sports Direct was "like nothing else I have ever seen".