Brandon Rainey, 21, was challenging the Department of Justice for ordering his recall amid fears he had taken drugs and posed a increased danger to the public.
But a judge in Belfast ruled it had been a lawful decision based on signs that he may not adhere to conditions.
Mr Justice Maguire said: “It surely could not be right that those whose job it is to manage the risk represented by a prisoner on licence have to don a blindfold in respect of pre-release conduct and stand back and await an actual breach of licence – which could have very serious consequences – before taking any action.”
In 2015 Rainey was jailed for the rape, sexual assault and attempted rape of a 12-year-old girl when he was aged 17.
Larne FC suspends player John Herron after he was spotted on camera wearing ‘tiocfaidh ar la’ assault rifle top
Irish republican rebel band Wolfe Tones lead giant Belfast crowd in singing Up the Ra ‘on the same night as singing Give Peace a Chance’
Feile organisers asked if they have warned Wolfe Tones about leading young people in pro-IRA chants
‘He needs to leave our club NOW!’ Strong reaction Larne FC’s John Herron being pictured in ‘tiocfaidh ar la’ gun top as club suspends him
Irish language group Kneecap accused of ‘grooming sectarian hatred’ with Feile mural
The victim’s mother caught the pair in bed together after they had exchanged social media messages.
Rainey was sentenced to two years in custody followed by an extended four-year period on licence.
On April 1, 2016 he was released on those licence arrangements, only to be recalled to prison around five hours later.
The department took the decision following requests and recommendations from the Probation Board and a Parole Commissioner.
Rainey had been assessed as posing a high likelihood of reoffending.
He was regarded as one of 20 in the most serious Category 3 offender group in Northern Ireland – with only three of them at liberty.
Reference was also made to a further exposure offence committed while he was in custody.
A range of accommodation options considered in the run-up to his release on licence were considered unsuitable.
Housing him with relatives in Ballymena, Bangor and Lurgan was ruled out, while no consent was provided for a possible referral to a secure personality disorder facility in England.
Hostel arrangements were deemed too risky after Rainey alleged issued a threat to slash other residents or staff, the court heard.
Instead, a plan was developed which involved him going immediately to the Northern Ireland Housing Executive to see if any appropriate accommodation could be found.
But on the day of his release the prison claimed he had “accessed substances and had been under the influence overnight”.
Rainey was also said to have refused a drugs test, stating that he had taken cannabis and would fail due to taking cannabis.
In the recall request a Probation Service area manager set out that the prisoner “stated that it is his intention to ‘get off his head and have sex on the day of his release’.
“Mr Rainey has already achieved the first of these.”
Based on the information provided, the department decided he must be returned to custody because the increased risk could not be safely managed in the community.
Rainey claimed he was entitled to automatic release on licence and denied being under the influence of illicit substances on the day he left prison.
He did not dispute once telling a forensic psychologist that on his first day of release he would get drunk and have sex – but insisted it was in response to a direct question about his ideal first day outside jail.
His lawyers argued that the recall was unlawful and incompatible with the European Convention on Human Rights.
But in a newly published verdict, Mr Justice Maguire dismissed the challenge.
“Those concerned with the risk the applicant represented in this case were entitled, indeed bound, to consider the signs that the applicant may not adhere to licence conditions,” he said.
“His apparent consumption of drugs; his refusal to take a drugs test; and his non-compliance with medication cannot be regarded as extraneous matters which have no relevance to the issue of recall.”