Desperately Seeking Lector

In Silence Of The Lambs, Starling notices the pictures Lector had done of the Palace in Florence. She quizzes him "all that memory just from detail?", to which he replies "Memory is what I have now I am denied a view."

It wouldn't take a genius to figure out Lector would want to go and see that view again once he was free.

In Hannibal, he does indeed go to Florence. In fact, he settles there and even gets a job.

So why did they never think to look for him there? Right in the place where he knew so well he drew from memory?