Actually when it comes to open world driving games for Me the actual map is more important than the driving physics if they are at least half decent.
Lets look at it this way you got FH3 with great handling all the nice cars and the game just gets boring after 30 minutes of cruising because youre basically doing laps around the map, then you got The Crew which has huge detailed map but we all know what the physics are, yet I enjoyed cruising in TC for hours even alone I could just jump into one of my favourite cars and just drive, it felt like an actual road trip.
But when it comes to handling Id say FH3 with TDU1 close second, third probably Driver SF.
On topic of FH i acutally didnt like FH1s it was too twitchy, like the cars felt too light, espeially smaller cars try driving the 205 Turbo 16 in FH1 its almost impossible to drive that thing straight.