Are you talking about the "Files" Tool Window or the "Open" files tool window? The help for "dismiss on select" (in the "File Open" section of the options) indicates the setting is just for the "Open". It didn't seem to have any affect on the "Files" window. No matter what I do, it always closes after double clicking a file.