[troyb]

If you uncheck, "Bring panel forward if it isn't frontmost" in the download preference pane does the problem go away?