I'm doing the same thing between Linux and Windows.
Using Windows as the server and Linux as the client.
I regularly notice the following issues:
1) When remote desktopping to the Windows box, I lose all control after a short while & can't regain it until I physicaly stop or restart synergy on the computer.
2) Copy / Paste (between computers) stops working & I'm pasting the wrong password to the Linux box over & over (though a KVM offers no copy / paste between computers).
3) In addition to the normal Windows "CTRL / ALT / Shift Stuck When They're Not" Issues, those keys sometimes seem to stop functioning on the client box, and sometimes, when I try to type on the server box (windows) hitting a key makes the active window go inactiv & I cannot type more than one character at a time without restarting synergy(making everything normal again)
3) On a rare occassion, my cursor gets stuck on the client machine & I have to kill the client software to get back to the main machine, I then have to physically access the client to sart up Synergy on itagain.
These issues may be caused by other software on my machines, and they are all easy to deal with (irritating, nonetheless), but by expecting them, at least you won't be baffeled by them at some point in the future.