Get a simple ssh agent helper on your Linux system.
SSHMan for Linux is a persistant management of ssh agents. Will only prompt for adding ssh keys to the agent one place at any one time. Will not slow down logging into X, the ssh password prompt will be started in the background. Will not try to start an agent if the user is logging in via ssh.