The
dune_artdaq
repository (among others) is hosted on the fermilab git repository, so changes need to be pushed there. But
np04-srv-*
machines are behind a firewall, so external traffic has to go through the web gateways or through ssh tunnels on
lxplus
. There's an additional complication that both CERN and FNAL use kerberos for authentication, so you need both CERN and FNAL tickets to be active in order to tunnel through
lxplus
to the fermilab git repos.
All of the following steps are done on the
np04-srv-*
machines and only need to be done once, unless otherwise stated.
The first step is to set up a modified kerberos configuration. Save the following as
krb5.conf.fnaltunnel
(say) somewhere that's accessible on the
np04-srv-*
machines:
[libdefaults]
default_realm = CERN.CH
ticket_lifetime = 25h
renew_lifetime = 120h
forwardable = true
proxiable = true
default_tkt_enctypes = arcfour-hmac-md5 aes256-cts aes128-cts des3-cbc-sha1 des-cbc-md5 des-cbc-crc
# PAR 2018-03-12: Use the "DIR" type of credentials cache so we can have multiple credentials
# See:
# https://web.mit.edu/kerberos/krb5-1.12/doc/basic/ccache_def.html
default_ccache_name = DIR:/tmp/krb5cc_%{uid}
[realms]
# PAR 2018-03-12: Copied from /etc/krb5.conf since we don't need any proxying funny business for CERN
CERN.CH = {
default_domain = cern.ch
kpasswd_server = cerndc.cern.ch
admin_server = cerndc.cern.ch
kdc = cerndc.cern.ch
v4_name_convert = {
host = {
rcmd = host
}
}
}
# Tunnel fnal kerberos through port 8899. Needs a proxy on that port, eg via:
# ssh -L 8899:krb-fnal-1.fnal.gov:88 -N lxplus
FNAL.GOV = {
kdc = localhost:8899
admin_server = krb-fnal-admin.fnal.gov
default_domain = fnal.gov
}
[domain_realm]
.cern.ch = CERN.CH
.fnal.gov = FNAL.GOV
[kadmin]
default_keys = arcfour-hmac-md5:pw-salt
(modified from
krb5.conf.tunnel
at
here
)
Make sure kerberos commands see this config file by setting
$KRB5_CONFIG
(eg
export KRB5_CONFIG=/path/to/krb5.conf.fnaltunnel
in your
~/.bashrc
). Also make sure the environment variable
$KRB5CCNAME
is
not set (eg
unset KRB5CCNAME
in your
~/.bashrc
), since it will override the setting in
krb5.conf.fnaltunnel
.
krb5.conf.fnaltunnel
tells kerberos commands to try to connect to the FNAL kerberos servers via
localhost:8899
so we need an
ssh
tunnel to forward connections to
localhost:8899
to the
actual FNAL kerberos servers. For example, run:
ssh -L 8899:krb-fnal-1.fnal.gov:88 -N your_cern_username@lxplus.cern.ch
This step has to be done each time you log in (and want to use FNAL machines).
Now make sure you have both CERN and FNAL kerberos tickets using
kinit
(replace
cern_principal
and
fnal_principal
with your actual usernames):
kinit cern_principal@CERN.CH
kinit fnal_principal@FNAL.GOV
You can check that both are active with
klist -l
which should show something like:
Principal name Cache name
-------------- ----------
fnal_principal@FNAL.GOV DIR::/tmp/krb5cc_123456/tktrsk9rM
cern_principal@CERN.CH DIR::/tmp/krb5cc_123456/tkt
The last step is to tell
ssh
to tunnel through
lxplus
to get to any FNAL machines. If you already have an
~/.ssh/config
that works for FNAL access when there's no firewall, you just need to add sections like this to
~/.ssh/config
(with the appropriate replacements):
Host *.fnal.gov
PreferredAuthentications gssapi-with-mic,gssapi,password
ProxyJump cern_principal@lxplus.cern.ch # Replace with your actual CERN username
User fnal_principal # Replace with your actual FNAL username
Host cdcvs.fnal.gov
PreferredAuthentications gssapi-with-mic,gssapi,password
ProxyJump cern_principal@lxplus.cern.ch # Replace with your actual CERN username
If you don't have an
~/.ssh/config
already, a complete example (again modified from
here
) looks like:
Host *.cern.ch *.fnal.gov
GSSAPIAuthentication yes
GSSAPIDelegateCredentials yes
PubkeyAuthentication yes
Host *.fnal.gov
PreferredAuthentications gssapi-with-mic,gssapi,password
ProxyJump cern_principal@lxplus.cern.ch
User fnal_principal
Host cdcvs.fnal.gov
PreferredAuthentications gssapi-with-mic,gssapi,password
ProxyJump cern_principal@lxplus.cern.ch
Host *.cern.ch
GSSAPITrustDNS no
Host *
IdentitiesOnly yes
ServerAliveInterval 5
ServerAliveCountMax 6
GSSAPIRenewalForcesRekey yes
GSSAPITrustDNS yes
ForwardX11 no
Protocol 2
Now that you have kerberos tickets for FNAL and CERN set up, along with the configuration to tunnel ssh connections through
lxplus
, you should be able to log in to any FNAL machine that you have access to (eg, the dune gpvms). If you can do that, then everything is set up correctly.
One last thing for pushing to git repositories is that the remote URL must be set correctly.
git remote -v
will show you the remote URL(s) for your repository. It should look like
ssh://p-PROJECTNAME@cdcvs.fnal.gov/cvs/projects/PROJECTNAME
(replace
PROJECTNAME
as appropriate). If not, you can change it with
git remote set-url origin ssh://p-PROJECTNAME@cdcvs.fnal.gov/cvs/projects/PROJECTNAME
.
--
PhilipAndrewRodrigues - 2018-03-13