This is the Nikhef group hints page on working cross-platform


Mac vs PC part 1, XKCD Confused looking Linus Mac vs PC part 2, XKCD

Preamble

Some users prefer Unix, some prefer windows, some prefer Mac. The key thing is to think about which is best for you, and then maximise your productivity by using the many available tools to work cross-platform when you need to.

  • X-servers, Virtual machines, SLC5, dual-booting, VNC and NoX are all possible alternatives.
  • Virtual Machines, SLC5 and dual-booting laptops let you install LHCb software locally, but then you must maintain the software distribution yourself.
  • CernVM is a virtual machine with lhcb software pre-installed, but with a minimal SLC5 kernel. You will find you are missing several unix commands and that it is in most cases unsatisfactory as a full desktop environment.
  • Connecting up your SLC5 or VM to afs allows you to use the afs-installed software locally, but it is difficult to obtain a good performance from afs without a lot of tinkering.
  • VNC and NoX are both remote-desktop-equivalents for unix-based platforms

Local Gateway

Nikhef has it's own local SLC5 gateway on the network running LHCb software and with afs-support. The gateway is login.nikhef.nl

Unix host

Most modern unix (linux) flavours are capable of natively running LHCb software, but depending on the exact kernel, you may need to re-compile the entire LHCb software stack. This is not quick and not simple, so in case you don't want to give your machine over to SLC5, then there are other (better) options.

All of the options which work on a non-unix host are probably also going to work for you. Additionally there is the option of using the changeroot system...

changeroot environment

A changeroot system is using the chroot(2) system call to basically lie to programs about the true root of the file system. This can be used to make LHCb software believe it is running on a real SLC5 system which is copied to some directory on a machine running some other flavour of Linux (the idea is used heavily by the group of the PI in Heidelberg (esp. Alexey Zhelezov) to drive the local computing cluster, the method and most of the scripts presented here are derived from that work - anything that does not work is probably my (Manuel Schiller) fault or due to broken LHCb login scripts/expired certificates/firewalls/... ;). The typical setup works like this (see below for detailed installation instructions):

  • The directory /slc5 contains an installation of SLC5 and whatever LHCb software the user desires.
  • Certain directories on that machine are crossmounted into the /slc5 directory hierarchy to have the available in the chroot environment, e.g. /home is made available also as /slc5/home. This is done for /home, /tmp, /proc, /sys, /dev, /afs and /var/run/gdm3 so that the programs running in the chroot environment think they are running on a real system.
  • Users and group memberships in the chroot environment are taken from the host installation with a startup script.
  • For laptops/personal workstations, if a user wishes to work with LHCb software in the chroot environment, (s)he executes a script ("slc5") in a shell which starts a shell in the chroot environment. Since the chroot system call requires root privileges, the user has to be in /etc/sudoers. To permit a user to become root, add a line user ALL=(ALL) ALL to /etc/sudoers. If you just want to give certain users access to the chroot binary without opening sudo to everyone, the line yourlogin ALL=(root) NOPASSWD:/usr/sbin/chroot should do the trick (the NOPASSWD instructs sudo not to ask for a password for this command). More fine-grained access restrictions are possible, see sudoers(5).
  • As an alternative, for login machines or batch nodes with many users, one can e.g. start an ssh server in the chroot to allow user login or teach the batch system to execute jobs in the chroot environment. To avoid trouble, the ssh host keys from the host installation should be copied to the chroot environment, and the sshd server running in the chroot env. cannot share a port with other running sshd daemon processes running on the system (e.g. you could have the slc5 chroot sshd running on port 22 (ssh default port), an sshd in an old slc4 chroot on port 24, a testing SLC5 chroot sshd on some other port and the sshd of the host OS installation on port 26 for admin logins - change the port numbers to whatever convention is suitable for your site...).

Installation is pretty simple (tested on a Debian squeeze system, but probably also fine for Ubuntu and other distros)

  • Get the tarballs from /data/bfys/manuelsr/chroot-slc5 at Nikhef
  • Untar them on your machine in /:
    xz -dx chroot-slc5.tar.xz | tar -C/ -xf -
    xz -dc chroot-slc5-startup-scripts.tar.xz | tar -C/ -xf -
  • A first SLC6 chroot tarball is available from /data/bfys/manuelsr/chroot-slc6/chroot-slc6.tar.xz. A tarball with startup scripts has not been prepared yet, but using the SLC5 ones and trivial search and replace should get you going. Feel free to give feedback and suggestions you may have to Manuel.

This will install the chroot environment under /slc5, the startup script /etc/init.d/mountslc5 (with symlinks in /etc/rc?.d/[SK][0-9][0-9]mountslc5 to start/stop the crossmounts during system boot/shutdown) and /usr/local/bin/slc5 to drop users into an SLC5 environment. Make sure that the directories /afs and /var/run/gdm3 exist, even if you are not using openafs or gdm3, or change /etc/init.d/mountslc5 accordingly. On distributions other than Ubuntu/Debian, /etc/init.d/mountslc5 might require some trivial hacks to integrate with the flavour of init scripts used. The SLC5 installation in the chroot environment has a few packages from the default installation removed (e.g. there is no point in running an X server in the chroot), and some others added - feel free to tailor this to your needs once the install is complete...

Once the installation has been completed, there are three things a user/admin can do:

  • Work in the chroot environment: slc5 drops you into a shell similar to what you would get on lxplus...
  • Update the slc5 installation (security updates/...): become root, then do
    slc5
    yum update/upgrade (as needed)
  • Install LHCb software: become root, then do
    slc5
    source /local/env.sh
    cd /local; python ./install_project.py -b DaVinci vXrY

A few pointers in case of hiccups:

  • If host name resolution in the SLC5 chroot does not work, but works fine in the host system, copy /etc/resolv.conf to /slc5/etc/resolv.conf. On systems with an IP address which changes frequently (e.g. a laptop), it is useful to make the dhcp client execute this command after each change of IP address - since this depends a lot on the distro/dhcp client you are using, I cannot provide a solution here that will work everywhere.
  • Login scripts for exotic shells are less well tested; bash and zsh work fine, csh/tcsh are less well tested, but also seem to work fine, ksh (Korn shell) currently does not work at all. If you are using the Korn shell (ksh/pdksh/mksh/...), I'd suggest you switch to either bash or zsh, they are both quite similar to ksh, and they are actively maintained and used by a large user community.
  • (to be added to if more issues become known)

Non-Unix Host

There are five basic ways to use Linux/UNIX commands on your windows system.

To natively run LHCb software

  • Dual Boot into Linux and your favourite OS (windows/Mac)
  • Run a virtual machine, either CernVM or Ubuntu

To remotely run LHCb software

  • Use a secure shell connection, to a UNIX machine once running an X server
  • Use a remote connection, to a UNIX machine once running a NoX/VNC server +ssh port forwarding
  • Install a compiled set of binaries for using UNIX commands directly under windows

- Dual Booting

Can be problematic, especially for very new or very very old laptops, but if it works it is one of the best options. However you may find it difficult to share files between OS-es, and find it annoying that you need to hibernate/resume to run your favourite programs. You also can't cut/paste between dual boots, of course.

- Virtual Machines

ThomasRuf keeps a fantastic description of virtual machines here and here

  • Virtual machines have almost all the advantages of dual booting, without the drawbacks of having to restart to switch, and usually simplify cut/pasting and file sharing.
  • CernVM is a virtual machine with lhcb software pre-installed, but with a minimal SLC5 kernel. You will find you are missing several unix commands and that it is in most cases unsatisfactory as a full desktop environment.

Windows host

Windows comes without any nice X-server, and without any nice command-line interface. If you want to avoid having to pay for your X-server on your private machine, here are some alternatives.

- Unix compiled for windows

Cygwin is a compiled set of UNIX binaries and scripts which runs well on the Win32 kernel (no evidence of compatibility with 64 bit core as of yet). It will be slower than using UNIX directly i.e. through dual booting, but allows you to use UNIX and windows at the same time.

Cygwin is FREE!

The Cygwin installer is tricky to use, you need around 1.5GB free to install and 1GB for temporary files downloaded during installation. Cygwin, (which is a different compilation to Cygwin-X), also comes with an X-server.

To install correctly, with an X-server and all features:

  • Dowload the installer for Cygwin
  • Run the installer to "download without installing", this will help you later. The files created can be deleted later.
  • Select a directory with a lot of space for these files ~1GB is enough
  • DON'T install everything, it doesn't work
  • DON'T use the default installation, only very basic things are added
  • Select from the list of components those that you'd like to get,
    • do this by changing the "skip" or "default" to "full"
    • make sure you select "shells", "graphics" and "X11".
    • Remember what you selected! It will be needed later.
  • Switch the view to the "Not Required, Skip, Not installed" view using the button at the top right
    • Select any component with X11 in the name that you missed.
  • DON'T go back to the package view, as this will reset everything to default
  • Next then should download the required files
  • Run the installer again, with "install from local directory"
  • Select only the components you downloaded earlier
  • Select all the components you downloaded earlier
  • Switch the view to the "Not Required, Skip, Not installed" view using the button at the top right
    • Select everything, as this is a list of the downloaded files you have missed
  • DON'T go back to the package view, as this will reset everything to default
  • Really you should install to your C-drive as long as you have enough space.
  • Just click OK, continue, whatever on any and all errors that appear during installation
  • Run the Cygwin program once it has installed. This will initially establish local home directories

To find your X-server:

  • Either: Use the command startx in the Cygwin window (this will open a new x terminal and x-server)
  • OR: find the batch file startxwin.bat in C:\Cygwin\usr\X11R6\bin and make a shortcut to it
    • this will open just one (bash) window

- SSH with an x-server

SSH stands for secure shell. A secure shell connection allows you to log into a Unix system (as if you were sitting at the desk) and use the resources as per usual.

X11R6 is the current UNIX standard graphics interface, it is the protocol used to give you graphics such as windows, buttons, etc. X11 port forwarding is the name for the forwarding method/process by which graphics can be packaged and displayed over a connection, i.e. for ssh. To do this you need a compatible server at each end, Linux, MacOS, or windows running an X-server.

The best X server is eXceed, but this is expensive. Alternatively use Cygwin as above.

Once you have such a server, you should use X11 port forwarding, either by ssh -X in Cygwin/UNIX or by changing your options in the ssh client you have chosen.

puTTy, Cygwin, SSH, provide ssh clients.

- VNC - remote desktop system

VNC stands for Virtual Network Computing and is a simple way of exporting your desktop for remote access anywhere with an internet connection.

If you're sick and tired of none-graphical interfaces and you want all the functionality of sitting in front of the machine itself, or even if you have a home PC you'd like to log into remotely from a unix machine Virtual Network Computing VNC is the way to go.

VNC is FREE!!

You need:

  • A VNC client (not mandatory) installed on your local system
  • Java installed on your local system, and plugged into your browser of choice
  • An SSH client (mandatory)

Then:

  • Run a vncserver on the HOST machine
    • UNIX -> open a terminal and type > vncserver :XX where XX is the display number you'd like to use (pick something between 0 and 99)
    • Windows -> Start the VNC server that you downloaded, should automatically start display 0 or 1 (port 5900 or 5901).

  • Set up an SSH tunnel on the client machine
    • UNIX -> open a terminal and type > ssh -X -L 59##:127.0.0.1:59XX username@hostname.com where XX is the display number of the host server you set up (i.e. 5900+the number, 5901 for display :1) and ## is the port number you'd like to forward to (pick something between 0 and 99)
    • Windows -> Cygwin, open a terminal and type > ssh -X -L 59##:127.0.0.1:59XX username@hostname.com where XX is the display number of the host server you set up (i.e. 5900+the number, 5901 for display :1) and ## is the port number you'd like to forward to (pick something between 0 and 99)
    • Windows -> PuTTy, start a putty session.
      • Use the hostname of the computer at which you established the host,
      • In the SSH->Tunnels configure a new tunnel, Source Port: 59##, Destination port: 127.0.0.1:59XX, where XX is the display number of the host server you set up (i.e. 5900+the number, 5901 for display :1) and ## is the port number you'd like to forward to (pick something between 0 and 99)
      • Save the session as something memorable in the Session configuration
  • Run a viewer on the client machine
    • Anywhere -> Most machines are not equipped with a viewer, so use a web-interface. Open a browser with Java and go to http://hostname:58XX/ where XX is the display number of the host server you set up, or http://ipaddress:58XX/ where ip address is the ip address of the host machine. This will only work for networks and machines without a million layers of protection. If you're going from windows server to linux client, this is the way to go.
    • UNIX: TightVNC java -> use the command > java VncViewer HOST localhost PORT 59##
    • Windows -> Run your client viewer to connect to localhost:## where ## is the port that you forwarded to in the previous step

Want your usual desktop?

  • Windows server-> automatically gives you control of the active desktop
  • UNIX/linux server -> edit the ~/.vnc/xstartup script (remove the comments from the two obvious places)

For more information on making the connection more secure, see here

Useful programs for windows/linux

  • WinSCP Secure Copying, nice interface.
  • GIMP, (java) image package, particularly useful for converting TO postscript/eps for windows
  • Emacs for windows
  • Ghost View for windows, particularly useful for viewing postscript
  • MiTeX latex compiler for windows
  • Eclipse fantastic all-singin-all-dancing IDE and TeX editor (java) for any platform
  • JaxoDraw (java) draw Feynman diagrams and export as postscript
  • TightVNC (java) Remote Desktop across platforms
  • NoX x-forwarding-free server


-- RobLambert - 24-Oct-2011

Topic attachments
I Attachment History Action Size Date Who Comment
PNGpng supported_features.png r1 manage 18.3 K 2011-10-25 - 10:00 RobLambert XKCD 1
JPEGjpg ubuntutux.jpg r1 manage 6.4 K 2011-10-25 - 09:59 RobLambert Confused looking linus
PNGpng xkcd_mvp.png r1 manage 13.3 K 2011-10-25 - 10:00 RobLambert XKCD 2
Edit | Attach | Watch | Print version | History: r10 < r9 < r8 < r7 < r6 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r10 - 2013-04-20 - ManuelSchiller
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback