The Argon file structure

Your home directory on Argon is /Users/hawkid; you will start out here every time you log in. All users are allowed 1 TB of storage in their home directory.

If you need more storage space than that, there are three other options worth knowing about:

/localscratch/Users/hawkid
/nfsscratch
Large scale storage (LSS), /Shared

The first two are “scratch” space, meaning that files are subject to deletion 30 days after they were created. Obviously, don’t keep anything critical there, but if you’re, say, writing a bunch of temporary files with intermediate results and need someplace to stash them, that’s what the scratch space is for. For more on the difference between the two scratch systems, see here.

The large scale storage option is attractive for shared projects and sensitive data. To set up an LSS share, you would have to work with research-computing@uiowa.edu and these shares are not free (although they are pretty cheap). Typically, these shares are set up as part of a project or grant. For example, if you have an RA with Dr. Peabody, there might be an LSS share set up at /Shared/Peabody.

Transferring files (Windows)

To do any meaningful work on Argon, you will probably have to transfer files back and forth (pushing data sets and programs you’ve written out to Argon, pulling results from Argon to back to your machine, etc.). These are the instructions for Windows; Linux instructions are below.

The easiest way to transfer files on a Windows machine is to simply mount your home directory on Argon so that it appears on your local machine as a regular directory. Click on Computer > Map network drive, then enter \\data.hpc.uiowa.edu\argon_home as the Folder. Done!

Be aware when mapping a drive in this way that you will be opening and editing files on a Windows machine that must be readable on to the HPC Linux machines. In particular, Windows and Linux files have different line endings, and this will cause problems depending on what program you use to edit files. If at any point you see error messages like bin/bash^M: bad interpreter: No such file or directory, or anything with a ^M in it, this is likely what has happened.

To fix this, if you’re using Notepad++, go to Settings > Preferences > New Document > Format (Line ending) and select “Unix (LF)”. To modify an existing file, go to Edit > EOL Conversion and select “Unix (LF)”.

It is possible to set this up on a Linux/Mac machine, although you will have to have IT set it up for you as it requires administrative privileges on your machine.

Transferring files (Linux/Mac: scp)

The basic way to transfer files from a Linux/Mac terminal is scp:

$

scp -P 40 file.txt argon.hpc.uiowa.edu:

This will copy the file file.txt from your current working directory to your home directory on Argon. Note the -P 40 option to specify port 40; this is not necessary on-campus (but doesn’t hurt either). To copy a directory, you need to specify the -r option:

$

scp -r -P 40 mydir argon.hpc.uiowa.edu:

Transferring files (Linux/Mac: rsync)

Another useful option to know about for directories is rsync. The advantage of rsync is that it only copies files that have changed between the two directories. So for example, suppose you copy a directory containing large amounts of data to Argon, analyze the data using Argon, then want to copy the directory back. This will be very inefficient, since the data is unchanged – you really just want to copy the files that have changed. You could do this manually, but rsync automates the process:

$

rsync -avz -e "ssh -p 40" --delete mydir/ argon.hpc.uiowa.edu:mydir/

This will synchronize the mydir directory on Argon with the one on your local machine. Note that the above code is a little aggressive; the --delete option means that if a file on mydir on Argon does not exist on your local mydir, it will be deleted from Argon (in other words, the two directories will look exactly the same once the above command is finished). You can read the full rsync documentation to find out more about all the options.

To synchronize in the opposite direction (i.e., make your local mydir folder look exactly like the one on Argon), simply reverse the order of the last two arguments:

$

rsync -avz -e "ssh -p 40" --delete argon.hpc.uiowa.edu:mydir/ mydir/

Again, be careful while doing this, as you could inadvertently delete files in your local directory.

Transferring files using the cloud (e.g., GitHub)

Both of the Linux/Mac options given above are not as useful as they used to be, now that the Argon cluster requires dual-factor authentication. Another option that I have found useful is to use cloud storage for file transfer using, say, GitHub or Amazon Web Services: from your local machine, push to the cloud, then pull down from argon. If you know how git works, then this is self-explanatory; if you don’t, well, that’s a separate tutorial.