[intrepid-notify] ACTION REQUIRED: Intrepid-fs0 cleanup

[Jini Ramprakash]

Dear Users,

Previously, we have not had to limit disk storage space or to require 
users to use the archive facility. Because /intrepid-fs0 is now 90% 
full, and we have projects that will be generating additional data soon, 
please conduct the disk space housekeeping below as soon as possible, 
though no later than August 15. Your timely assistance will prevent any 
need to suspend the scheduler.

Please note: We are requesting action from ALL users, regardless of how 
much data they currently store.

- If you have data you don't need, please delete it.
- If you have data you don't want to delete, but don't need to access 
immediately, please archive it and either have HPSS delete it after it 
moves it (-d option)--preferred because it frees space on the fly, or 
archive it and then delete it when it is moved. See the directions below 
for how to use HPSS and for URLs to related documentation.

Please check these three primary places for housekeeping opportunities:

- /intrepid-fs0/users/<username> (this is the most likely place)
- any shared project space (some projects have a central location where 
they share data)
- /intrepid-fso/gpfs1-migration/gpfs1 (This space was used when we moved 
over from the old file system. Please delete any data there that you are 
no longer using. If you are using the data, please begin migrating away 
from it. As we announced when we migrated, this space was intended to be 
temporary. We will send a separate email regarding the decommissioning 
of that space soon.

Our primary goal, as always, is to provide the resources our users need 
to complete their work. However, due to the storage space shortage, we 
ask that you make an honest and thorough assessment of your current 
storage requirements, and take steps, where possible, to delete or 
migrate data per above.

We appreciate your timely response to this request.

-------------------------------
For instructions on HPSS, go to: 
https://wiki.alcf.anl.gov/index.php/Using_the_High_Performance_Storage_System_HPSS

HPSS quick start:

To archive your data:

you should have access to HPSS, so to archive your data you can do 
something like this:

hsi -O <output.log> "lcd /intrepid-fs0/users/<username>; put -d -h -P -p 
-R -U scratch" & (This command line will archive, then delete, all files 
under /intrepid-fs0/users/<username>/scratch)

the format:

hsi -O <output.log> "lcd <source directory>; cd <HPSS directory>; 
<operation> <options>  <local disk path>" NOTE: Your default HPSS 
directory is /home/<username> Wild card characters permitted for <local 
disk path>

the options:
   -d delete the file after successfully moved to tape
   -h create a symlink in HPSS if a symlink exists on disk
   -P create intermediate subdirectories if they don't exist
   -p preserve timestamps
   -R recursively store directories
   -U Update option. Only copy file to HPSS if local time stamp is newer 
or HPSS file does not exist.

Your default HPSS path is /home/<username> as shown above. So, if 
/intrepid-fs0/users/<username> had two subdirectories foo and bar, in 
HPSS you would see /home/<username>/foo and /home/<username>/bar. If you 
wanted to archive to a subdirectory under /home/<username> you could 
include  "mkdir <subdirectory>; cd <subdirectory>" prior to issuing the put.

If the hsi command does not work for you, please send email 
support at alcf.anl.gov and ask for your access to be enabled.

Additional documentation can be found here: 
http://www.mgleicher.us/index.html/hsi/

As always, please feel free to contact us with any questions, comments 
or concerns.

Thank You,
The ALCF Support Team.