You find here solutions for problems with exceeded quota on the HOME and WORK file system.

If your project needs larger quotas, please contact your consultant to discuss your needs and how we can help you.

Please visit File Systems for the properties of the file systems HOME, WORK, and PERM.

Please periodically check whether your scientific results data stored on the file systems should be periodically transferred to the user's home institution. To maintain good I/O performance for all user jobs, the usage of the shared file systems for all resources needs to be limited.

Quota defaults

The default values for user quota are reasonable for average use cases. We are aware that certain projects need larger quota for their workflows. If your project needs larger quotas, please contact your consultant to discuss your needs and how we can help you.

File system quotas are enforced

• on HOME and WORK with respect to group ownerships
• on PERM with respect to user ownerships on Lise in Berlin

^✦ there is user ownership only

Quota is available on the three file systems HOME, WORK and PERM. On each file system we distinguish

• quota for blocks, that is the disk space your files allocate, and
• quota for inodes, that is the number of files and directories.

Each quota consists of two numbers, the soft limit and the hard limit.

• Once your usage achieves the soft limit, the grace peroid of 2 weeks starts to count. By the end of the grace period you are not able to write files. As soon as you drop down below your soft limit the grace period is reseted.
• Once your usage achieves the hard limit, you are not able to write files.

Check your quota

Check your disk quota limits on HLRN with using the command hlrnquota show-quota. Once If your account user ID is a member of a UNIX group, you find quota informations additionally for all one or more project groups you will also get quota information for these project groups. Each of your files belongs to your account or to one of your UNIX groups. The output of hlrnquota and directories is owned by your user ID (user ownership) and by exactly one of the group IDs you are a member in (group ownership). The data base for the output of show-quota is updated twice per hour.

blogin4> hlrnquotablogin7:~ $ show-quota
Home
                           Blocks used   Soft limit   Hard limit     Inodes used   Soft limit   Hard limit    
  myaccount  (users)          89.45 GiB       40 GiB      100 GiB         187.3 k            -            -    
  prj00012  (projects)        68.03 GiB       40 GiB      100 GiB         659.6 k            -            -    

Work
                           Blocks used   Soft limit   Hard limit     Inodes used   Soft limit   Hard limit    
  myaccount  (users)          1.202 TiB        3 TiB       30 TiB          1.74 k        250 k        1.5 M    
  prj00012  (projects)        2.512 TiB       12 TiB       30 TiB               1          1 M          6 M    

File system quotas at HLRN are realized

• on HOME and WORK with respect to the UNIX groups
• on PERM with respect to the user account.

Quota on HOME and WORK

For each stored file on the file systems HOME and WORK the UNIX group of this file controls the attribution to the quota to this UNIX group. For quota, the directory of a file (/scratch/usr/${USER} or /scratch/projects/<projectID>) does not matter.

Each user account is a member in one or more UNIX groups. You can check your membership in UNIX groups with the command groups.

...

...

If your HOME quota is exceeded, we recommend

1. to check whether you can either delete files or move files from HOME to WORK (no backup!) or to PERM
2. to contact your consultant, explain your storage needs, and ask for a higher quota limits.

If your WORK quota is exceeded, we recommend

1. to check whether you can delete files or move files from WORK to PERM
2. to change the group ownership of files to "move" their attribution from the quota of your primary group (user ID) to the quota of another group (e.g. prj00012)
3. to contact your consultant, explain your storage needs, and ask for higher quota limits.

Quota and unix groups

For each file and directory stored on the HOME or WORK file systems:

The group ownership affects the disk quota of that particular group – no matter where the file or directory itself is located within file system.

Sometimes this can make it hard to track down the disk quota when files or directories of a given group ownership are distributed over different user and/or project directories.

Each user ID is member of one or more groups. Group memberships can be checked with the groups command.

blogin4:~ $ groups myaccount
myaccount prj00012

Once When, for example, you decide to set change the UNIX group ownership of a file to prj00012 for a file, the used quota for the UNIX group prj00012 includes this file size, the size of this file will affect the disk quota of group prj00012.

blogin4:~ $ chgrp prj00012 somefile.txt
blogin4>blogin4:~ $ ls -l somefile.txt
-rw-------  1 myaccount prj00012 237271040 Jul  3  2020 somefile.txt

To change all small number of files in a directory recursively, use "chgrp -R".  To update many files, especially on the Lustre filesystems, you should use "lfs find" instead:

blogin4:~ $ lfs find somedirectory | xargs chgrp --recursiveprj00012 prj00012
somedirectoryblogin4:~ blogin4>$ ls -ld somedirectory
drwx------  1 myaccount  prj00012      4096 Jul  3  2020 somedirectory

General hints to solve quota issues

As described in File Systems the default quota limits are reasonable for average use cases.
For projects requiring higher quota limits we kindly ask you to contact your consultant to discuss your needs and how we can help you.

Please keep in mind that the overall performance of a global parallel file system depends on the free capacity (better performance with high free capacity).

HOME quota exceeded

The HOME file system (HOME=/home/${USER}) is intended to be used for the following data:

• configuration files
• source code and executables

You might experience an exceeded HOME quota due to unintended usage of the HOME file system, e.g.

• large files
• parallel input/output for your jobs

If your HOME quota is exceeded, we recommend

1. to check whether you can either delete files or move files from HOME to WORK (no backup!) or to PERM
2. to contact your consultant, explain your storage needs, and ask for a higher quota limits.

WORK quota exceeded

The WORK file system (/scratch) is intended to be used for the following data:

• large files
• parallel input/output for your jobs

If your WORK quota is exceeded, we recommend

...

If your file or directory names contain spaces, you will need to use "lfs find -0 ... | xargs -0 ..." instead.

Access for group members

Once a file (or a complete directory) has the group ownership, the file might be accessed by all users in the group.

blogin4:~ $ ls -l somefile.txt
-rw-------  1 myaccount prj00012 237271040 Jul  3  2020 somefile.txt
blogin4:~ $ chmod g+r somefile.txt
blogin4:~ $ ls -l somefile.txt
-rw-r-----  1 myaccount prj00012 237271040 Jul  3  2020 somefile.txt

blogin4:~ $ ls -ld somedirectory
drwx------  1 myaccount  prj00012      4096 Jul  3  2020 somedirectory
blogin4:~ $ lfs find somedirectory |xargs chmod g+rX
blogin4:~ $ ls -ld somedirectory
drwxr-x---  1 myaccount  prj00012      4096 Jul  3  2020 somedirectory

Quota exceeded

Disk space, Blocks

Once you experience too many Blocks you should check for disk usage in a directory.

blogin4:~ $ du -sh /home/myaccount/somedirectory
1.4G /home/myaccount/somedirectory
blogin4:~ $ du -sh /home/myaccount/somedirectory/*
227M /home/myaccount/somedirectory/subdir1
627M /home/myaccount/somedirectory/subdir2
...

Number of files, Inodes

Once you experience too many Inodes you should count the number of files in a directory.

blogin4:~ $ lfs find /home/myaccount/somedirectory | wc -l
201

Related articles