Network Attached Storage (NAS) is provided to Broadies by BITS (Broad Institute IT Services) in several forms:
- The BITStore (a platform that sells storage services to groups for scientific work)
- Home Directories (UNIX & Windows)
- Temporary, high-performance storage at /broad/hptmp (see below)
Storage Options
Various storage technologies are supported at Broad, each with their own performance characteristics.
An overview of storage options and disaster protection can be found on the BITStore page.
Temporary, communal, high-throughput storage: /broad/hptmp
The file system /broad/hptmp/ is a storage location provided by BITS for Broad computational researchers that need a potentially large temporary space for running analyses across the UGER cluster. Users and groups who need to re-examine or keep output written to this temporary space should copy data files from /broad/hptmp to a permanent location soon after running jobs have completed.
This space is not intended for long-term storage of data: there are no recovery services available for data in this space. Users can point their applications to this file system, which "lives" on higher performance (Isilon) storage. Used in this way, high-performance disk should always be available as a resource for the entire Broad community.
In this spirit, users have a soft quota limit of 3TB in this location with the ability to scale up to 10TB on a temporary basis. There is a grace period that is activated once the soft quota has been reached. Upon expiration of the grace period, if the user's space usage is still above 3T, it must be brought down to less than 3T in order for the user to be able to write to /broad/hptmp.
IMPORTANT: Files in /broad/hptmp that are idle for more than two weeks will be deleted. Please note: if the total available space in this file system falls below 5%, file retention time is automatically reduced to 10 days and files 11-14 days old will be deleted.
We recognize that there may be an occasional need to maintain large intermediate files on /broad/hptmp/ for further analysis, but all files on /broad/hptmp should be removed in a timely manner. If groups are crunched by the 10-14 day file expiration, they should email help for assistance.
To improve our ability to maintain this file system, we ask that you not write data directly into the top level of /broad/hptmp. Please create a subdirectory and write data there instead.
Assessing your present files and storage needs
Platform and program members at Broad are often unaware that they are storing older, unaccessed files on expensive storage hardware. Sometimes, members are hesitant to delete data if they are unsure about its value. Some directories are good candidates for relocation to archive storage while other data should be deleted. Discussions about data and hardware are essential for determining the best plan moving forward.
BITS is able to provide consultative services to assist in data management. If you or someone on your team would like to discuss your current storage footprint and how you may be able to save money please submit a request here: https://broad.service-now.com/sp?id=sc_cat_item&sys_id=b0502aa5db5a63000f1b6033ca9619e3&sysparm_category=d28665f14ff662005c1423d18110c7e4
How to request storage
To request a new file share please visit the BITS ServiceNow Portal at: https://broad.service-now.com/sp. Under the Service Catalog link you'll see "Storage" as an option. Just complete the Storage Request form to request a new file share.
For all additional needs please visit broad.io/help to submit a help request.
Broadies Login
