Reseach Cloud National Allocation Scheme (RC-NAS) Policy



Version

Version

Date

Author

Change

Pass 13

2016-03-05

Sebastian Barney

Proposed policy from WP8

0.1

2017-06-09

Paul Coddington

Changes to align with existing process

0.2

2017-07-06

Paul Coddington

Incorporating feedback from Nectar Allocation Committee and Core Services Control Group

1.0

2017-07-26

Paul Coddington

Incorporating changes requested by the PSC, which endorsed this policy. 


This document defines the Nectar Research Cloud National Allocation Scheme (RC-NAS) Policy. It describes the purpose of the scheme, who is eligible, resources that can be requested, how requests will be processed and other key information about the scheme.

0    Background

The WP8 Allocations project delivered a proposed policy for a National Allocation Scheme for the Nectar Research Cloud. This proposed policy was not approved by the PSC and it was thought it would take a lot of time and effort to implement, since it was significantly different to the existing process. It was suggested that a simpler interim policy and process, more closely aligned with the existing allocations process, should be documented and agreed. Future work could revisit the proposed policy, in the context of planning for the Australian Research Data Cloud proposed under the national research infrastructure roadmap. This document specifies the interim policy and process for the RC-NAS.

This document was produced by editing the proposed allocations policy document produced by WP8. A version of this document, tracking changes from the policy proposed by WP8, gives an indication of the differences between the current allocations process (documented here) and the process proposed by WP8.

 

1    Introduction

This policy is built on a set of principles recognising the funding programmes and agreements that have created and enabled the Nectar Research Cloud. 

The Australian Government Super Science Initiative, financed through the Education Investment Fund (EIF), and the National Collaborative Research Infrastructure Strategy (NCRIS) have provided funding to Nectar to: 

  • Contribute to the generation, utilisation and awareness of science and research knowledge through investing in research infrastructure to support innovation and provide access to the best facilities for the greatest number of researchers; 

  • Enhance research collaboration and research outcomes through seamless connectivity of research instruments, analysis systems and data resources; 

  • Support international research collaboration, and collaboration between industry and the research community. 


Through its funding of the Research Cloud Nodes, Nectar aims to:

  • Enhance research collaboration through the development of eResearch infrastructure capable of having a national impact and, through this, enhancing national research outcomes;

  • Deploy eResearch Infrastructure and services not otherwise available to publically funded researchers; and 

  • Extend the use of these eResearch capabilities to a wider cross-section of publically funded researchers more quickly than would otherwise occur. 


This policy aims to enable an on-demand feel for users requesting resources on the Research Cloud as a mechanism to minimise barriers to research. 

This policy recognises that resources on the Research Cloud are limited. To manage this challenge, this policy defines rules to promote fair access to and use of these resources. Under this policy resource allocations are time limited, but applicants will be able to request extensions to allocations where continued access is desired, or delay another allocation until the resource is required again. 

This policy aims to create a culture where resources are used when required and then returned. This approach will ensure the Research Cloud can positively impact the greatest amount of applicants. This policy aims to dissuade a culture sometimes seen, where people hold onto resources they do not need in case they need them again in the future. 


2    Eligibility requirements

Eligibility criteria are set on the applicant requesting resources, and the intended use of the resources. 

2.1    Applicant eligibility

An allocation can be requested by and awarded to one or more eligible researchers. Eligible researchers are defined as: 

Employees or students undertaking research at an Australian organisation that receives research funding and/or research infrastructure funds from the Australian federal government, and/or one or more Australian state/territory governments. This definition includes employees of the Nectar Research Cloud node operators. 

Note that an eligible researcher can be either the project owner (the allocation requestor) or the Chief Investigator specified in the allocation request, for example where the project owner is not a researcher but a member of IT or research support staff.

Approved allocations can be used by research partners of the requestor, including international and industry-based research collaborators. 

In practice, the eligibility requirement for applicants is enforced by:

  • Restricting allocation requests to people with an Australian Access Federation (AAF) identity;

  • Requiring users to agree to Terms of Use which specify the requirements;

  • The Nectar Allocation Committee can investigate further if they have reason to believe that the applicant is not an eligible researcher.

2.2    Intended use eligibility

Under the RC-NAS policy, a request for resources can be approved for any of the following purposes, in compliance with NCRIS guidelines:

  • To undertake and/or disseminate research;  

  • To provide services to eligible researchers to undertake and/or disseminate research; or 

  • As a platform to support training for researchers. 

Requests for resources for training and to provide services will be assessed on their impact to eligible researchers, although they can be open to other users as well. 

Local cloud resources allocated by Nodes outside of the RC-NAS can also be used for broader activities, including teaching.


3    Project trial allocations 

All eligible researchers get access to a project trial allocation when they first access the Research Cloud. The purpose of this allocation is to provide a frictionless introduction to the Research Cloud that enables researchers, students and support staff to understand the Research Cloud, and how it can be used to undertake research. This also provides an on-demand functionality for the Research Cloud.

Note that some institutions have requested that their researchers require institutional approval to be given access to project trial allocations, and this is expected to be implemented in future, with notification to their researchers on the Nectar user support site.

A project trial allocation will give an applicant access to limited resources. The technical description of resources provided as part of a project trial are detailed in Appendix 1. If an eligible researcher wants to continue using the cloud, they can submit an allocation request to access additional resources. An allocation request can extend a project trial allocation if more resources are required. 

Researchers are notified in advance of their project trial expiry, when they have either used 80% of their quota or have 1 month remaining before the end date of their allocation. Once the project trial quota or end date is reached, the project trial expiry process will begin unless a project allocation request has been submitted to extend the project trial, or the allocation request has been rejected. Once a project trial has expired, researchers cannot cannot create any new instances, although running instances will continue. One month after expiry, any instances still running are snapshotted and terminated. Three months after expiry, archived snapshots of instances are deleted. 

Eligible researchers are limited to a single project trial allocation, and these allocations cannot be pooled, or the project owner changed. Once a project trial has been extended with an allocation request the pooling and changing of project owner limitations no longer apply and the allocation is treated as an allocation request.


4    Allocation request process

Eligible researchers who want to access Research Cloud resources through the RC-NAS beyond a project trial will need to submit an allocation request. The allocation request process ensures fair access to resources by ensuring the requests are of an appropriate size and provisioned based on merit, to support nationally significant research. 

The allocation request process is started when an eligible researcher submits a request. When a request is submitted it will be assessed by the Nectar Allocation Committee (NAC). This assessment should be completed in 2-3 weeks of the request being submitted. 

If the assessment is successful the request will be  approved for provisioning under the RC-NAS. If the assessment is not successful the request will be rejected, in which case feedback should be provided to the applicant on the reasons for rejection. Alternatively, the request may be approved by a Node for provisioning under a local Node share, rather than the national share which is used for the RC-NAS. As part of this process larger requests may be rightsized. 

If the information provided in the allocation request is not sufficient to make an assessment, the assessor may request further information from the requestor, and ask them to resubmit a revised allocation request. 

When an allocation is approaching its expiry date, the project owners and members will be notified and asked if they wish to renew the allocation, following the Procedure for Renewal and Expiry of Project Allocations.

Once a provisioned allocation meets its expiry date and is not renewed, it will be decommissioned following the process specified in the Procedure for Renewal and Expiry of Project Allocations 

A submitted request can be updated and re-submitted. 

A provisioned request can be updated and re-submitted. In this scenario the provisioned resources will remain active. 

Rejected, expired and decommissioned requests can be updated and re-submitted. 

Project owners and their nominees should be notified each time their request changes status. 

Nodes can provision requests using their discretionary local share as they see fit. A project allocation can include a fraction of the allocation that is from the national share, and a fraction that is a local node allocation. This may be appropriate for large allocation requests that the node wishes to support.

The allocations assessment form on the Nectar dashboard allows the assessor to specify what percentage of the allocation is provided through national funding (the national share, i.e. the RC-NAS) or local funding (the Node’s local share), or a mix of the two (e.g. 25% national and 75% local). 


5.    Resources available for merit requests

The type of resources that can be requested under the RC-NAS are:  

  • Compute resources – including cores, RAM and instance storage;

  • Volume storage; 

  • Object storage. 


All allocations will have a maximum duration and a maximum allocation of cores, instances, volume storage and object storage. Allocations are currently not limited by core-hours but this should be implemented in future.

The size of the allocation may differ for a research request and a service request. A research request is submitted by an eligible researcher on behalf of themselves, a research group or a research collaboration. A service request is submitted to provide eResearch services to a community of researchers, research groups and/or research collaborations. An example of a service request is the Nectar Virtual Laboratories, who provide a service to the Australian research community. Note that this is currently not a formal categorisation that is explicitly specified in the allocation request.

Once an allocation has been provisioned it is possible to request an extension to the allocation before it expires, allowing continued usage if the extension request is approved. 

To ensure fair access, an project is limited to having at most 5% of any resource available as part of the RC-NAS – cores, RAM, volume storage and object storage. 


6    Nectar Allocation Committee

The Nectar Allocation Committee (NAC) exists to fairly review requests for eligibility for the RC-NAS and to approve the amount of resource allocated. 

The NAC decides whether an allocation should be approved as national or local, i.e., to be provisioned as part of the national share (which could be used on any Node of the cloud) or the local Node share (which can only be used at a specific Node).

The NAC will be comprised of members that are nominated by each Node. 

The chair of the NAC will be the Nectar Deputy Director, Research Platforms.  The chair is responsible for reviewing decisions to approve or reject a request for allocation to the RC-NAS, and the size of allocations, and to raise any issues of concern about specific allocations or the allocations process to the NAC or the PSC.

Members of the NAC will need to be on appropriate mailing lists (for notifications concerning allocation requests, or communications to the committee) and be given permissions to view and assess allocation requests in the Nectar Research Cloud dashboard. Hence any change in membership of the NAC should be communicated to the chair and Nectar Core Services by the relevant Node.

The NAC Terms of Reference (ToR) details the membership, structure and processes relevant to the NAC. Note that the NAC ToR is still to be developed.


7    Allocation assessment

All requests under the RC-NAS that meet the eligibility criteria in Section 2 will be assessed against a set of merit criteria to determine their suitability for approval under RC-NAS. 

7.1    RC-NAS Merit Criteria

A project allocation request can only be approved for the RC-NAS if the project is submitted by an eligible researcher and is :

Note that if a project meets the above criteria, it is not automatically entitled to a national allocation. In some cases it may be more suited to a local allocation.

7.2    Review process

Allocation requests should be reviewed and approved (or rejected) by a member of the NAC from a Node that has an association with the home institution of the Chief Investigator specified in the allocation request. Here “an association” means that the institution is a member or partner of the Node, or the Node has some agreement with the institution to provide cloud resources to that institution.

Allocations that request resources specific to a particular Node must be approved by a NAC member from that Node. For example, volume storage is a resource that must be requested at a specific Node (or Nodes) and provisioned at that Node, hence a member of the NAC from that Node must agree that the Node is able to provision the requested amount of volume storage. 

If resources (e.g. volume storage) are requested at multiple specified Nodes, the approver of the allocation request must confirm with a NAC member from each of these Nodes that they approve the provisioning of the resource allocation at their Node.


8    Rightsizing merit requests

Requests can be rightsized by the NAC reviewer(s) as part of the approval process. The purpose of rightsizing is to ensure that appropriate resources and timeframes are awarded as part of a request, and to ensure the best utilisation of the resources in line with the objectives of Nectar.  

Where the resourcing requirements are unclear, approvers should approve the minimum set of resources required to confirm the resource requirements. Once the resource requirements are known the researchers can submit a request of an appropriate size, using their experience as the justification for the resources requested. 

In general the principle should be to approve the minimum amount of resource that the applicant is expected to require, and only increase the allocation once there is a demonstrated need for more resource.

  

9    Duration of allocations

The maximum duration for approved allocations should be one year. Allocations can be approved for shorter time periods (e.g. 3 or 6 months). 

Nodes can offer longer allocation periods through their local allocation share.


10    Renewing and extending allocations

Once a resource request has been approved it is possible to apply for an extension to enable additional resources and/or continued access to the approved resources. 

Access to the resources made available through an approved request will continue while an extension request is considered. If an extension request is declined, access to the previously approved resources will continue until the allocation expiry date. 

It is also possible and encouraged to re-request resources after the original allocation has expired and a period of inactivity has passed. This allows other groups to get value out of these resources, allowing the Research Cloud to support more research and researchers.

 

11    Projects with multiple funding sources

Projects may have resources funded from multiple sources, including the RC-NAS. For example, a project may have compute resources funded through both RC-NAS and a Nectar node’s local discretionary share or locally funded resources, and have storage resources funded through the RDS programme. 

The allocations approval interface allows the assessor to specify that an allocation is approved with a mix of both national funding (i.e. through the RC-NAS) and local funding (through the Node’s local discretionary share or locally funded resources), e.g. 25% national and 75% local. 


12    Pooling resources

Approved resources can be pooled together into a single request or project, with the exception of project trial allocations. Examples of pooling include:

  • A group of researchers submitting a single request—the assessment of such requests will recognise the number of people who will use the resources.

  • An applicant who wants their resources added to an existing project so that they can leverage other resources that have been allocated. 


13    Change of project owner

There are a number of scenarios where there is a need to change the project owner of an allocation. For example, when a researcher changes research institutions, but will continue their research; or when a new service owner takes over the responsibility for an allocation. 

A change of project owner is allowed providing the new project owner meets the eligibility criteria for the type of request, with the exception of project trial allocations. 

A request to change a project owner can come from the Chief Investigator identified in the request, a project owner, any delegate authorised by the project owner, the project owners two-up manager, or the Deputy Vice Chancellor of the project owner’s institution. By default the project owner is the original applicant. 

Any resources allocated to the project should be transferred as per the original approval. 


14    Termination

Applicants or approved delegates can terminate resource allocations awarded as part of a request at any time. 

Project owners and their delegates should receive regular information on their usage of resources, and warnings indicating when access to resources are going to expire. When resources expire that are still in use, access to these resources should be restricted, and instructions sent on requesting an extension of access or saving any data before the resources are terminated. 

The schedule of termination notifications is listed in the Procedure for Renewal and Expiry of Project Allocations


15    Overallocation of resources

Experience has shown us that not all projects use all of their allocated resources all the time. To ensure resources are used as effectively as possible, some resources may be over-allocated. While there is a risk that some resources may be unavailable when a researcher tries to use them, in general this will allow the research cloud to be used more efficiently to support more research. 

The NAC are responsible for reviewing the total amount of resource usage in the Nectar cloud, and the usage at their Node, and managing the level of over allocation (by approving, rejecting or rightsizing allocations) based on the usage and availability of cloud resources. 

Nectar should provide appropriate allocation and usage information to the NAC to allow them to make these decisions on capacity management, however for some resources (including Node volume storage capacity and usage, and the number of free cores at the Node) it is expected that this information would be provided by the Node. 


Appendix 1. Project trial resources

A project trial expires when the quota of resources (core-hours) is exhausted, or the end date is reached, whichever occurs first.

Resource

Limit

Instances

2

Cores

2

Core-hours

 (6-months of 1-core always-on) 4,500

RAM (GB)

8

Volume Storage (GB)

0

Object Storage (GB)

10

End-date

12-months from first login